Analytics work needs to be discoverable. Let’s talk how to get there [and how a doc workspace like Hyperquery can help]. — Back when I was a data scientist, I spent a substantial amount of time doing product analytics work — opportunity sizing, experiment deep dives, ad-hoc checks. But although I worked across a wide range of tools — Jupyter/Python, tidyverse, superset, internal tools, even Java UDFs — the bulk of this…

Introducing a better SQL-writing workflow, and your first glimpse into Hyperquery. — Last year, we launched a data discovery tool called Dataframe. We wanted to solve the following pain point: it’s hard to find and get context on tables in your warehouse. But by working with customers over the last year we’ve learned that standalone data discovery, while effective at controlling data…

And why your data science team needs it. — 🗃️ First, how Airbnb’s data discovery tool changed my life. In my career, I’ve been fortunate enough to work on some fun problems: I studied the mathematics of rivers during my Ph.D. at MIT, worked on uplift models and open-sourced pylift at Wayfair, and implemented novel homepage targeting models & CUPED improvements at Airbnb. But in all of this work…

Why we should move to a world where analyses are the default first step — It’s been nearly 2 years since Seth Rosen’s trashboard tweet, and yet the trashboard epidemic is still quite real. We still encounter companies that are curiously happy with having thousands of dashboards, their analysts left to spelunk deep caverns of ad hoc requests with naught but an IDE. There are…

How an over-emphasis on technical work has been disastrous for analytics — Feedback loops are powerful things. Positive feedback can produce highly leveraged effects, while negative feedback can subdue change. Moreover, these patterns can be extremely resistant to disruption. If you don’t address the root driving mechanism, feedback loops will labor on. After all, you can’t throw water on the sun.

Why I hate the IDE — at least, for analytics work. — Article originally published in Win With Data — subscribe for regular updates! Before I condemn the IDE, I want to say that I do look back fondly on my days writing in SQL IDEs. The process was spare and brutally technical in a way that nourished my obsession with Unix…

Great product teams are user-centric and proactive. Great data teams would benefit from doing the same. — Last year, Emilie Schario and Taylor Murphy proposed this wonderful idea of “running your data team like a product team”. The key premise of the article was this: product teams have a lot of great practices that data teams would benefit from adopting. But somewhere along the way, we lost…

Before your next SQL dumpster dive: ask why and document everything — When I was a data scientist at Airbnb, the most common question I got was: “Do you know how many people clicked on this button?” But the curious thing about this question: it was rarely the question that the stakeholder actually wanted answered. Ad hoc questions like this are an…

Relegating all data consumption to self-service is not as effective as we’d hoped — There was a period in the 80s-90s in the U.S. where the world collectively dreamt of building increasingly complex car dashboards. …