Opinion

The bad loop ruining analytics

How an over-emphasis on technical work has been disastrous for analytics

Image by author.

Feedback loops are powerful things. Positive feedback can produce highly leveraged effects, while negative feedback can subdue change. Moreover, these patterns can be extremely resistant to disruption. If you don’t address the root driving mechanism, feedback loops will labor on. After all, you can’t throw water on the sun.

Why all the talk of feedback loops? Well, I suspect there’s a single negative feedback loop responsible for all things broken in analytics. And like other feedback loops, it’s hard to disrupt.

I imagine you’ve all felt the effects of this — our impact has never felt quite as high as it should be. We are rarely viewed as thought partners. We do excellent technical work, but it hardly draws a glance from the people that matter. Sure, we have a seat at the table when strategic decisions are being made, but only as the resident SQL technician. And it’s not due to lack of trying. We build self-service systems, we hold office hours, we build elaborate courses to level up our peers, but these initiatives never effect lasting change.

So what is the loop? It’s quite simple: everything we do overemphasizes technical work, setting the expectation that we only do technical work. Our best efforts to extend our impact are thereby undermined. In what follows, I’ll discuss this loop, convince you of its pervasiveness, and finally, share why I believe tooling is paramount in solving this problem.

As always, if you’re looking for the bias, I’m the Chief Product Officer at Hyperquery, where we’re making a bet on how better tooling can fix this sort of thing, but I’ll stay measured — I promise.

The core bad feedback loop

At first glance, there appear to be quite a few different feedback loops trapping us in this no man’s land. I’ll go through a few to establish some resonance with you, but I’ll later argue that they all stem from the same root feedback loop. But first, let’s go through these common patterns of brokenness:

• “Analytics is about speed”. This has been a common narrative over the past few years around analytics. But if you’re mainly valued by the speed by which you do your work, you’re going to have to get faster, which means less time for interpretation, considering business impact. The scope of your work narrows, pushing you to a reactive supporting role.
• Dashboard overuse. With every new dashboard we make, we are, on the one hand, enabling greater self-service capabilities for the rest of our business. But if we rely too heavily on dashboards as our main medium of communication, we’re perpetuating the myth that data is obvious — that our interpretation isn’t required. Obviously, we need to make dashboards, but we default to them far too often. We instinctively point our stakeholders to dashboards, rather than understanding their questions. They become empowered in the wrong way, and their increasingly narrow understanding of our value pigeonholes us into a data curation role.
• We jump to querying too fast. We’ve all done this: you’re in a meeting with an exec and, in the desire to provide quick value, you go full Minority Report, live coding. There’s a time and place for this, but we are far too eager to go there when we should thinking instead — synthesizing and drawing on learnings from the wealth of past experiments and deep dives we’ve already conducted. We fail to provide leveraged value, and instead seek approval for our flashy (unrivaled, sure!) technical prowess. And this quietly becomes the value others expect of us.
• Our tools are overly technical. Our tools are the nail in the coffin. They are made for technical work, and provide these capabilities at the expense of simple, clear delivery. Technical capacity is prioritized above all else. We like this, of course. It appeals to the aspiring hacker in us all. But your SQL queries and notebooks convey precisely the kind of reductive representation of your value add that you should be trying to avoid — they are code-first, so you are seen as code-first. We try to break out of this by writing up our work in more shareable tools, but our adherence to our own rules is often low. After all, why use lot tool when few tool do trick?

And the common thread starts to emerge. Industry narratives, decades of BI culture, our tools, even our own hubris all push us toward a world in which our value is relegated to pulling data. And the fundamental feedback loop condemning us on these fronts: we present and value ourselves for our technical skill, and so others value us for our technical skill.

Image by author.

And that’s it. This is the pathology that oppresses us. It’s quite simple: we want impact, yet we chase excellency in what amounts to a trade skill — our SQL chops. We hunger for influence outside of data, but our eyes are fixed only on the data. This is likely why we still haven’t proven out the value of analytics executives. Or why you still see analytics directors writing SQL for other executives. Or why our jobs are reactive, and why we can’t seem to break out of the cycle no matter what we do.

Technical work, of course, is a necessary part of analytics. You can’t share an insight without pulling and analyzing data. I’m not saying we should stop — that’d obviously be impossible. But we tend to over-index so heavily on the technical aspects of our work that we forget that analytics is not a purely technical domain. Analytics is about providing value through data. The “through data” part is certainly technical. But “providing value” is not. And the latter is where truly leveraged impact can come from.

The tooling problem: you can’t become a Youtube star without Youtube.

All that said, I want to draw attention to an undervalued lever in this causal calculus: our tooling. I don’t generally believe tools are critically important. People and process are the lifeblood of any successful data initiative. And the first fundamental requirement to break out of our technical trap is a mindset shift.

But sometimes tools are critical. You can’t become a Youtube star without Youtube, after all. And it’s hard to ignore that our impact is hamstrung by the tools available to us. Our tools — the IDE, the Jupyter notebook, BI tools — are largely technical (surprise, surprise). And just like us, our tools focus on creation, all the while keeping us trapped in our bad loops.

My main thesis: the right tool can break the bad loops. For example, the right tool could break the loops I laid out above:

• Break the “speed” loop: Our tools shouldn’t empower us to pull and analyze data faster. They should empower us to add context to our work and deliver knowledge rather than data just as easily and quickly.
• Break the dashboard overuse loop: Our tools should push us to share visualizations that can sit alongside context, rather than pushing us to create more dashboards.
• Push for alignment, not dumpster diving: Nudge us towards alignment and impact, not querying first. Text should always be first, because business objectives should always be first. SQL or Python can follow thereafter.
• Is fundamentally not creation-first, but impact-first. We need to stay focused on the core: delivery of words, interpretation. And so the fundamental primitive should be text and an ecosystem around it.

We need a tool that pushes us beyond technical work — a tool premised on the belief that our value comes not from the code we push, but from the insights we craft. We’re having a Henry Ford moment, and we need to stop yearning for faster horses. We’ll ultimately need a novel interface that lets us easily craft analyses that elevate us, not more tools that funnel us deeper down the technical rabbit-hole. While I can’t say for certain what it’ll look like, I have my suspicions, and certainly times are ripe for change.

Final comment

Reactivity, help desk analytics, neglected counsel — these problems point to a systemic problem: we are stuck in a negative feedback loop. And no amount of water will put out the fire. It’s time we actively fight our tendency to get nerd-sniped and seriously reconsider our tooling while we’re at it.