Keep Moving

Chris Hillman — Tue, 23 Jun 2026 08:00:00 +1000

Some days I open my laptop and by 5pm I genuinely cannot tell you what I did.

Not because it was complicated. Not because there were emergencies. The stand-up happened. A few Teams messages were sent. A ticket was groomed. A document was “reviewed”. A meeting was attended where everyone agreed something was important and then the meeting ended and nothing changed.

And then somehow it was evening and the pipeline I meant to fix was exactly as broken as it was in the morning.

I used to think this was a time management problem. I’ve since come to believe it’s a physics problem.

An object at rest

Actual productive work — the kind that moves a platform forward — probably occupies about two or three hours of a given workday. Maybe four if you’re lucky. The rest is coordination, context-switching, Teams, and the ritual of almost starting.

I’ve seen senior engineers arrive late and leave early and outship everyone else on the team. I’ve seen people in meetings from 9 to 5 who are technically “at work” but haven’t touched a pipeline in three weeks. Hours in office — or in Jira — are a spectacularly poor proxy for output.

The hard part isn’t the work itself. Once you’re in it — once the editor is open, the query is running, the dbt model is taking shape — it flows. The painful part is the gap between intending to start and actually starting. That gap has a gravitational pull all its own.

I have a version of this routine I’m not proud of: open laptop, check email, check Teams, open the ticket I meant to work on, re-read my own notes from last week, decide I need more context, open Confluence, fall into a 20-minute rabbit hole of documentation I wrote six months ago, circle back to Teams, check if anyone replied to that thread, think about getting coffee, get coffee, and then, somewhere around 11am, finally open the editor and actually write the thing that takes forty-five minutes.

The work was never the problem. It was everything I let stand between me and starting.

This isn’t a character flaw. It’s inertia. Physics applies to engineers too.

The only way to build a platform

There’s a way I’ve come to think about what separates data platforms that actually mature from the ones that are permanently “almost there”.

It’s not the architecture. It’s not the tooling. It’s not the quality of the backlog or the rigour of the sprint ceremonies or the neatness of the Confluence hierarchy.

It’s whether the team is moving forward — every day, in some small way — or whether it’s standing still.

That sounds obvious until you try to do it. Because there are a hundred forces, every single day, designed to stop you from moving.

Some of them are internal. The engineer who won’t write a single line until the data model is “finalised”. The team that spends three sprints documenting the current state before touching anything. The migration project that requires a complete source system audit before the first table lands in Snowflake. All reasonable-sounding activities. All forms of not moving.

Some of them are external. That’s where it gets more interesting.

Cover fire

The data tooling industry has discovered something effective: if you can keep teams busy reacting to you, they don’t have time to ship.

Not consciously, probably. But the effect is the same.

Joel Spolsky wrote about this pattern more than twenty years ago in Fire and Motion. In infantry terms: you fire at the enemy not because you expect to hit much, but because it forces them to keep their heads down while you advance. In software terms, the big players generate a constant barrage of new technologies and standards so that everyone else burns their days keeping up instead of building. He was writing about Microsoft in 2002. The names have changed. The strategy hasn’t.

Every six months there is a new architectural paradigm your team is supposed to evaluate. The lakehouse. The semantic layer. Data contracts. The medallion architecture (deprecated). The medallion architecture (rehabilitated). Streaming-first. The modern data stack (thriving). The modern data stack (dead). AI-native pipelines. Agents that write your dbt models for you.

The vendor ecosystem has learned to generate enough noise that a conscientious engineering team can spend most of its time just keeping up: reading the announcements, watching the conference talks, debating whether to adopt the new pattern, setting up a proof-of-concept, discovering the PoC is more complex than advertised, abandoning it, and then starting the cycle again when the next announcement drops.

Meanwhile your users still can’t get a reliable enrolment count. The Oracle migration is in month fourteen. The dashboard that takes eleven seconds to load still takes eleven seconds to load.

The Snowflake vs Databricks wars, dbt Fusion vs dbt Core, Iceberg vs Delta vs everything else: some of it is genuinely important and you should stay informed. But a lot of it is just noise that benefits the people generating it, not the people consuming it. While your team is deep in an evaluation of a new ingestion paradigm, the basic pipelines you already have aren’t getting more reliable. The stakeholders who’ve been asking for a new data source for four months are still waiting.

They don’t care which open table format you’re using. They care whether their data is there in the morning.

When a sales engineer from a major vendor asks whether your platform supports the latest hot acronym — and they will, and it will be in a room full of your stakeholders — that question is doing work. It’s designed to make you feel behind. To make you wonder if you should be rebuilding rather than shipping. To redirect your energy toward something that benefits them.

The right response is rarely to immediately build the thing they’re asking about. The right response is usually to keep moving on the things that actually matter to the people you serve.

What moving forward looks like

I want to be specific about this, because “keep moving” is easy to say and genuinely hard to operationalise.

It doesn’t mean rushing. It doesn’t mean skipping the thinking or writing code you’ll hate in six months. It means that every sprint, every week, every day — something is better than it was. A pipeline is more reliable. A model is documented. A test passes that didn’t pass before. A stakeholder gets a data source they couldn’t access last month.

Small, consistent, compounding. The platform that looks dramatically better in twelve months is almost always the one that had someone quietly improving things every single day, not the one that attempted a big-bang rewrite.

On the personal side: the most useful thing I’ve found is to remove the decision of whether to start. If you have to decide to begin every morning, you’re giving inertia a fighting chance. The editor should be open before you check Teams. The model you’re working on should be the first tab, not the fifth. Pair programming works partly because it eliminates optionality. You can’t not start when someone is sitting there waiting.

On the team side: the most dangerous status update is “working on it”. It sounds like movement. It often isn’t. The rituals that actually correlate with forward progress are the boring ones: small PRs merged frequently, tickets moving from in-progress to done in days not weeks, users getting things they asked for.

On the strategic side: have an honest answer for what you’re choosing not to evaluate right now, and why. “We’re not looking at that this quarter because we’re migrating the Oracle GL tables and that’s more valuable” is a complete sentence. You don’t need to defend it at length.

The platform that keeps moving wins

I don’t think data engineering is particularly hard compared to what else people do with computers. The craft is learnable. The patterns are well documented. The tooling is genuinely better than it was five years ago.

What’s hard is sustaining forward motion through the accumulated weight of meetings and tool evaluations and perfectly reasonable reasons not to ship today.

The teams I’ve seen build platforms that people actually trust and use are almost never the ones with the most sophisticated architecture. They’re the ones that kept moving — through organisational changes, through vendor noise, through the days when it felt like nothing was getting done.

There’s a cost to this approach, and it’s worth naming. If you treat most of the noise as noise, occasionally you’ll be wrong. Something you dismissed will turn out to be the real thing, and you’ll arrive eighteen months after the early adopters, reading their write-ups instead of writing your own. That’s the trade, and I’d take it. Being late to a paradigm is recoverable. A platform nobody trusts is not.

The job, at every level, is to move forward a little every day. Fix the pipeline. Write the test. Ship the model. Even when it’s small. Even when it’s imperfect.

The platform gets better a little at a time, or not at all.