There’s a specific kind of silence in data engineering that I’ve learned to fear.

Not the silence of a system that’s working well. Not the comfortable quiet of a team in flow. I mean the silence of a project that’s been running for three weeks and you still can’t point to a single visible thing it has produced. The kind of silence where, if your manager stopped you in the hallway and asked “how’s that migration going?”, you’d say “fine” because saying anything more accurate would require explaining things you haven’t fully articulated yet — even to yourself.

I watched a talented engineer do this once. Let’s call her Mia. She’d been handed a significant refactor: six months of accumulated technical debt in a set of dbt models that sat at the centre of our revenue reporting. The kind of work that is genuinely unglamorous, invisible by nature, and deeply important. She approached it the way a lot of good engineers approach hard problems — she went quiet and started digging.

For three weeks, she showed up, she worked hard, and she said almost nothing. Her pull requests were sparse. Her standups were “still investigating.” Her Teams messages were infrequent. She wasn’t slacking off. She was grinding through some of the most complex lineage work I’d ever seen someone untangle.

Then a finance stakeholder messaged me — not her, me. Someone had noticed a spike in our snowflake credit usage. They were nervous. They’d been burned by silent changes before.

I didn’t have an answer. Mia had one, but nobody knew to ask her.

I went to her and said, “walk me through what you’ve been doing.” What she showed me was extraordinary. She’d traced dependency chains through twelve models. She’d found three bugs that had been silently wrong for months. She’d drafted a migration plan that would cut query time in half. It was genuinely impressive work — and it had been completely invisible for twenty-one days.

That’s the moment I understood something that Jeff Atwood wrote in 2008, in a post called Don’t Go Dark, that I wish someone had made me read before I managed my first data team. Atwood’s essay was aimed at software engineers. It quoted a Microsoft engineering rule that said, in essence: three weeks is going dark. The rule wasn’t about laziness. It wasn’t about bad engineers. It was about a failure mode so common in technical work that it needed a name.

Mia wasn’t lazy. She was going dark.



What Atwood Actually Meant

Atwood’s post is short — shorter than most people expect from the weight it carries. He builds on two sources: a software engineering principle from Jim McCarthy’s Dynamics of Software Development, and an observation from open-source contributor Ben Collins-Sussman. Together they identify a pattern that every engineering lead will recognise the moment they read it.

The pattern is this: programmers don’t want to show work in progress. They want to disappear into the problem, solve it completely, and emerge holding something finished. Collins-Sussman pinned it to fear — not arrogance — arguing that most developers treat coding as an arena for personal heroics rather than an inherently social activity. They’ll share code once it’s polished. They’ll share failures once they’re resolved. But the messy in-between, the half-formed thinking, the models with todos and the branches with four commits and no description? That stays private.

McCarthy’s rule was direct: manage your tasks in short enough increments that you always have a visible deliverable at the end of each one. Three weeks without something to show is too long. The early warning of slippage — one day, discovered this week — is worth ten times more than six months of slippage discovered at deadline.

Atwood’s additional observation is the one that has aged best: agile development structures made going dark mechanically difficult. If you’re running sprints, you’re producing something reviewable every two weeks regardless. The iteration boundary forces surfacing. Joel Spolsky, Atwood’s co-founder at Stack Overflow and another writer in this intellectual lineage, made the same argument about daily builds a few years earlier: a team that builds its software every day has, at minimum, a daily signal that things are either working or not. The build is the forcing function. It converts invisible progress into visible evidence.

Open source projects, Atwood observed, face the highest risk of going dark — because there’s no manager, no sprint ceremony, no daily build process unless someone chooses to implement one. The discipline has to be entirely internal. The only currency that matters on an open source project is visible, reviewable work.

Data engineering, as a discipline, has no such forcing function. Our work doesn’t end at a sprint boundary with a demo. Our pipelines run on schedules that have nothing to do with how long the underlying thinking took. Our models transform quietly, in the background, producing outputs that are visible only when something breaks — or worse, when something breaks subtly and doesn’t look broken at all.

That’s the gap Atwood’s advice never closed. He fixed going dark for software teams. Nobody translated it for us.



Why Data Engineering Goes Dark More Than Anyone Else

There’s a structural reason data work is so prone to invisibility,.

When a front-end engineer ships a feature, there’s an artifact: a button, a page, a flow you can click through. When a backend engineer merges a PR, there’s a diff: before and after, visible in the tool, reviewable by anyone. The work and the evidence of work are coupled. They arrive together.

Data work decouples them. Benn Stancil, writing about the data industry, put it well when he described the fuzzy boundary between production and everything else in data teams — there’s often no clean staging/prod line, no clear “shipped” moment, no obvious artifact to point to. A dbt model runs. A dashboard refreshes. A pipeline completes. Did anything change? Did anything improve? Is this better than it was last week? The answers require knowing the context that lives in the engineer’s head, which is precisely the context that isn’t written down.

This gets worse the longer the project. Migrations are the canonical example. A warehouse migration — say, moving from Redshift to Snowflake, or rebuilding a legacy reporting layer in dbt — might take three months of continuous work before a stakeholder sees a single dashboard change. During those three months, the data engineer is doing real, difficult, valuable work: tracing lineage, resolving naming conflicts, negotiating data contracts with upstream producers, writing tests that have never existed before. All of it is invisible. All of it looks like nothing to anyone watching from the outside.

There’s also the problem that Locally Optimistic describes as the distinction between linear and circular projects. Linear projects have a known path and a clear endpoint. Circular projects — most of the genuinely interesting data work — have outcomes that depend heavily on what you discover along the way. The best-laid migration plan doesn’t survive contact with a production schema that hasn’t been properly documented since 2019. Exploration takes time that doesn’t map cleanly to sprint cards or progress percentages.

When you combine those two things — structural invisibility and genuinely unpredictable progress — you get a discipline that almost naturally produces going-dark behaviour, even among engineers who are working diligently and in good faith. Mia wasn’t hiding. The nature of her work was hiding it for her.



The IC Chapter: Staying Visible Without Performing Busyness

Staying visible doesn’t mean performing busyness. It doesn’t mean posting standup updates about tasks you haven’t started. It doesn’t mean narrating every dead end like it’s a breakthrough. That’s noise, and experienced leads learn to tune it out.

What it does mean is producing artifacts — real, durable, reviewable traces of your thinking — on a cadence that keeps your work legible to the people around you. The goal is to make silence meaningful. When there’s nothing from you for three days, the people you work with should know that’s fine because you’ve pre-declared it and pre-committed to what you’ll surface when you come up for air. When you’re grinding through something genuinely hard, the work itself should leave evidence that can be reviewed asynchronously, without requiring a meeting to explain it.

Draft PRs as the lowest-friction visibility tool you already have.

Open a pull request on day one of a significant piece of work. Not when it’s ready for review — when you have a scaffold and an intent. GitHub’s draft PR feature exists exactly for this: your code can’t be merged, reviewers aren’t auto-requested, and everyone in the team can see that the work exists, what it’s targeting, and roughly how it’s going. Update it with a comment as you progress. “Blocked on upstream source delay — will resume Thursday.” “Discovered three extra joins needed — revised estimate to end of week.” “Models passing, running full test suite now.”

This pattern turns the PR into the canonical artifact of the work — a living document that replaces scattered messages, status check-ins, and “hey, how’s that thing going?” conversations. The PR description, written well, tells the story of what changed, why it changed, and how it was tested. Six months from now, when a model breaks at 2am and the engineer who built it is on leave, that description is the thing that saves the on-call.

Commit messages as team communication.

There’s a widely-cited piece by Chris Beams on how to write a good git commit message that’s been floating around engineering circles since 2014. The core argument is simple: the subject line tells you what changed; the body tells you why. A commit message that says fix bug costs the next engineer ten minutes of archaeology. A commit message that says Fix null handling in fct_orders when currency column is missing and then explains that a vendor renamed a field and the defensive .get() was silently returning zero for three months — that commit message is worth something.

The Conventional Commits specification formalises this further: feat:, fix:, refactor:, docs:, chore:, each with optional scope. It sounds bureaucratic until you try to understand a six-month-old change at 9pm during an incident, and someone had the discipline to write refactor(revenue): consolidate gross margin calculation into shared macro in the subject line. Then you understand immediately.

dbt documentation as a visibility discipline.

Every model in a dbt project can carry descriptions at the model level and at the column level, written in the same schema.yml file that defines your tests. Most teams don’t write them. Or they write them at launch and never update them. Or they live in a separate Confluence page that nobody visits.

Here’s the thing: schema.yml descriptions review in the same PR as the code they describe. They can’t silently drift from the implementation because they’re changed in the same commit. A model description that says “Revenue aggregated by market, excluding internal transfers — see ADR-004 for the exclusion logic” is a piece of team communication that persists across every engineer who will ever touch that model.

Go further. Use dbt Exposures to document which dashboards and downstream artifacts depend on your models. A handful of YAML lines, committing which report or ML model consumes each table, converts “will this refactor break anything?” from a Slack archaeology project into something you can run a query against. When a finance stakeholder asks if their dashboard will be affected by a model change, you can show them the dependency chain. That’s a conversation that previously required guesswork. Now it requires a terminal command.

Architecture Decision Records for the decisions people will question.

ADRs — short markdown files committed to docs/adr/ in your repository — document the decisions that will otherwise live only in one engineer’s memory. Why did you choose Snowflake over BigQuery? Why does the revenue model exclude returns that arrived after 30 days? Why is this model incremental rather than table? These questions come up six months later, in a meeting, when the person who made the decision has moved on or forgotten the reasoning. An ADR is three paragraphs and a status field. It takes fifteen minutes to write. It has saved me more than one painful meeting where the alternative was reconstructing a decision from Slack history.

Async standups that actually work.

The daily standup ritual made sense when software teams sat together and needed a quick synchronising pulse. For data teams — often distributed across timezones, frequently doing work that looks the same from the outside regardless of whether it’s going well or terribly — the fifteen-minute morning video call has become a source of collective fiction. Everyone says “still on X, should be done soon” and nobody learns anything.

Async standup tools like Geekbot (for Slack) or equivalent bots for Teams send each person the standup questions by DM on a schedule and post responses to a shared channel. The format that consistently performs best isn’t the classic three questions — it’s a single question: “What would you like the team to know today?” That question has enough latitude to surface a blocker, a discovery, a risk, or a win. It invites signal rather than mandating a ritual. Responses posted to a shared channel are searchable, linkable, and skimmable — a week of them gives a manager more information than five daily standups would.

The written standup format also decouples availability from communication. A data engineer in Melbourne doesn’t need to be online at the same time as her lead in London for him to understand what’s happening. The post is there when he arrives. She’s not interrupted in her maker’s block to attend a meeting she’ll spend eight minutes waiting through before she has sixty seconds to talk.

One discipline worth enforcing: when you name a blocker in a written standup, link to it. Don’t write “blocked on upstream data delay” — write “blocked on upstream raw_orders freshness, opened Issue #247 to track.” The standup post becomes navigable rather than descriptive. Anyone who wants to understand the blocker can follow the link. Anyone who just needs the pulse can read the post and move on.

Data diffs in CI — making the invisible visible before it ships.

The traditional CI check for a dbt project tells you whether models compile and whether your schema-level tests pass. Unique keys, not-null constraints, accepted values — these are important. They catch structural problems before they reach production.

What they don’t catch is when a model that was returning 65.00 starts returning 65.10, or when a currency denomination silently shifts from whole dollars to cents because an upstream source changed its representation without changing its schema. Those changes are invisible to schema tests. They’re invisible to the log output. They’re invisible right up until a finance stakeholder notices that last month’s revenue looks 100× higher than expected in a board deck.

Data diff tooling — whether through Datafold integrated into GitHub CI, or the open-source audit_helper package in dbt, or a homegrown comparison query — runs value-level comparisons between the current state of a model and a reference point. Merge a PR that refactors how gross margin is calculated, and the CI comment shows you a row-level diff of what changed. Every affected row. Every affected column. Before the code reaches production.

This changes the nature of what a PR review means. Instead of reviewing whether the SQL is logically correct — a genuinely hard thing to assess from reading code alone — reviewers can look at the data that would result. A refactor that should be semantically identical shows zero diff. One that inadvertently changes three percent of records in a specific market shows exactly which records and by how much. The artifact created by data diff in CI is one of the most powerful anti-going-dark tools available to a data team, because it makes the consequence of code changes legible to anyone who can read a table — not just the engineers who wrote the SQL.

Async documentation that travels with the work.

One pattern that consistently separates data teams that are legible from those that aren’t: Loom or short screen recordings attached to PR descriptions on significant changes. Not for every fix. Not as a replacement for the written description. But for a model redesign, a migration milestone, or a new data product, a two-minute screen recording walking through the dbt DAG and explaining the decisions is worth more than three paragraphs of prose that nobody will read.

The video doesn’t have to be polished. It has to exist. It converts work that is legible only to the person who did it into work that is legible to a finance analyst, a product manager, or a new data engineer joining in six months. It replaces the meeting where you’d have explained this anyway, except now it’s available asynchronously, can be rewatched, and is linked from the artifact it describes.



The Distributed Team Problem

Remote and distributed data teams face a compounded version of the going-dark problem. The structural invisibility of data work — no natural demo format, no clean shipped/not-shipped boundary — gets worse when your team spans timezones and your primary communication channel is a text-based messaging app with a scroll-back limit.

The teams that navigate this best tend to share a principle: optimize for knowledge retrieval, not knowledge transfer. The goal isn’t to ensure that information passes from person A to person B in a meeting. The goal is to ensure that the information exists in a form that person B can find when they need it, regardless of whether person A is online.

GitLab, operating as a fully remote company since its founding, formalised this as handbook-first communication: any decision, policy, or significant piece of information gets documented before it gets communicated or implemented. Not as bureaucracy, but because documentation creates the retrievable artifact. A verbal decision made in a meeting exists only in the attendees’ memories. A written decision linked from a PR exists for the lifetime of the repository.

For data teams, this translates practically into a few habits. Meeting decisions get a one-paragraph summary posted to a Teams thread, linked from the relevant GitHub issue. A verbal conversation about model design gets a brief written follow-up: “Following on from our chat — going with incremental strategy for fct_orders because full refresh at scale was timing out in staging. ADR drafted at docs/adr/0012-incremental-fct-orders.md.” The conversation happened. Now it has an artifact that travels.

Long-running projects — migrations, platform changes, major refactors — benefit from a written brief at the start that defines the scope, the success criteria, and the milestone cadence. Not a Jira epic with fifty tasks. A two-page document that answers: what are we doing, why are we doing it, how will we know when we’re done, and what are the milestones where we’ll regroup. The brief is the thing you hand to a new stakeholder who asks what’s happening with the platform migration. The brief is the thing you update when the scope changes. The brief is what prevents the three-week silence.

Locally Optimistic describes a useful scripting pattern for circular projects — the kind where the outcomes depend heavily on what you discover along the way. Rather than promising a deliverable, you promise a process: “We will investigate X, Y, and Z over the next two weeks. Those might not give us a conclusive answer, but we’ll regroup to discuss the findings and determine next steps.” That framing is honest. It sets appropriate expectations. And critically, it includes a defined moment of resurfacing — the regroup — which prevents the silence from becoming open-ended.

Julia Evans has written persuasively about the value of maintaining a running document of your own accomplishments — not as a vanity project but as a memory aid. The problem she identifies is real: at review time, you’ve forgotten half of what you did in the last six months, and your manager has forgotten more than that. A brag document is a log of things that mattered: bugs found before they became incidents, migrations completed, models refactored, stakeholders unblocked, junior engineers mentored.

This isn’t about inflating your achievements. It’s about making them visible in a context where the work itself produces no natural evidence trail.

Equally useful: weekly notes. A short message to your team lead, every Friday, that says “this week I did X, I’m picking up Y on Monday, and I want to flag Z as a potential blocker.” It doesn’t have to be long. It doesn’t have to be polished. It just has to exist. Will Larson calls this the drip — communication on cadence, regardless of whether there’s exciting news. The drip is what makes silence informative. When you’ve been sending a Friday note every week for three months and then you stop, the absence is a signal. When you’ve never sent one, silence tells nobody anything.

Tanya Reilly’s glue trap — for senior ICs especially.

If you’re a staff or principal data engineer, you’re probably doing a lot of work that doesn’t look like work from a promotion perspective: reviewing designs, unblocking others, noticing what’s slipping, maintaining relationships with stakeholders. Reilly calls this glue work — the coordination and communication and knowledge-transfer that holds projects together. It’s real work. It’s often the most valuable work on the team. And it’s completely invisible without a deliberate effort to create artifacts.

Design proposals, written-up meeting decisions, documented onboarding pathways, group emails that summarise a decision made in a verbal conversation — these are the artifacts that make glue work visible. Without them, you’re building organisational infrastructure that has no evidence it exists.



The Fear Dimension: What’s Really Happening When Engineers Go Dark

Here’s something I think we don’t talk about honestly enough.

Most going-dark behaviour isn’t strategic. It isn’t laziness. It’s fear — specifically, the fear that surfacing in-progress work will reveal that you don’t have it as together as you implied you did.

Collins-Sussman put this precisely in the piece Atwood quoted: developers don’t want their peers to see mistakes or failures. They want to present themselves as infallible. They’re comfortable sharing code once it’s polished. The messy middle, the exploratory branches, the models with big question marks in the comments — that stays private.

This fear is rational in the short term. Showing a half-finished thing invites feedback that can derail you. Showing a broken thing invites questions you can’t answer yet. Staying silent avoids both. The problem is that in the medium term, silence accumulates into something much worse: an engineer who surfaces three weeks into a problem with nothing to show, facing a stakeholder conversation where the first question is “why didn’t you just tell us?”

Paloma Medina’s BICEPS framework identifies six core needs that, when threatened, produce withdrawal: Belonging, Improvement, Choice, Equality, Predictability, Significance. Lara Hogan, who has done more than anyone I know of to translate this framework into engineering management practice, makes the observation that going dark is often a flight response to one of these needs being threatened. A project pivot that makes the last three months of work feel wasted threatens Significance — and the engineer who feels that their work no longer matters is likely to disengage before they articulate why. An abrupt change in priorities threatens Predictability and Choice simultaneously. A team reorganisation threatens Belonging.

The leader’s job, when they notice going-dark behaviour, isn’t to increase accountability pressure. It’s to identify which need has been threatened and address it directly. Fournier and others recommend this as the actual content of the 1:1 when something feels off: not “how’s the project going?” but “how are you doing?” The project status comes second. The human state comes first.

Hogan’s Red/Yellow/Green check-in is a lightweight mechanism for this. At the start of a 1:1, an engineer can say “red” without having to explain why — just to signal that something is wrong. The permission to say “red” without an explanation is itself the thing that makes the saying possible. When explanation is required before disclosure, disclosure gets deferred until the explanation is formed — and by then, the problem has grown. Hogan’s observation: only having to say red, and not having to explain the why, is huge. The absence of the explanation requirement is the mechanism. It lowers the cost of disclosure below the threshold where people start editing themselves.

Amy Edmondson’s research on psychological safety is directly relevant here. Her finding, counterintuitively, was that better hospital units didn’t make fewer medication errors — they reported more of them. Her reframe: the better units weren’t more error-prone, they were more willing to surface problems early. The error rate looked higher because the environment was safe enough that people actually said what was happening.

Google’s Project Aristotle, which studied 180 teams over several years, found the same pattern at scale: the single strongest predictor of team performance was whether team members felt safe to take interpersonal risks. Safe to say “I’m stuck.” Safe to say “I found a bug.” Safe to say “I don’t know how long this will take.”

The data engineering implication is uncomfortable but important: if your team never has visible incidents, never surfaces broken assumptions, never flags a model that’s been silently wrong — that’s probably not a sign of excellence. It’s probably a sign that the environment doesn’t feel safe enough to tell the truth. The team that surfaces the most broken pipelines, the most failed tests, the most “we got this wrong” disclosures might be your best team. They’re the ones telling you what’s actually happening.

The “I’ll tell them when it’s fixed” trap is the going-dark failure mode in its purest form. An engineer discovers a problem — a data quality issue, a budget overrun, a model that produces subtly wrong results. Rather than surface it, they decide to fix it first. Each day that passes raises the psychological cost of disclosure. The problem grows. The fix gets harder. Eventually, either the engineer surfaces in a state of exhaustion with a fix and hopes nobody asks hard questions, or a stakeholder finds the problem externally — at which point the disclosure isn’t just about the bug. It’s about the three weeks of silence.

Fixing the number is not the same as fixing what the number did to people who trusted it.



The Lead Chapter: Building Conditions Where Your Team Doesn’t Go Dark

If you lead a data team, the going-dark problem is partly yours to own — not because your engineers are your responsibility in a parental sense, but because the conditions that make going dark feel safe are conditions you set.

Camille Fournier’s framing, in The Manager’s Path, is that 1:1s are like oil changes: if you skip them, plan to get stranded. The 1:1 is the primary diagnostic for whether someone is going dark, and it works only if it creates genuine psychological safety — which means it has to be something other than a status update. If every 1:1 is “what are you working on this week?”, you’ll get accurate project status and zero signal about whether someone is actually stuck, scared, or burning out.

Lara Hogan recommends opening the relationship with simple questions: what makes you grumpy? How will I know when you’re grumpy? What do you need from me when you’re struggling? These aren’t soft questions — they’re diagnostic instruments. They establish that the 1:1 is a place where the real state of things can be named.

Her Red/Yellow/Green check-in is a lightweight version of the same tool. At the start of a 1:1, an engineer can say “red” without having to explain why — just to signal that something is wrong. The permission to say “red” without an explanation is itself the thing that makes the saying possible. When explanation is required before disclosure, disclosure gets deferred until the explanation is formed — and by then, the problem has grown.

Will Larson frames visibility for leads as a mechanical practice rather than an art form. The goal isn’t to inspire your team to communicate — it’s to build systems that make communication the path of least resistance. A weekly update template, committed to as a team norm. An async standup channel where the format is so lightweight that posting takes two minutes. A retrospective cadence where surfacing what went wrong is the expected behaviour, not the exception.

Warning signs to watch for.

Going dark rarely announces itself. The early signals are easy to miss.

Standups that shift from specific to vague — “still on X” across multiple days with no detail — are an indicator. PRs that open in draft and stay there, accumulating commits but never progressing to review. Estimates that consistently slip by a day or two, week after week, without the engineer naming the slippage explicitly. A previously engaged person who starts attending meetings on camera-off, contributing less, leaving threads unresponded to.

The most useful signal is often external. When a stakeholder or adjacent team member asks you what’s happening with a project before the engineer working on it has raised anything — that’s the canary. The information is flowing around the engineer rather than through them. That’s a conversation to have in the next 1:1, and it’s worth naming directly: “I heard from finance about the revenue models. Tell me where things actually are.”

Deep work is not going dark — and the distinction matters.

There’s a real risk of over-correcting. Data engineering requires extended, cognitively-demanding focus. Paul Graham’s framing of the maker’s schedule versus the manager’s schedule is directly applicable: a data engineer tracing lineage through a DAG, rebuilding a model from first principles, or debugging an intermittent pipeline failure needs hours of uninterrupted context, not a constant stream of check-in messages.

Deep work and going dark are not the same thing. The difference is legibility.

Deep work has a pre-declared output and timebox. Going dark does not. Deep work is legible at a coarse grain — “I’m heads-down on X until Thursday, update Friday” — even when it’s illegible at a fine grain. Going dark is illegible at every grain.

The mechanism that makes the distinction: the cadence contract. Before going heads-down, the engineer declares: here’s what I’m working on, here’s when you’ll hear from me, here’s what would make me surface early. “If I’m stuck for more than a day, I’ll tell you. If I’m on track, Friday update. Ping me in #urgent if there’s a fire.” With that contract in place, silence during the week means “on track.” Without it, silence means nothing — or worse, triggers anxiety that prompts the exact interruptions that fragment deep work.

The goal is to make silence meaningful. That requires prior investment in communication, not more communication during the work itself.



The Visible Data Team Playbook

There are a handful of practices that the best data teams I’ve seen use consistently. None of them are revolutionary. Most of them are embarrassingly simple. They work because they’re habitual, not because they’re clever.

Weekly insight posts.

A practice described from data teams at companies like Netlify involves a dedicated Teams channel where each analyst posts one insight per week — not a status update, not a task list, but one thing they found interesting in the data. The format is designed to answer five questions: what am I looking at, why should you care, what caught my eye, where can you learn more, and who do you ask if you have questions.

The explicit benefit isn’t just visibility — it’s demonstrating the range of what the data team does. Stakeholders who only see the data team when something breaks, or when they submit a request, have no model of what the team actually knows. A weekly insight post gives them one. Over time, it becomes the evidence base for why the team should be resourced, why data work is complex, why the business should care about data quality. It’s not reporting. It’s positioning.

Parity dashboards during migrations.

One of the most effective anti-going-dark tools for long migrations is a parity dashboard: a side-by-side view of critical metrics computed in the old and new systems simultaneously. Before you’ve finished the migration, you build the parallel calculation and make it visible. Divergence between the two numbers becomes the artifact of progress — something demoable, something reviewable, something that gives a stakeholder a concrete thing to look at even when the underlying models are still being rebuilt.

The parity dashboard also builds confidence in a way that purely technical communication can’t. When a finance stakeholder sees that their ARR number matches between the legacy and new system for three weeks running, they start to trust the migration in a way that no amount of “we’ve tested this thoroughly” can produce. Evidence beats assertion.

Incident communication as a team norm.

The instinct to fix a data quality issue quietly and move on is almost universal among data engineers. The impulse is understandable — nobody wants to be the person who caused the problem, and surfacing it draws attention to the failure. But the silent fix has a consistent failure mode: it gets discovered later, retrospectively, in a context where the silence becomes the story rather than the fix.

Monte Carlo’s research showed that the data teams with the highest stakeholder trust tend to over-communicate around incidents — sending post-mortems after significant issues, flagging discoveries before fixes are complete, maintaining a clear status trail from “investigating” to “resolved.” What one team described as transformative wasn’t their mean time to resolution. It was the shift from reactive firefighting to proactive communication that demonstrated awareness and control. The communication is the product, as much as the fix.

A simple practice: any time you discover a data quality issue, the first act is naming it in writing — a Teams message, a GitHub issue, an incident channel post — before you start fixing it. Not to perform distress, but to start the paper trail. “Found an issue with the fct_orders null handling — investigating now, will update by EOD.” That message, and the chain of messages that follow it, is what turns an incident into evidence of a well-run team rather than evidence of a careless one.

ADRs for the decisions people will question.

The architecture decisions that feel obvious when you make them feel arbitrary when someone encounters them six months later without context. Why incremental and not table? Why Snowflake and not BigQuery? Why is revenue calculated at invoice date rather than payment receipt date? These questions have answers. The answers live in someone’s head, or in a Confluence page that nobody linked to the model.

ADRs — short markdown files in docs/adr/, committed to the repository — are the durable artifact of those decisions. They don’t need to be long. They need a title, a status, a description of the context, the decision, and the consequences. Once written and committed, they’re part of the repository’s history. They review in PRs. They can be linked from model descriptions. They don’t disappear when someone leaves the team.

dbt Exposures for impact legibility.

If you’re using dbt and you haven’t adopted Exposures, this is worth doing this week. Exposures are YAML-defined references to downstream artifacts — dashboards, ML models, reverse-ETL syncs — that extend the dbt DAG past the warehouse. A few lines in schema.yml that say “this model feeds the Weekly Revenue dashboard” means that when someone runs dbt ls --select +exposure:weekly_revenue, they get the complete list of upstream models that would need to change to affect that dashboard.

For a migration, this means you can answer “will this change break the CEO’s dashboard?” without archaeology. For a refactor, it means PR reviewers have a concrete list of stakeholders to notify. For a data quality incident, it means you know who to tell before they find out themselves.



Going Back to Mia

After my conversation with Mia — the one where she walked me through three weeks of extraordinary, invisible work — we had a more direct conversation about why none of it had been visible.

Her answer was the one Collins-Sussman identified in 2008: she didn’t want to show something until it was right. She was worried about raising a false alarm. She was concerned that surfacing work in progress would invite questions she couldn’t answer yet. There was also something she said that stuck with me: “I thought if I kept my head down long enough, I’d come up with something worth showing.”

All of those concerns were reasonable. None of them were wrong, exactly. What she’d underestimated was the cost of the silence — not to herself, but to the people around her. The finance stakeholder who’d gone to me with questions would have gone to Mia first if they’d had any signal that the work was in progress and legible. My not knowing, in a week where I’d been asked directly by leadership about the revenue model timeline, was a problem I hadn’t been equipped to solve.

What I should have done differently, and what I’ve done differently since: established the cadence contract at the start of the project, not after the silence had already built. Agreed, before the first commit, what “on track” communication would look like — what the update cadence would be, what would trigger an early surface, and what silence during a declared heads-down period meant. With that contract in place, three weeks of quiet is three weeks of “on track, as agreed.” Without it, three weeks of quiet is three weeks of “nobody knows what’s happening.”

The artifact we built together after that conversation was simple. For the remaining six weeks of the migration, Mia opened a draft PR on Monday morning with a brief plan for the week. She posted a comment each Friday with what had been done, what was pending, and any surprises. She added a parity dashboard that the finance lead could check at any time. She wrote three ADRs for the decisions that had taken the most deliberation.

The work didn’t change. The visibility changed. And the experience of the migration — for Mia, for the stakeholders, for me — changed entirely.

At the end of the project, the finance lead sent a message that I still think about. She said: “I’ve never felt so informed during a data migration. Usually I find out things are done when the thing I was waiting for just starts working.”

Mia had gone dark for three weeks. For the remaining six, she hadn’t. The difference wasn’t effort. It was artifacts.



The Closing Thought

Jeff Atwood closed his post with something direct: don’t go dark. Don’t be the developer in the room who hides their code until it’s done. Sharing work in progress feels riskier than it is. The feedback and connection it produces are worth more than the protection the silence provides.

He was right in 2008. The principle has only compounded since, as data teams have become more distributed, more remote, and more responsible for the kind of long-horizon work that naturally produces invisibility.

The engineers I’ve seen advance fastest in their careers are rarely the ones who do the most impressive work in isolation. They’re the ones who make their work legible — to their leads, to their stakeholders, to their future selves at 2am when a pipeline fails and the PR description is the only thing standing between a five-minute fix and a three-hour investigation.

Amy Edmondson’s hospital study keeps coming back to me. She’d expected to find that better hospital units made fewer errors. Instead she found that they reported more. Her reframe: maybe the better units don’t have fewer problems. Maybe they’re just more willing to say so.

I think the best data teams are the same. They’re not teams with fewer broken pipelines. They’re teams where a broken pipeline doesn’t stay quiet for long. Where the discovery of a bug is the beginning of a conversation, not a secret to be fixed before anyone notices. Where “we found an issue” is treated as evidence of diligence rather than evidence of failure.

Don’t go dark. Not because you have to perform being busy, or narrate every dead end, or flood your stakeholders with status updates that cost more attention than they’re worth. But because the work you’re doing matters to people who can’t see it. And the silence that feels protective is usually just silence.

Make the artifact. Open the draft PR. Write the commit message properly. Fill in the description. Post the Friday note. Put it in writing before you fix it.

Three weeks is going dark. One week is generous. One good artifact is enough.