Career Development

Don't Go Dark: Visibility Is a Data Engineering Skill

There’s a specific kind of silence in data engineering that I’ve learned to fear. Not the silence of a system that’s working well. Not the comfortable quiet of a team in flow. I mean the silence of a project that’s been running for three weeks and you still can’t point to a single visible thing it has produced. The kind of silence where, if your manager stopped you in the hallway and asked “how’s that migration going?”, you’d say “fine” because saying anything more accurate would require explaining things you haven’t fully articulated yet — even to yourself.

Saturday, May 23, 2026 Read

Five Worlds of Data Engineering

You watch a conference talk about implementing data contracts, and nobody mentions that the advice assumes you have multiple teams producing data — which you don’t. You read a post declaring “if you’re still using stored procedures in 2026, you’re doing it wrong,” and the comments erupt. Half the people are nodding along. Half are furious. Both sides are right. They’re just living in different worlds and don’t realise it.

Saturday, May 2, 2026 Read

You Don't Need Permission to Fix Your Data

Let me tell you about a junior engineer called Sam. Sam had been on the team about four months when I noticed something in a pull request. Tucked between two routine model changes was a new schema.yml entry — five accepted_values tests on a column called customer_status that had been silently accumulating fourteen different spellings of “active” for the better part of a year. Nobody asked Sam to do this. It wasn’t in a sprint. There was no Jira ticket. Sam had just been working in that part of the warehouse, noticed the mess, and decided to clean it up on the way through.

Saturday, March 21, 2026 Read

Your Friends Will Be There for You. Your Work Won't.

There’s a table in a rented house in Queenscliff — nothing fancy, just whatever furniture came with the place — with a hand-drawn map spread across it, dice scattered at the edges, and snacks slowly migrating toward the centre of the board. Once a year, Steve, one of our friends, organises for us to make a trip down to Queenscliff to play tabletop RPGs for a weekend. More recently, we’ve started playing monthly or fortnightly at home too. Nothing glamorous about any of it. We’re not Glass Cannon Network or Critical Role — no cameras, no production values, no audience watching us roleplay with solemn gravity.

Wednesday, March 18, 2026 Read

What an NBA Coach Can Teach Data Leaders About Building Teams That Actually Work

I was three hours into a retrospective that had devolved into blame-shifting when the most senior engineer on the team finally spoke up. “Look,” he said, “we can keep pointing fingers at the data model, or we can admit we don’t actually trust each other enough to have an honest conversation about what went wrong.” The room went quiet. He was right. That moment stuck with me because it exposed something I’ve seen destroy more data teams than bad architecture ever could: the absence of genuine connection between people who spend forty-plus hours a week depending on each other.

Monday, February 2, 2026 Read

Context Engineering: The New Must-Have Skill for Data Engineers

Last year I watched a colleague ask AI to help write a dbt model. The AI spit out perfectly functional SQL—clean syntax, proper CTEs, the works. Looked great. Then I noticed the table would eventually hold 800 million rows. No partitioning. No clustering. Just a raw, unoptimised heap waiting to turn into a query performance nightmare (that would likely become my nightmare to fix). The engineer wasn’t at fault. The AI wasn’t at fault either, really. The AI simply didn’t know that our environment clusters large tables by date. It didn’t know our team’s conventions around incremental models. It couldn’t know, because nobody had told it.

Saturday, January 31, 2026 Read

The Duct Tape Data Engineer

The Engineer Who Ships I want to tell you about a data engineer I worked with. Let’s call her Sarah. Sarah had a reputation. When business stakeholders had an urgent question—the kind that arrives at 4 PM on a Friday with the CEO’s name in the subject line—they went to Sarah. Not to the senior architect with the impeccable data model. Not to the platform team with their carefully orchestrated Airflow DAGs. They went to Sarah.

Saturday, January 24, 2026 Read

The 2026 Data Engineering Strategy Nobody's Writing (But Everyone Needs)

What if I told you the biggest threat to your data platform isn’t technology—it’s that we’ve stopped building the next generation of engineers who’ll run it? Not the latest database that promises to solve everything. Not whether you picked the right orchestrator. The real crisis is that we’ve systematically broken our talent pipeline. And in 2026, that decision is going to start costing us in ways that no amount of tooling can fix.

Thursday, January 15, 2026 Read

The Guerrilla Guide to Data Engineering Interviews

The Scenario That Changes Everything Picture this: You’re sitting in an interview room—or more likely these days, staring at a Zoom window with your carefully curated bookshelf background—and the interviewer asks you about data quality. “Tell me about your experience with data quality,” they say. You have two choices. Choice A: “Data quality is really important in data engineering. It involves ensuring data is accurate, complete, consistent, and timely. I believe strongly in implementing data quality checks throughout the pipeline.”

Sunday, January 11, 2026 Read

Why Your Ideas Die in Planning Meetings

The silence that kills good ideas One morning, I sat in yet another meeting where we just spent two weeks backfilling a table then we found it was riddled with issues with the data. Even if we resolve the issue, it would then be another 2 weeks to backfill the data, there has to be a better way. “So, what do we think? Give me your best ideas for tackling this.”

Wednesday, January 7, 2026 Read

The Science of Conversation for people who hate small talk

One morning, I watched a data engineer struggle with using AI for thirty minutes, trying to debug a DBT job. The problem wasn’t the LLM’s capabilities—it was how the engineer framed the question. No context about what they’d already tried. No explanation of the expected versus actual output. Just “fix this code” followed by a massive code dump. This same engineer had similar struggles with stakeholders. Presentations that assumed too much context. Emails that buried the ask. Meetings where they answered questions nobody asked.

Sunday, January 4, 2026 Read

Financial Independence: Your Shield Against Job Loss Fear

The Fear That Follows You Home One evening, after pushing another commit past midnight, I couldn’t bring myself to sit up. Not because I was tired—though I was. Not because the commit had issues—it went smoothly, and tested all fine. I couldn’t get up because I’d spent the entire day with a knot in my stomach, wondering if our team would survive the next round of “organizational restructuring.” Here’s what made it worse: I had no idea if my fear was rational. Were we really at risk? Or was I just catastrophizing? The uncertainty was eating me alive.

Sunday, November 2, 2025 Read