Ghost in the data
  • Home
  • About
  • Posts
  • Start Here
  • Topics
  • Resources
  • RSS
  • Categories
  • AI
  • Career
  • Data Engineering
  • Data Modelling
  • Data Quality & Governance
  • Leadership
  • Life
  • Platforms & Cloud
Hero Image
Nobody Reads 7,000 Words the Night Before an Interview

Across two posts, I’ve written close to 7,000 words on data engineering interviews. Mastering Data Interviews walked through the formats and how to prepare for each one. The Guerrilla Guide to Data Engineering Interviews made the case that you get hired on what you’ve built far more than on what you’ve memorised. All of that is no help the night before an interview, when there’s a tickle of anxiety about how you’ll do and you’re trying to remember how you’d explain a window function (it always comes up). It’s even less help to the person interviewing you, who opens their notes five minutes before an interview they forgot was in the diary.

  • Interviews
  • Hiring
  • Career Development
  • Technical Assessment
  • Skills Matrix
  • Structured Interviews
  • SQL
Saturday, November 21, 2026 Read
Hero Image
The Style Isn't the Skill

I was a kid when I first watched Bruce Lee, soon after and my friend signed up for kung fu lessons. What hooked me wasn’t the fighting. It was the speed at which Bruce moved. The more you watched, though, the more you noticed something sitting behind the speed. He was obviously smart, fast, cool, and clearly thinking several moves ahead of everyone else in the room. Even as a kid you could tell the fighting was the visible bit of something much bigger.

  • Leadership
  • Career Development
  • Performance Reviews
  • Self Evaluation
  • Data Modelling
  • Streaming
  • Skills
  • Philosophy
Monday, August 31, 2026 Read
Hero Image
The Competitive Moat That AI Can't Replicate

The Restaurant That Refused to Take Bookings Online Let me tell you a story about a restaurant owner who became obsessed with human connection. He didn’t want people booking online. He wanted them to call. He wanted the ritual of a human voice, the small exchange about an anniversary or a first date, the warmth of being recognised. His team thought he was losing his mind. Online bookings were standard. Everyone did it. Why make customers work harder?

  • Human Connection
  • Trust
  • Customer Experience
  • Leadership
  • AI
  • Employee Experience
  • Team Culture
Saturday, June 13, 2026 Read
Hero Image
Don't Go Dark: Visibility Is a Data Engineering Skill

There’s a specific kind of silence in data engineering that I’ve learned to fear. Not the silence of a system that’s working well. Not the comfortable quiet of a team in flow. I mean the silence of a project that’s been running for three weeks and you still can’t point to a single visible thing it has produced. The kind of silence where, if your manager stopped you in the hallway and asked “how’s that migration going?”, you’d say “fine” because saying anything more accurate would require explaining things you haven’t fully articulated yet — even to yourself.

  • Communication
  • Data Quality
  • GitHub
  • dbt
  • Remote Work
  • Career Development
  • Team Culture
  • Psychological Safety
Saturday, May 23, 2026 Read
Hero Image
Five Worlds of Data Engineering

You watch a conference talk about implementing data contracts, and nobody mentions that the advice assumes you have multiple teams producing data — which you don’t. You read a post declaring “if you’re still using stored procedures in 2026, you’re doing it wrong,” and the comments erupt. Half the people are nodding along. Half are furious. Both sides are right. They’re just living in different worlds and don’t realise it.

  • Data Engineering
  • Modern Data Stack
  • Enterprise
  • Data Architecture
  • Career Development
  • Leadership
Saturday, May 2, 2026 Read
Hero Image
What an NBA Coach Can Teach Data Leaders About Building Teams That Actually Work

I was three hours into a retrospective that had devolved into blame-shifting when the most senior engineer on the team finally spoke up. “Look,” he said, “we can keep pointing fingers at the data model, or we can admit we don’t actually trust each other enough to have an honest conversation about what went wrong.” The room went quiet. He was right. That moment stuck with me because it exposed something I’ve seen destroy more data teams than bad architecture ever could: the absence of genuine connection between people who spend forty-plus hours a week depending on each other.

  • Leadership
  • Team Building
  • Team Culture
  • Data Teams
  • Remote Work
  • Psychological Safety
Monday, February 2, 2026 Read
Hero Image
Context Engineering: The New Must-Have Skill for Data Engineers

Last year I watched a colleague ask AI to help write a dbt model. The AI spit out perfectly functional SQL—clean syntax, proper CTEs, the works. Looked great. Then I noticed the table would eventually hold 800 million rows. No partitioning. No clustering. Just a raw, unoptimised heap waiting to turn into a query performance nightmare (that would likely become my nightmare to fix). The engineer wasn’t at fault. The AI wasn’t at fault either, really. The AI simply didn’t know that our environment clusters large tables by date. It didn’t know our team’s conventions around incremental models. It couldn’t know, because nobody had told it.

  • AI
  • dbt
  • Data Quality
  • SQL
  • Productivity
  • VSCode
  • Claude
Saturday, January 31, 2026 Read
Hero Image
The Duct Tape Data Engineer

The Engineer Who Ships I want to tell you about a data engineer I worked with. Let’s call her Sarah. Sarah had a reputation. When business stakeholders had an urgent question—the kind that arrives at 4 PM on a Friday with the CEO’s name in the subject line—they went to Sarah. Not to the senior architect with the impeccable data model. Not to the platform team with their carefully orchestrated Airflow DAGs. They went to Sarah.

  • Data Engineering
  • DuckDB
  • Architecture
  • Pragmatism
  • Career Development
  • Technical Strategy
  • Data Platform
  • Kimball
  • Data Modelling
Saturday, January 24, 2026 Read
Hero Image
The 2026 Data Engineering Strategy Nobody's Writing (But Everyone Needs)

What if I told you the biggest threat to your data platform isn’t technology—it’s that we’ve stopped building the next generation of engineers who’ll run it? Not the latest database that promises to solve everything. Not whether you picked the right orchestrator. The real crisis is that we’ve systematically broken our talent pipeline. And in 2026, that decision is going to start costing us in ways that no amount of tooling can fix.

  • Strategy
  • Team Building
  • Cost Optimisation
  • DuckDB
  • AI Tools
  • Career Development
  • 2026 Trends
  • Future of Work
Thursday, January 15, 2026 Read
Hero Image
The Guerrilla Guide to Data Engineering Interviews

The Scenario That Changes Everything Picture this: You’re sitting in an interview room—or more likely these days, staring at a Zoom window with your carefully curated bookshelf background—and the interviewer asks you about data quality. “Tell me about your experience with data quality,” they say. You have two choices. Choice A: “Data quality is really important in data engineering. It involves ensuring data is accurate, complete, consistent, and timely. I believe strongly in implementing data quality checks throughout the pipeline.”

  • Interviews
  • Career Development
  • Technical Assessment
  • SQL
  • Data Modelling
  • Problem Solving
  • Delta Lake
  • dbt
  • Data Quality
Sunday, January 11, 2026 Read
Hero Image
The Science of Conversation for people who hate small talk

One morning, I watched a data engineer struggle with using AI for thirty minutes, trying to debug a DBT job. The problem wasn’t the LLM’s capabilities—it was how the engineer framed the question. No context about what they’d already tried. No explanation of the expected versus actual output. Just “fix this code” followed by a massive code dump. This same engineer had similar struggles with stakeholders. Presentations that assumed too much context. Emails that buried the ask. Meetings where they answered questions nobody asked.

  • Communication
  • Soft Skills
  • Leadership
  • Career Development
  • Team Building
  • Stakeholder Management
Sunday, January 4, 2026 Read
Hero Image
The Invisible PR You're Building Right Now

The Email That Changed Everything I received a meeting invite labelled “Check In” and it was to discuss news about the restructure. In a few months, my role would be gone. I was still processing it, sitting with that particular brand of numbness that comes when your career gets upended. In the following weeks, my inbox started filling up. LinkedIn messages. Texts. Emails from people I’d worked with years ago.

  • Professional Relationships
  • Trust
  • Reputation
  • Leadership
  • Career Development
  • Team Culture
Monday, December 8, 2025 Read
  • ««
  • «
  • 1
  • 2
  • »
  • »»