Data Modelling

World Building: Your World Still Needs a Why

On the 17th of October, people sat down in the Great Hall of Stirling Castle to play Call of Cthulhu. I was in Scotland at the time. I didn’t make it. That still annoys me. What caught my attention wasn’t the castle (well fine, it kind of was the castle). It was what sat underneath the show. The game came out of A Viking in the Sun, a University of Edinburgh research project on Harald Hardrada, and its sister show in Istanbul earlier this year was built from real archaeological objects, historical sources and Icelandic sagas. Historians did months of careful work so that four players could walk into a room and go wherever they liked.

Friday, October 30, 2026 Read

The Style Isn't the Skill

I was a kid when I first watched Bruce Lee, soon after and my friend signed up for kung fu lessons. What hooked me wasn’t the fighting. It was the speed at which Bruce moved. The more you watched, though, the more you noticed something sitting behind the speed. He was obviously smart, fast, cool, and clearly thinking several moves ahead of everyone else in the room. Even as a kid you could tell the fighting was the visible bit of something much bigger.

Monday, August 31, 2026 Read

Ghost Skills: Teaching AI Agents to Think Like Data Engineers

Another week, another skills repo on the GitHub trending page. I know. There are roughly seventeen of them now, all promising to turn your AI coding agent from a confident intern into a slightly-less-confident intern. Most of them are great. Most of them are also built by solo devs, for solo devs, on solo-dev codebases that fit comfortably in a context window. Which is fine, if that’s your world. Less fine if your world involves a Snowflake warehouse with four tables that could be the source of truth for “customer”, an SCD2 someone half-built in 2021 and quietly walked away from, and a dbt project where stg_users_final_v3_actually_use_this is, somehow, the one you’re meant to use. (Don’t laugh. You’ve seen worse.)

Sunday, June 14, 2026 Read

The Duct Tape Data Engineer

The Engineer Who Ships I want to tell you about a data engineer I worked with. Let’s call her Sarah. Sarah had a reputation. When business stakeholders had an urgent question—the kind that arrives at 4 PM on a Friday with the CEO’s name in the subject line—they went to Sarah. Not to the senior architect with the impeccable data model. Not to the platform team with their carefully orchestrated Airflow DAGs. They went to Sarah.

Saturday, January 24, 2026 Read

The Guerrilla Guide to Data Engineering Interviews

The Scenario That Changes Everything Picture this: You’re sitting in an interview room—or more likely these days, staring at a Zoom window with your carefully curated bookshelf background—and the interviewer asks you about data quality. “Tell me about your experience with data quality,” they say. You have two choices. Choice A: “Data quality is really important in data engineering. It involves ensuring data is accurate, complete, consistent, and timely. I believe strongly in implementing data quality checks throughout the pipeline.”

Sunday, January 11, 2026 Read

Why Dimensional Modelling Isn't Dead—It's Just Getting Started

The Great Data Modelling Debate Nobody Asked For Another meeting where someone confidently declared, “We don’t need data modelling anymore—just dump everything in the data lake and let analysts figure it out.” I’ve heard variations of this statement for years now, in meetings or at conferences. The pitch is always the same: traditional data warehousing is dead, dimensional modelling is a relic from the 90s, and modern big data tools have made structured modelling obsolete. Schema-on-read is the future. Agility over architecture.

Friday, November 7, 2025 Read

dbt Fusion: The Engine Upgrade That's Got Everyone Talking

When Your Favourite Tool Gets a Makeover You know that feeling when your favourite app suddenly changes its interface? That mix of excitement and anxiety about whether the changes will actually improve your workflow or just mess with muscle memory you’ve spent years building. That’s exactly what happened when dbt Labs dropped dbt Fusion on the analytics engineering community. The reactions were… let’s call them passionate. Some folks were celebrating like they’d just discovered fire, while others were questioning whether this marked the beginning of the end for open-source dbt.

Saturday, June 21, 2025 Read

Mastering Data Interviews: A Comprehensive Guide

Introduction After nearly two decades in the data engineering field, I’ve sat on both sides of the interview table countless times. Whether you’re a seasoned professional looking to change roles or a newcomer trying to break into the field, the interview process for data engineering positions can be both challenging and mysterious. There’s often uncertainty about what questions you’ll face, what skills you need to demonstrate, and what interviewers are really looking for beneath the surface.

Saturday, February 22, 2025 Read

Embracing Defensive Engineering: A Proactive Approach to Data Pipeline Integrity

Introduction Have you ever had a data pipeline fall apart due to unexpected errors? In the ever-evolving landscape of data, surprises lurk around every corner. Defensive engineering, a methodology focused on preempting and mitigating data anomalies in data pipelines, plays a crucial role in building reliable data pipelines. It’s not just about fixing problems as they arise; it’s about anticipating potential issues and addressing them before they wreak havoc. Below I’ll explore the various facets of defensive engineering, from the basics of handling nulls and type mismatches to the more complex challenges of ensuring data integrity and handling late-arriving data. Whether you’re a seasoned data engineer or just starting out, understanding these principles is key to creating data pipelines that are not just functional, but also robust and secure in the face of unpredictable data challenges.

Data Modelling

Sunday, February 11, 2024 Read

Navigating the Data Labyrinth: The Art of Data Profiling

Introduction Imagine navigating a sprawling network of interconnected threads, each strand holding a vital clue. That’s the world of data for us, and profiling is our key to unlocking its secrets. It’s like deciphering a cryptic message, each character a piece of information waiting to be understood. But why is this so important? Ever encountered an error in your analysis, or a misleading conclusion based on faulty data? Data profiling helps us avoid these pitfalls by ensuring the data we work with is accurate, consistent, and ready to yield valuable insights. It’s like building a sturdy foundation before constructing a skyscraper.

Data Modelling

Sunday, January 28, 2024 Read

Taming the Chaos: Your Guide to Data Normalisation

Introduction Have you ever felt like you were drowning in a sea of data, where every byte seemed to play a game of hide and seek? In the digital world, where data reigns supreme, it’s not uncommon to find oneself navigating through a labyrinth of disorganised, redundant, and inconsistent information. But fear not, brave data navigators! There exists a beacon of order in this chaos: data normalisation. Data normalisation isn’t just a set of rules to follow; it’s the art of bringing structure and clarity to your data universe. It’s about transforming a jumbled jigsaw puzzle into a masterpiece of organisation, where every piece fits perfectly. Let’s embark on a journey to demystify this hero of the database world and discover how it can turn your data nightmares into a dream of efficiency and accuracy.

Data Modelling

Sunday, January 21, 2024 Read