Ghost in the data
  • Home
  • About
  • Posts
  • Start Here
  • Topics
  • Resources
  • RSS
  • Tags
  • 2026 Trends
  • AI
  • AI Agents
  • AI Bubble
  • AI Business Applications
  • AI Communication
  • AI Concepts
  • AI Ethics
  • AI Productivity
  • AI Prompting
  • AI Tools
  • AI Workflows
  • Analytics
  • Analytics Engineering
  • Anonymisation
  • Apache Airflow
  • Apache Iceberg
  • API Integration
  • Architecture
  • Astronomer
  • Athena
  • Automation
  • AVRO
  • AWS
  • AWS Glue
  • Banking
  • Bedrock Edition
  • Best Practices
  • Big Data
  • Blue-Green Deployment
  • Business Case
  • Business Value
  • Career Development
  • Change Management
  • Chapter Lead
  • ChatGPT
  • CI/CD
  • Claude
  • Claude Code
  • Cloud Computing
  • Cloud Gaming
  • Code Comments
  • Code Review
  • Communication
  • Conceptual Data Modelling
  • Continuous Learning
  • Cost Optimisation
  • CSV
  • Customer Experience
  • Dagster
  • Data Architecture
  • Data Contracts
  • Data Democratisation
  • Data Engineering
  • Data Ethics
  • Data Freshness
  • Data Governance
  • Data Impact
  • Data Ingestion
  • Data Minimisation
  • Data Modelling
  • Data Observability
  • Data Ownership
  • Data Pipelines
  • Data Platform
  • Data Privacy
  • Data Quality
  • Data Reliability
  • Data Solutions
  • Data System Resilience
  • Data Teams
  • Data Testing
  • Data Transformation
  • Data Validation
  • Data Vault
  • Data Warehouse
  • Database Design
  • Datafold
  • Dbt
  • Decision Making
  • Delta Lake
  • Development
  • Development Tools
  • DevOps
  • Dimensional Modelling
  • Documentation
  • DuckDB
  • Emotional Intelligence
  • Empathetic Design
  • Employee Engagement
  • Employee Experience
  • Employee Productivity
  • Engineering Leadership
  • Enterprise
  • Estimation
  • ETL
  • Family Gaming
  • Feedback
  • File Formats
  • Financial Crisis
  • FinOps
  • Fivetran
  • Frameworks
  • Friendship
  • Future of Work
  • GCP
  • GDPR
  • Git
  • GitBash
  • GitHub
  • GitHub Actions
  • Grief
  • Hiring
  • Historical Load
  • Human Connection
  • Idempotency
  • Incentives
  • Incident Response
  • Industry Trends
  • Innovation
  • Inspirational Quote
  • Intergroup Conflict
  • Interviews
  • Journal
  • Journaling Techniques
  • JSON
  • Junior Engineer
  • Kimball
  • Knowledge Management
  • Lakehouse
  • Lambda
  • Leadership
  • Legacy Systems
  • Life
  • LLM
  • Loss
  • MacOS
  • Mental Health
  • Mentorship
  • Mindfulness Practices
  • Minecraft
  • Modern Data Stack
  • Moral Development
  • Motivation
  • MWAA
  • Obsidian
  • Onboarding
  • One-on-One Meetings
  • Open Source
  • OpenFlow
  • ORC
  • Orchestration
  • Organizational Culture
  • Parquet
  • Pattern Bank
  • Performance Optimisation
  • Performance Reviews
  • Personal
  • Personal Finance
  • Personal Growth
  • Philosophy
  • Pipeline
  • Pipeline Architecture
  • Pipeline Design
  • Platform Strategy
  • PostgreSQL
  • Pragmatism
  • Prefect
  • Problem Solving
  • Production Issues
  • Productivity
  • Professional Relationships
  • Project Management
  • Promotion
  • Psychological Safety
  • Public Speaking
  • Python
  • RAG
  • Refactoring
  • Remote Work
  • Reputation
  • Requirements Gathering
  • Retention Policies
  • RFC 4180
  • Risk Management
  • Robbers Cave Experiment
  • ROI
  • Roleplaying
  • S3
  • Salesforce
  • SCD
  • Schema Drift
  • Schema Evolution
  • Self Evaluation
  • Self-Awareness
  • Self-Reflection
  • Semantic Layer
  • Server Setup
  • Service Design
  • Shadow IT
  • Skills
  • Snowflake
  • Soft Skills
  • Solution Design
  • SQL
  • SQL Standards
  • SSH
  • SSH Keys
  • Staff Engineer
  • Stakeholder Management
  • Star Schema
  • Starburst
  • Step Functions
  • Strangler Fig
  • Strategy
  • Streaming
  • Strengths
  • Success Habits
  • Supervisory Control
  • Tabletop RPG
  • Team Building
  • Team Collaboration
  • Team Culture
  • Team Enablement
  • Technical Assessment
  • Technical Debt
  • Technical Strategy
  • Testing
  • Tools and Access
  • Trino
  • Trust
  • User Experience
  • UV
  • Value Creation
  • Vector Databases
  • Vector Search
  • Virtual Environments
  • Visualisation
  • VSCode
  • Wellbeing
  • Windows
  • Workplace Communication
  • Write-Audit-Publish
  • Zsh
Hero Image
Your Data Model Isn't Broken, Part II: The Refactoring Playbook

In [Part I], I made the case that your legacy data model isn’t the disaster it looks like. That the strange WHERE clauses, the bridge tables nobody can explain, and the slowly-changing-dimension-within-a-slowly-changing-dimension aren’t bugs — they’re business rules earned through years of production reality. I argued that big-bang rebuilds fail at alarming rates, that the complexity you’re fighting is mostly essential rather than accidental, and that the impulse to “start from scratch” is driven more by cognitive bias than by engineering judgment.

  • Data Engineering
  • Refactoring
  • Data Warehouse
  • dbt
  • Snowflake
  • Apache Iceberg
  • Write-Audit-Publish
  • Strangler Fig
  • Data Quality
Saturday, March 28, 2026 Read