Ghost in the data
  • Home
  • About
  • Posts
  • Start Here
  • Topics
  • Resources
  • RSS
  • Tags
  • 2026 Trends
  • AI
  • AI Agents
  • AI Bubble
  • AI Business Applications
  • AI Communication
  • AI Concepts
  • AI Ethics
  • AI Productivity
  • AI Prompting
  • AI Tools
  • AI Workflows
  • Analytics
  • Analytics Engineering
  • Anonymisation
  • Apache Airflow
  • Apache Iceberg
  • API Integration
  • Architecture
  • Astronomer
  • Athena
  • Automation
  • AVRO
  • AWS
  • AWS Glue
  • Banking
  • Bedrock Edition
  • Best Practices
  • Big Data
  • Blue-Green Deployment
  • Business Case
  • Business Value
  • Career Development
  • Change Management
  • Chapter Lead
  • ChatGPT
  • CI/CD
  • Claude
  • Claude Code
  • Cloud Computing
  • Cloud Gaming
  • Code Comments
  • Code Review
  • Communication
  • Conceptual Data Modelling
  • Continuous Learning
  • Cost Optimisation
  • CSV
  • Customer Experience
  • Dagster
  • Data Architecture
  • Data Contracts
  • Data Democratisation
  • Data Engineering
  • Data Ethics
  • Data Freshness
  • Data Governance
  • Data Impact
  • Data Ingestion
  • Data Minimisation
  • Data Modelling
  • Data Observability
  • Data Ownership
  • Data Pipelines
  • Data Platform
  • Data Privacy
  • Data Quality
  • Data Reliability
  • Data Solutions
  • Data System Resilience
  • Data Teams
  • Data Testing
  • Data Transformation
  • Data Validation
  • Data Vault
  • Data Warehouse
  • Database Design
  • Datafold
  • Dbt
  • Decision Making
  • Delta Lake
  • Development
  • Development Tools
  • DevOps
  • Dimensional Modelling
  • Documentation
  • DuckDB
  • Emotional Intelligence
  • Empathetic Design
  • Employee Engagement
  • Employee Experience
  • Employee Productivity
  • Enterprise
  • Estimation
  • ETL
  • Family Gaming
  • Feedback
  • File Formats
  • Financial Crisis
  • FinOps
  • Fivetran
  • Frameworks
  • Friendship
  • Future of Work
  • GCP
  • GDPR
  • Git
  • GitBash
  • GitHub
  • GitHub Actions
  • Grief
  • Hiring
  • Historical Load
  • Human Connection
  • Idempotency
  • Incentives
  • Incident Response
  • Industry Trends
  • Innovation
  • Inspirational Quote
  • Intergroup Conflict
  • Interviews
  • Journal
  • Journaling Techniques
  • JSON
  • Junior Engineer
  • Kimball
  • Knowledge Management
  • Lakehouse
  • Lambda
  • Leadership
  • Legacy Systems
  • Life
  • LLM
  • Loss
  • MacOS
  • Mental Health
  • Mentorship
  • Mindfulness Practices
  • Minecraft
  • Modern Data Stack
  • Moral Development
  • Motivation
  • MWAA
  • Obsidian
  • Onboarding
  • One-on-One Meetings
  • Open Source
  • OpenFlow
  • ORC
  • Orchestration
  • Organizational Culture
  • Parquet
  • Pattern Bank
  • Performance Optimisation
  • Performance Reviews
  • Personal
  • Personal Finance
  • Personal Growth
  • Philosophy
  • Pipeline
  • Pipeline Architecture
  • Pipeline Design
  • Platform Strategy
  • PostgreSQL
  • Pragmatism
  • Prefect
  • Problem Solving
  • Production Issues
  • Productivity
  • Professional Relationships
  • Project Management
  • Promotion
  • Psychological Safety
  • Public Speaking
  • Python
  • RAG
  • Refactoring
  • Remote Work
  • Reputation
  • Requirements Gathering
  • Retention Policies
  • RFC 4180
  • Risk Management
  • Robbers Cave Experiment
  • ROI
  • Roleplaying
  • S3
  • Salesforce
  • SCD
  • Schema Drift
  • Schema Evolution
  • Self Evaluation
  • Self-Awareness
  • Self-Reflection
  • Semantic Layer
  • Server Setup
  • Service Design
  • Shadow IT
  • Skills
  • Snowflake
  • Soft Skills
  • Solution Design
  • SQL
  • SQL Standards
  • SSH
  • SSH Keys
  • Staff Engineer
  • Stakeholder Management
  • Star Schema
  • Starburst
  • Step Functions
  • Strangler Fig
  • Strategy
  • Streaming
  • Strengths
  • Success Habits
  • Tabletop RPG
  • Team Building
  • Team Collaboration
  • Team Culture
  • Team Enablement
  • Technical Assessment
  • Technical Debt
  • Technical Strategy
  • Testing
  • Tools and Access
  • Trino
  • Trust
  • User Experience
  • UV
  • Value Creation
  • Vector Databases
  • Vector Search
  • Virtual Environments
  • Visualisation
  • VSCode
  • Wellbeing
  • Windows
  • Workplace Communication
  • Write-Audit-Publish
  • Zsh
Hero Image
The 2026 Data Engineering Strategy Nobody's Writing (But Everyone Needs)

What if I told you the biggest threat to your data platform isn’t technology—it’s that we’ve stopped building the next generation of engineers who’ll run it? Not the latest database that promises to solve everything. Not whether you picked the right orchestrator. The real crisis is that we’ve systematically broken our talent pipeline. And in 2026, that decision is going to start costing us in ways that no amount of tooling can fix.

  • Strategy
  • Team Building
  • Cost Optimisation
  • DuckDB
  • AI Tools
  • Career Development
  • 2026 Trends
  • Future of Work
Thursday, January 15, 2026 Read
Hero Image
Building AI Agents with Claude Code

Introduction Imagine you’re reviewing a pull request with dozens of SQL files, each containing complex queries for your data pipeline. You spot inconsistent formatting, or syntax which doesn’t work with your infrastructure. Sound familiar? It’s common for data professionals to struggle with maintaining consistent SQL standards across their projects, especially when working with specialised platforms and it can be time consuming to review these elements within a peer review. It would be better use of time to focus on the hard thinking elements, like logic etc. However these small syntax or style issues, can be distracting. Well at least they are for me.

  • Claude Code
  • AI Agents
  • Starburst
  • Delta Lake
  • Trino
  • Data Validation
  • dbt
  • Data Engineering
  • AI Tools
  • VSCode
Saturday, September 13, 2025 Read