Ghost in the data
  • Home
  • About
  • Posts
  • Topics
  • Categories
  • Analytics Engineering
  • Artificial Intelligence
  • Best Practices
  • Big Data
  • Business Technology
  • Career Development
  • Cloud Computing
  • Communication
  • Conflict Resolution
  • Data Engineering
  • Data Modeling
  • Data Modelling
  • Data Pipelines
  • Data Quality
  • Data Storage
  • Data Warehousing
  • Database Design
  • Dbt
  • Delta-Lake
  • Development
  • Development Tools
  • DevOps
  • Employee Engagement
  • Gaming Servers
  • Google Cloud Platform
  • Hiring
  • IT Management
  • Leadership
  • Life Hacks
  • Mindfulness
  • Minecraft
  • Personal Development
  • Pipeline
  • Pipeline Design
  • Productivity
  • Professional Development
  • Professional Growth
  • Promotion
  • Psychology
  • Python
  • Python Tools
  • Setup Guide
  • Stakeholder Management
  • Team Building
  • Team Management
  • Technology Trends
  • Tutorial
  • Version Control
  • Workplace Dynamics
Hero Image
Setting Up Your Data Engineering Environment on Windows

Introduction Setting up a development environment for data engineering on Windows requires some specific considerations that differ from Unix-based systems. This guide will walk you through creating a robust Python development environment on Windows, with detailed explanations of each component and why it’s important. Clean Slate: Removing Existing Python Installations Before starting, it’s important to remove any existing Python installations to avoid conflicts: Open Windows Settings > Apps > Apps & Features Search for “Python” Uninstall any Python versions listed Also check and remove Python from these locations:

  • Python
  • DBT
  • Windows
  • UV Package Manager
  • VSCode
Monday, February 3, 2025 Read
Hero Image
Setting Up Your Data Engineering Environment on MacOS

Introduction Setting up a development environment for data engineering on MacOS requires careful consideration of package management, Python version control, and tool configuration. This guide will walk you through the process, explaining not just how to set up these tools, but why each component is important. Clean Slate: Removing Existing Python Installations Before we begin, it’s important to ensure we’re starting with a clean slate. Multiple Python installations can cause confusion and conflicts. Let’s remove any existing Python installations:

  • Python
  • DBT
  • MacOS
  • UV Package Manager
  • VSCode
Sunday, February 2, 2025 Read
Hero Image
Automating Python Virtual Environments with Zsh on macOS

Automating Python Virtual Environments with Zsh on macOS Managing Python virtual environments can be tedious - having to manually activate and deactivate them as you move between projects. I decided to create a script that will automatically activate virtual environment. When you cd into a directory containing a .venv folder. When you leave, it will deactivate it. Installing Zsh While macOS comes with zsh as the default shell since Catalina (10.15), you may want to ensure you have the latest version. The easiest way to install or update zsh is using Homebrew:

  • Python
  • Virtual Environments
  • Zsh
  • macOS
  • Automation
Tuesday, January 14, 2025 Read