Pipelines that don't break at 3am
Platform migrations, warehouse modernization, and pipeline development with validation testing and clear documentation. No duct tape.
Who this is for
You're a data team or engineering leader dealing with:
- A platform migration that's been "in progress" for months
- Pipelines that break silently and surface bad data downstream
- A data warehouse that's grown organically and nobody trusts
- Technical debt that's slowing down your analytics team
- A contractor handoff where documentation is missing
Common problems I solve
- "We have data everywhere and don't trust it"
- "Our Snowflake costs are out of control"
- "The migration was 80% done, then the contractor left"
- "Pipelines are brittle and nobody knows why"
- "We need to consolidate 5 different data sources"
- "Our dbt project is a mess"
What you get
Platform Migrations
Complete migrations between Snowflake, Databricks, BigQuery, and Redshift. Includes schema translation, query conversion, and reconciliation testing to verify data integrity.
Pipeline Development
End-to-end ETL/ELT pipelines using Spark, dbt, Airflow, or Azure Data Factory. Built with proper error handling, monitoring, and alerting from day one.
Data Modeling
Dimensional modeling, data vault, or whatever fits your use case. Clean schemas that your analytics team can actually query without a translator.
Reconciliation Frameworks
Automated tests that verify row counts, aggregates, and business logic match between source and target. Catch issues before your stakeholders do.
Technologies I work with
I pick the right tool for the job, not the trendy one. Here's my core stack:
Databricks, Snowflake, BigQuery, Redshift, Postgres
Spark, dbt, Delta Lake, Iceberg, Python, SQL
Airflow, Azure Data Factory, Prefect, Dagster
AWS, Azure, GCP
Typical engagements
Audit & Plan
- Current architecture review
- Data quality assessment
- Migration or build roadmap
- Risk identification
- Effort estimation
Migration Sprint
- 2-week delivery sprints
- Schema and query migration
- Reconciliation testing
- Documentation
- 30-day support
Ongoing Retainer
- Pipeline monitoring
- Bug fixes and optimizations
- New feature development
- Priority support
- Monthly reviews
What I need from you
- Access to source systems (read-only is fine)
- A point of contact who can answer questions
- Requirements doc or stakeholder interview time
- Access to target platform (or I can set it up)
- NDA signed before accessing sensitive data