Dan Kruse

Data Engineering Consultant

👤 Profile

Experienced Data Engineer Consultant with a proven track record of designing and optimising scalable data platforms. I specialise in end-to-end solutions, from data ingestion and transformation to presentation, with a focus on efficiency and future readiness.


I’ve worked on key projects, including rescuing stalled cloud migrations and leading greenfield data platform development, ensuring robust, scalable, and cost-effective solutions.

🛠️ Skills

  • Cloud Platforms: GCP, AWS, Azure, Snowflake, Databricks
  • Programming: Python, SQL, JavaScript
  • Orchestration: Airflow
  • Transformation: dbt
  • Data Lake: S3, Glue Data Catalog, Apache Iceberg
  • Databases: PostgreSQL, MySQL, MongoDB, SQL Server
  • Data Warehouses: BigQuery, Redshift, Snowflake
  • Infrastructure as Code: Terraform, Pulumi
  • Data Visualisation: Tableau, QuickSight, GoodData, Power BI

🎓 Education

BSc (Hons) Civil Engineering

Anglia Ruskin University, 2006 - 2009

📜 Certifications

  • Building Batch Data Pipelines on GCP, Coursera (Sept 2021)
  • Modernizing Data Lakes and Warehouses with GCP, Coursera (Aug 2021)
  • Advanced React, Bostype Inc. (Jul 2021)
  • React for Beginners, Bostype Inc. (Jan 2021)
  • Master Gatsby, Bostype Inc. (Nov 2020)
  • Google Cloud Big Data and ML Fundamentals, Coursera (Nov 2018)
  • Relational Database Support for Data Warehouses, Coursera (Jul 2017)
  • Data Warehouse Concepts and Integration, Coursera (Jan 2017)
  • Database Management Essentials, Coursera (Nov 2016)
  • Web Development Bootcamp, Lighthouse Labs Vancouver (Sept 2015)

💼 Experience

Data Engineering Consultant - Patchwork Health

Dec 2022 - Present

Spearheaded a data platform transformation to enable scale-up ambitions:
  • Documented the existing infrastructure and identified improvement opportunities.
  • Collaborated with stakeholders to gather technical and business requirements.
  • Conducted vendor and tool evaluation to support data science goals.
  • Redesigned and implemented a robust data model and platform architecture.

Technical Skills: AWS, dbt, Redshift, Airflow, Docker

Data Engineering Consultant - Dr. Martens

Jan 2024 - Jun 2024

Revamped a stalled cloud migration initiative:
  • Performed in-depth discovery to understand the existing setup and challenges.
  • Optimised the technology stack for scalability and efficiency.
  • Streamlined and consolidated data pipelines to simplify operations.
  • Ensured client alignment and clear communication throughout the project.

Technical Skills: Azure, Airflow, Terraform, Python, SQL

Senior Data Engineer - HealthHero

Jan 2022 - Oct 2022

Led the development of a new analytics platform, integrating datasets from multiple business entities:
  • Implemented IaC and automated deployment pipelines for scalable infrastructure.
  • Designed and deployed containerised pipelines to integrate datasets.
  • Developed a data lake and warehouse for efficient large-scale data management.
  • Optimised resource usage and costs with orchestration and autoscaling.

Technical Skills: AWS, Airflow, dbt, Docker, SQL

Senior Data Engineer - MADE.COM

Jan 2021 - Jan 2022

Mentored junior engineers, reviewed squad code, and designed scalable solutions:
  • Oversaw engineering code for quality and maintainability.
  • Designed and implemented scalable engineering solutions.
  • Collaborated with stakeholders to align engineering efforts with business goals.

Technical Skills: GCP, Docker, Kubernetes, BigQuery, Python, SQL

Data Engineer - MADE.COM

Jun 2018 - Jan 2021

Contributed to the design and implementation of scalable data solutions to support business operations and analytics:
  • Built and maintained robust ETL pipelines for data ingestion and transformation.
  • Optimised data models to enhance query performance and reduce processing times.
  • Managed workflows and task automation using orchestration tools to improve efficiency.
  • Implemented automated testing and deployment processes to ensure reliability and consistency.
  • Developed custom tools and scripts to address unique business requirements.

Technical Skills: GCP, Docker, Kubernetes, BigQuery, Python, SQL

Data Developer - King's Health Partners

Apr 2016 - Jun 2018

Redesigned a clinical data warehouse covering over 200,000 diabetic patients:
  • Developed interactive data visualisations for research and publications.
  • Extracted and transformed data for clinical research and service design.
  • Created SQL triggers, functions, and stored procedures for data processing.

Technical Skills: SQL, Tableau, R