Reliable data.

Layered warehouse design, dimensional modelling and versioned ELT with CI/CD and quality assertions across 3,000+ tables, plus catalog and lineage across 100+ sources.

Three areas, one question: can we trust this data enough to decide?

Projects

Production work on this topic.

  1. P—08

    Completed

    Data Governance, Metadata Management and MDM

    Data governance, metadata management and master data management (MDM) across 100+ source systems, with data catalog and lineage tooling.

    • Data governance
    • Data lineage
    • Data quality

    Role

    Implementation · Big Data Management & Data Architecture Manager

  2. P—01

    Completed

    Featured

    Enterprise Data Warehouse Migration to Google BigQuery

    80 TB warehouse moved from SQL Server and a 15-node Cloudera cluster to BigQuery: $550K annual savings, 75% lower infrastructure cost, 7x faster workflows.

    • Data platform
    • Data quality
    • Analytics-ready data
    • Orchestration
    • Cloud

    Role

    Design and lead · Senior Data Engineering & Data Architecture Group Manager

  3. P—13

    Completed

    SSIS Framework for ETL Pipelines

    Metadata-driven SSIS ETL framework with dependency tracking, error handling, restartability and automated alerts, adopted by 80+ pipelines.

    • Data pipelines
    • Orchestration
    • Data observability
    • Data quality

    Role

    Framework development · Senior Business Intelligence Developer

Architecture katas

Design studies, not delivered systems. Their figures are design assumptions.

  1. K—01

    Reliable data

    Featured

    Cold-Chain Telemetry and Compliance Platform

    20,000 refrigerated vehicles, 72-hour offline gaps, and GDP reports that an auditor can verify instead of having to trust.

    • EMQX
    • Apache Kafka
    • Apache Flink
    • ClickHouse
    • Apache Iceberg
    • Ceph

    Sector

    Pharmaceutical logistics

  2. K—02

    Reliable data

    Featured

    Open Lakehouse with Data Contracts and a Shared Semantic Layer

    Six brands keep their query engines, shared data is written once as an open table, guarded by a contract in CI and one metric definition.

    • Apache Iceberg
    • Apache Polaris
    • Trino
    • OpenMetadata
    • Open Policy Agent
    • Apache Airflow

    Sector

    E-commerce group

  3. K—03

    Reliable data

    Regional Health Information Exchange

    Eight EHR systems into one FHIR record per patient, reversible patient matching, consent on every read, ten-year audit trail.

    • Apache Camel
    • Apache Kafka
    • HAPI FHIR
    • PostgreSQL
    • Splink
    • Open Policy Agent

    Sector

    Healthcare

  4. K—04

    Reliable data

    Enterprise Data Warehouse Migration to BigQuery

    Move a 60 TB SQL Server warehouse and 1,900 ETL packages to BigQuery in 12 months, wave by wave, with every table reconciled.

    • BigQuery
    • Dataform
    • Debezium
    • Apache Kafka
    • Cloud Composer
    • Cloud Storage

    Sector

    Retail data platform

All katas

Publications

Articles and talks on this topic.

  1. Blog

    Moving a 60 TB Warehouse to BigQuery Without Missing a Single Month-End Close

    ahmetkalafat.com

Working on something like this?

Contact