Data Intermediate Cohort

Production data pipelines with Airflow

Design, ship, and operate reliable batch and streaming pipelines that survive contact with real traffic.

Duration
32 hours
Modules
3
Lessons
9
Format
Cohort

What this course is about

A cohort-based course for engineers who already write SQL and Python and want to operate pipelines that other teams trust. We move from a single notebook to a scheduled, monitored, retried, and documented pipeline — the kind production actually wants. By the end you have a portfolio pipeline running against a realistic dataset, plus the patterns to keep it running.

What you leave with

  • Build a production-grade pipeline end to end, including retries and SLAs
  • Operate a real pipeline for two weeks with the cohort and an on-call rotation
  • Ship a portfolio piece with monitoring, docs, and a data contract

Skills you practice

AirflowDagster basicsBackfill strategySLA contractsData quality gates

Prerequisites

  • SQL at an intermediate level
  • Python (functions, classes, basic typing)

Time by module

Minutes per module. Hover or focus a bar for the exact count. M1 is the first module in the syllabus.

Minutes per module for Production data pipelines with Airflow
CategoryMinutes
M164
M274
M372