Apache Airflow 3 is finally here! This long awaited major release is packed with new features and improvements, from DAG versioning to event-driven scheduling to Airflow architecture changes.
This Quick Notes document contains a quick summary of all you need to know to get started developing DAGs in Airflow 3. You’ll learn:
- How to run Airflow 3 locally today and navigate the new UI (in dark mode, of course 😎)
- How to trigger backfills from the UI and API (including sample code)
- How DAG versioning works and how to set up a GitDagBundle
- How to write DAGs with the new @asset-oriented approach
- How to schedule a DAG based on messages appearing in a Kafka topic
- What architecture changes you need to be aware of in Airflow 3 and which features they enable
- What to look out for when upgrading from Airflow 2 to Airflow 3
All in a brief, handy reference document, made for the busy data engineer who just wants to get to writing pipelines.