Apache Airflow 3.3: Context for your AI agent

Apache Airflow 3.3 introduces new features specifically designed to make Airflow more resilient and to make managing failures easier. This reference guide is designed as context for your AI agent so it can answer questions and generate code with full context on the latest version of Airflow.

The reference guide includes:

• Guidance on when to use the task state store, and partitioned asset scheduling, and retry policies
• Example code for customizable retry policies for different errors, including letting LLM’s decide whether to retry tasks at runtime.
• Guidance on when to use and how to implement fan-out and roll-up partitioned asset schedules, and the experimental Java and Go Task SDKs

You can upload this reference to your favorite AI agent and ask questions like:

• How do I avoid my task retrying 10 times for a SQL syntax error?
• Where in my Dags should I use Airflow 3.3 features?
• Should I use the task state store, the asset state store, or XCom to store how much of this data I processed?