This webinar covers task groups, a great Airflow feature that allow you to to organize complex DAG structures. While they were originally designed as a UI feature, task group functionality has been expanded to help with everything from task organization, to task state management, to modularizing common patterns in your pipelines. Using task groups is a beginner-friendly way to take your Airflow DAGs to the next level.
Watch the recording to learn everything you need to know to use task groups both in simple and advanced ways, including:
- How to create and manage task groups with classes and decorators.
- How to parameterize task groups.
- How to dynamically map task groups.
- How to modularize common patterns into task groups.
- How to use task groups included in provider packages like Cosmos and the Astro Databricks provider.
All code covered in this webinar can be found in this repo. You can also learn more about task groups in our Academy course and Airflow concept guide.
