Manage Dependencies Between Airflow Deployments, DAGs, and Tasks

Summary

More often that not, your Airflow components will have a desired order of execution particularly if you are performing a traditional ETL process—for example, before the Transform step in ETL, Extraction had to have happened in an upstream pipeline. In this webinar we will discuss how to properly set up dependencies and define an order of execution or operation for your pipelines using dependencies.

What we will Cover:

• Cross Deployment Dependencies
• Cross DAG Dependencies
• Basic Task Dependencies
• Trigger Rules
• Branching
• Dependencies within task groups
• Dependencies within dynamically generated tasks