This webinar provides an overview of how to use three new Airflow features and supporting open source community projects to simplify your DAG authoring process. We cover how to utilize these tools to write less code and design more straightforward DAGs, as well as use cases that most benefit from them. Questions covered in this session include:
- How can I use datasets to implement cross-DAG dependencies that have increased visibility?
- How can I use dynamic task mapping to create DAGs that dynamically update at runtime while also remaining scalable?
- How do I use the new Astro Python SDK to rapidly develop ELT workflows by turning my SQL queries and Python functions into Airflow tasks with no boilerplate code?
All of the code shown in this webinar can be found in this repo.
