How to pass data between your Airflow tasks

Recorded On April 11, 2023

Tamara Fingerlin Developer Advocate Astronomer
Kenten Danas Senior Manager, Developer Relations Astronomer

This webinar provides a deep dive into passing data between your Airflow tasks. We cover everything you need to know to choose and implement a method for passing data based on your use case, infrastructure, scale, and more. Questions covered in this session include:

What are some best practices to follow when using XComs, Airflow’s built-in cross-communication utility?
How can I pass data between tasks and DAGs using TaskFlow and traditional operators?
How does the Astro Python SDK use XComs to move data between relational stores and Python data structures, and how can it simplify my pipelines?
How can I set up a custom XCom backend and implement custom serialization methods?
How can I pass data between tasks that are run in isolated environments like the KubernetesPodOperator?

All code covered in this webinar can be found in this repo.