TaskFlow API in Airflow 2.0

Watch Video On Demand

Hosted By

  • Viraj Parekh

Note: This webinar was recorded in March 2021 and while the information is still accurate, many exciting features (including more Airflow decorators) have been added since then. We recommend you also check out the more recent Writing Functional DAGs with Decorators webinar, our comprehensive guide Introduction to the TaskFlow API and Airflow decorators and the free Astronomer Academy module Airflow: Taskflow API.

Check out the code used in the above webinar and instructions to set up your own Xcom backend!

What is the TaskFlow API?

Prior to Airflow 2.0, Airflow did not have an explicit way to declare messages passed between tasks in a DAG.

XComs could be used, but were hidden in execution functions inside the operator.

The TaskFlow API is a functional API that allows you to explicitly declare message passing while implicitly declaring task dependencies.

TaskFlow API Features

TaskFlow API Functionality Includes:




Task decorator allows users to convert any Python function into a task instance using PythonOperator.

DAG decorator allows users to instantiate the DAG without using a context manager.


Custom Xcom Backends

The TaskFlow API supports a new `xcom_backend` parameter, which allows you to

Future Work

The TaskFlow API is a new Airflow feature, and will likely be expanded on in the future.

Development of additional decorators to support other operators is already ongoing.

The easiest way to get started with Apache Airflow 2.0 is by using the Astronomer CLI. To make it easy you can get up and running with Airflow by following our Quickstart Guide.

Ready to Get Started?

Get Started Free

Try Astro free for 14 days and power your next big data project.