What is the TaskFlow API?
Prior to Airflow 2.0, Airflow did not have an explicit way to declare messages passed between tasks in a DAG.
XComs could be used, but were hidden in execution functions inside the operator.
The TaskFlow API is a functional API that allows you to explicitly declare message passing while implicitly declaring task dependencies.
TaskFlow API Features
TaskFlow API Functionality Includes:
- XCom Args, which allow task dependencies to be abstracted and inferred as a result of the Python function invocation
- A task decorator that automatically creates PythonOperator tasks from Python functions and handles variable passing
- Support for Custom XCom Backends