Airflow Provider Framework

Providers are Python packages containing all relevant modules for a specific service provider (ie. AWS, Snowflake, Google etc.)

Unlike Apache Airflow® 1.10, now Airflow 2.0 is delivered in multiple, separate but connected packages. The core of the Airflow scheduling system is delivered as apache-airflow package and there are around 60 Airflow Provider Packageswhich can be installed separately.

In Airflow 2.0, Providers are decoupled from core Airflow releases:

• Providers can now be independently versioned, maintained, and released
• Users just `pip install` the provider package and then import modules into their DAG files
• Providers can now be built from any public repository

The Astronomer Registry

Recently we announced The Astronomer Registry: a discovery hub for Airflow providers designed to bridge the gap between the Airflow community and the broader data ecosystem.

The Airflow providers framework has long been a valuable asset for the community. DAG authors benefit from access to a deep library of standard patterns while maintaining code-driven customizability; it enables a tight integration between Airflow and almost any tool in the modern data stack.

The easiest way to get started with providers and Apache Airflow® 2.0 is by using the Astronomer CLI. To make it easy you can get up and running with Airflow by following our Quickstart Guide.