Introducing

Apache Airflow 2.0

The future of workflow orchestration is here.

What's in Airflow 2.0?

Check out our Airflow 2.0 video series for a walkthrough of all of the great new features.

Airflow, made better by Astronomer

Run production-grade Airflow out-of-the-box with Astronomer.

Apache Airflow just got new superpowers.

  • A New, Highly Available SchedulerNew

    Expect faster performance with near-zero task latency. Launch Scheduler replicas to increase task throughput and ensure high-availability. Read more

  • Full REST APINew

    Build programmatic services around your Airflow environment with Airflow's new API, now featuring a robust permissions framework. Airflow Docs

  • Smart SensorsNew

    Fit Airflow to event-driven paradigms with resource-saving sensors that operate as single, long-running tasks and don't take up a worker slot. Airflow Docs

  • Task Flow APINew

    Pass information between tasks with clean, efficient code that's abstracted from the task dependency layer. Includes support for custom XCom backends. Airflow Docs

  • Task GroupsNew

    Replace SubDAGs with a new way to group tasks in the Airflow UI. Task Groups don't affect task execution behavior and do not limit parallelism. Airflow Docs

  • Simplified KubernetesExecutorNew

    Make the most of the pod_override parameter for easy 1:1 overrides and the new yaml pod_template_file, which replaces configs set in airflow.cfg. Airflow Docs

  • Independent ProvidersNew

    Pull the latest version of any Airflow Provider at anytime, or follow an easy contribution process to build your own and install it as a Python package. Airflow Docs

  • UI/UX ImprovementsNew

    Embrace the a new look for the Airflow UI, now boasting a clean, modern color palette and an auto-refresh toggle in the Graph view. Airflow Docs

Read our Introducing Airflow 2.0 blog post and the official Apache Airflow 2.0 release announcement.

Apache Airflow 2.0 Dashboard

Do Airflow the easy way.

Run production-grade Airflow out-of-the-box with Astronomer.

dag

Apache Airflow Core Principles

Airflow is built on a set of core ideals that allow you to leverage the most popular open source workflow orchestrator on the market while maintaining enterprise-ready flexibility and reliability.

Flexible

Fully programmatic workflow authoring allows you you to maintain full control of the logic you wish to execute.

Extensible

Leverage a robust ecosystem of open source integrations to connect natively to any third party datastore or API.

Open Source

Get the optionality of an open-source codebase while tapping into a buzzing and action-packed community.

Scalable

Scale your Airflow environment to infinity with a modular and highly-available architecture across a variety of execution frameworks.

Secure

Integrate with your internal authentication systems and secrets managers for an platform ops experience that your security team will love.

Modular

Plug into your internal logging and monitoring systems to keep all of the metrics you care about in one place.

Airflow, made better by Astronomer

Start building your next-generation data platform