Supercharge dbt Orchestration with Astronomer Cosmos and Apache Airflow®

Cosmos, Astronomer’s open-source standard for orchestrating dbt with Airflow, continues to evolve—making data pipelines faster, smarter, and more observable. With over 21 million monthly downloads, Cosmos makes it easy to orchestrate dbt transformations as part of your Airflow pipelines, unifying how you manage data transformations alongside other key data engineering workflows within a transparent, open-source framework.

Introducing dbt Fusion Support in Cosmos: Next-Level Performance

Cosmos 1.11 alpha now features support for dbt Fusion, the next-generation dbt engine designed for speed, state-aware orchestration, and real-time development.

With dbt Fusion, Cosmos users benefit from:

• Blazing-fast performance: Faster parsing speeds for large dbt projects using Fusion’s Rust-based engine.
• Efficient Orchestration: State-aware execution builds only what's changed, significantly reducing warehouse costs and runtime.

“Cosmos has sped up our adoption of Airflow for orchestrating our dbt Core projects without requiring deep knowledge of Airflow. The greatest time-saver is using the Cosmos DbtTaskGroup, which dynamically creates Airflow tasks while maintaining the dbt model lineage and dependencies that we already defined in our dbt projects.”

— Karen Connelly, Senior Manager, Business Intelligence, System1

Get started with dbt Fusion and Cosmos:

You can run dbt Fusion with Cosmos in three simple steps:

1) Install Cosmos 1.11.0a1 using your preferred Python package installation method

pip install astronomer-cosmos --pre

2) Install dbt Fusion in your Airflow deployment.
Fusion is a standalone binary and must be installed in your Airflow environment. If you're using Astro, update your Dockerfile as follows:

USER root
RUN apt install -y curl
RUN curl -fsSL https://public.cdn.getdbt.com/fs/install/install.sh | sh -s -- --update

After installation, ensure your shell recognizes the new binary (see dbt Fusion install docs for details).

3) Configure Cosmos to use the dbt Fusion library
Update your DbtDag or DbtTaskGroup to use the dbt Fusion binary. This is an example updating your DbtDag:

DbtDag(
    ...,
    execution_config=ExecutionConfig(
        dbt_executable_path="/home/astro/.local/bin/dbt"
    )
)

For detailed, step-by-step instructions and advanced configuration options, visit the Cosmos dbt Fusion configuration guide.

Cosmos does not bundle or distribute dbt Fusion. In accordance with dbt Fusion’s licensing (ELv2), users must install the dbt Fusion binary themselves. Once installed, Cosmos will seamlessly recognize and orchestrate dbt Fusion projects.

Upgraded for Modern Data Platforms

Cosmos has evolved to meet the needs of today’s Airflow environments, including powerful upgrades to align with Airflow 3:

• Full Airflow 3 Support: Enjoy enhanced support for assets with the new @asset decorator and improved observability in a future-proof orchestration layer.
• Flexible execution modes: Run dbt models effortlessly on local, virtualenv, Docker, Kubernetes, AWS ECS, GCP Cloud Run, or Azure Container Instances.
• Expanded database support: Native integrations now include DuckDB, Oracle, and Teradata, ensuring versatile connection handling.
• Community-driven enhancements: Enjoy robust error handling, incremental dependency management, and enhanced exposure selectors—allowing you to orchestrate and monitor dbt exposures as Airflow tasks.

Explore the full details in our release notes.

Why Choose Cosmos?

Cosmos transforms your data operations, integrating seamlessly with Airflow for a unified DataOps experience:

• Complete visibility: Orchestrate dbt projects as observable Airflow DAGs or task groups,
• Operational Control: Monitor dbt execution in real-time directly from Airflow UI.
• Platform Independence: Cosmos runs on Astro, MWAA, or self-hosted Airflow without vendor lock-in.
• Future-Proofing: Stay ahead with built-in Airflow 3 and dbt Fusion support.

Astronomer is committed to supporting compatibility of all versions of dbt with Cosmos, including dbt Fusion. We plan to deliver full GA support once Fusion is generally available as well. We welcome your input—report issues or share your experience by opening an issue on GitHub!

Ready to dive in?

Start Your Journey with dbt Fusion and Cosmos Today

Follow our Getting Started guide for step-by-step instructions.

Experience Cosmos with a guided product tour on Astro, our managed DataOps platform.