dbt has become one of the most popular tools to define transformations in ELT pipelines, while Apache Airflow is the standard for workflow orchestration. Cosmos is an open-source package that helps you run dbt Core and dbt Fusion projects as part of your Airflow pipelines, so you can take advantage of Airflow features for your dbt transformations and have enhanced visibility into each step of your data pipelines.

If you’re using dbt and Airflow, Cosmos is the next step to run dbt projects as part of your Airflow pipelines. This eBook, written in two parts, part 1 for both beginners and part 2 for advanced users, offers a practical guide to Cosmos. From getting started in minutes to optimizing the performance of your projects, you’ll learn:

• How to set up Airflow and Cosmos for local development, including example dags for data warehouses such as Snowflake, Databricks and BigQuery
• How to deploy your Cosmos project to Astro
• Best practices for improving Cosmos parsing and execution performance
• Additional tips for running Cosmos, including how to set up dbt docs hosted in Airflow and upload dbt artifacts to blob storage

About Cosmos

Cosmos is a free open-source package developed by Astronomer that allows you to run your dbt projects as part of your Airflow dags. Since its release in 2022, Cosmos has grown thanks to a vibrant open-source community of 140+ contributors and it continues to evolve with new features and expanded support. Today, it serves as the recommended way for orchestration dbt projects in Airflow.