Astro includes several features that enhance the Apache Airflow development experience, from dag writing to testing. To use these features, you might need to modify how you write your dags and manage the rest of your code.
Use this documentation to learn about the key differences between managing dags on Astro versus on other platforms.
To develop and run dags on Astronomer products, your dags must belong to an Astro project. An Astro project contains all files required to run your dags both locally and on Astronomer. In addition to dag files, an Astro project includes Python and OS-level packages, your Astro Runtime version, and any other files your workers need to access when you run tasks.
See Create an Astro project to learn more about how to create and run Astro projects.
When you migrate to Astro from another Apache Airflow service, there are a few differences to note with regards to how Astro handles versioning, upgrades, and runtime builds.
Dockerfile of your Astro project. Unlike with open source Airflow, upgrading Astro Runtime does not require you to manually migrate your metadata database. To upgrade your version of Airflow, you only have to change the Astro Runtime version listed in your project’s Dockerfile and rebuild your project. See Upgrade Astro Runtime for instructions on how to upgrade.There are two ways to run dags within the Astro ecosystem: Either in a local Airflow environment or in an Astro Deployment. Each has its own purpose in the development lifecycle.
Whether you’re building your project locally or deploying to Astro, you can run unit tests with the Astro CLI to ensure that your code meets basic standards before you run your dags. The Astro CLI includes a default set of unit tests which you can run alongside your own tests in a single sequence. See Test your dags for more information.
Astro includes several features that enhance open source Apache Airflow functionality.
In local Airflow environments, you can use the Airflow UI to check your dag runs, task logs, and component logs just as you would in any other Airflow environment.
In the Astro UI, you have access to the Dags page in addition to the Airflow UI. From here, you can manage dag and task runs for any Deployment in your Workspace.
Astronomer recommends using the UI that best fits the need your team. If you prefer managing your dags from a single place and find the way that the way the Astro page is designed helpful, you don’t need to use the Airflow UI. If you’re a longtime user of Apache Airflow, you might feel more comfortable in the Airflow UI and don’t need to use the Astro UI.
See View logs and Manage dag runs for more information.
Astro supports a set of alerting features that in many cases replace Apache Airflow SLAs or failure notification. There are some circumstances where Astronomer recommends configuring Astro alerts instead of Airflow SLAs or failure notifications because it can simplify your dag code and make it easier to manage alerts across multiple dags. See When to use Airflow or Astro alerts for your pipelines on Astro.
In addition to its commercial products, Astronomer maintains Cosmos an open source tool for orchestrating dbt Core projects from a dag. Cosmos gives you more visibility into every step of your dbt project and lets you use Airflow’s data awareness features with your dbt models. See the Cosmos documentation for more information.