Run, monitor, and manage Airflow on Azure with Astronomer

Deploy Apache Airflow on Azure Kubernetes Services

Astronomer allows you to easily run, monitor, and scale Apache Airflow deployments on Azure. Whether you’re looking to move your existing ETL and ML pipelines to Airflow or just want a more stable AKS cluster to run your data pipelines on, we’ve got you covered.

  1. Seamless developer experience- Astronomer makes it easy to spin up, deploy code to, and configure isolated Airflow environments on your AKS cluster.
  2. DevOps-friendly experience- Astronomer includes a full Prometheus and Grafana monitoring stack, user permission control, and a flexible ElasticSearch/Fluentd/Kibana logging system for full-text log search.
  3. Enterprise-grade support and access to the leading Apache Airflow experts

Astronomer is available on the Microsoft Azure Marketplace! Get it today.

Why run Apache Airflow with Azure?

We believe that anything connected to data should be developer-led. Apache Airflow is a battle-tested solution for data orchestration used by thousands of companies around the world. Thanks to the vibrant open-source community surrounding the project, you can benefit from their support and experience instead of reinventing the wheel.

If your company deals with multiple pipelines across multiple teams Apache Airflow can help you:

  • Perform operational checkpoints
  • Kickoff pipelines
  • Monitor end-to-end execution
  • Improve the cooperation between teams
  • Eliminate data silos

Get in touch with our experts to see how Airflow on Astronomer can enhance your Azure experience.

Deploy Apache Airflow on AKS with Astronomer!

Using ADF with Airflow?

If you’re using Azure Data Factory (ADF) for data orchestration, you can still benefit from Apache Airflow by combining the tools. On its own, ADF comes with certain disadvantages, like:

  • Difficult development and integration of custom tools
  • Limited integrations with services outside of Azure
  • Limited orchestration capabilities
  • Complex management of custom packages and dependencies

With Apache Airflow you can broaden your Azure experience. ADF jobs can be run using an Airflow DAG, giving the full capabilities of Airflow orchestration beyond using the Azure service alone. This allows users that are comfortable with ADF to write their jobs there, while Airflow acts as the control plane for orchestration.

If you want to learn more about using Airflow with ADF check out our webinar!

Powered By

Apache Airflow

Apache Airflow is an open-source tool to programmatically author, schedule and monitor your data pipelines. Learn why it's the industry standard!

Learn Why

Built on a strong and growing community.

Github Stars
Slack Users