Introducing Apache Airflow 2.9
The Airflow 2.9 release brings significant enhancements to user-favorite features like data-aware scheduling, dynamic task mapping, and object storage.
Ensuring Data Pipeline Integrity: A Comprehensive Guide to Testing Airflow DAGs
An introduction to testing strategies, best practices, and implementation techniques.
Introducing Cohort 2 of the Astronomer Champions Program for Apache Airflow!
Our beta cohort of 10 is now joined by 23 hand-selected individuals who, we believe, truly embody what it means to champion the Apache Airflow Project.
Leveraging Apache Airflow and Kubernetes for Data Processing
Get insights on how to use Apache Airflow and the Kubernetes Executor for data processing, along with proven best practices and tips for scaling your workloads.
Test Airflow Upgrades with the Astro CLI
The Local Upgrade Test command in the Astro CLI eliminates upgrade pains and ensures safe upgrades, allowing users to confidently identify and resolve compatibility issues, and DAG import errors.
Enhanced Astro Workspace Roles for more granular permissions
In this blog post, we will dive into the details of the Astro’s Role-Based Access Control (RBAC) and new Workspace Role updates, and explore improvements to popular use cases of Astro.
ETL in Airflow: A Comprehensive Guide to Efficient Data Pipelines
This guide covers everything you need to know about creating efficient data pipelines with Apache Airflow. Get tips to speed up your pipelines, improve their reliability, and make them easier to manage.
Run ETL with Astro and CrateDB Cloud in 30min - fully up in the cloud
Use Apache Airflow with CrateDB to run ETL processes, deploy with ease thanks to Astro and CrateDB Cloud.
Introducing Cosmos 1.0: the best way to run dbt Core in Airflow
Turn your dbt projects into entire Airflow DAGs with less than 10 lines of code.
6 Lessons Learned in Building Astronomer’s Developer Documentation
The Astronomer Approach to Clear and Effective Technical Documentation
Expert tips for monitoring the health and SLAs of your Apache Airflow DAGs
Metrics in the form of tables, charts, or graphs are useful for monitoring the health of your Airflow system and maintaining the SLA of your data pipelines. They provide a quick and easy way to identify issues and take corrective actions. While Airflow's built-in UI is a good starting point along with notifications, using Airflow’s REST API to extract metadata about your DAGs and tasks provides advanced observability options with an added overhead of cost and maintenance.
.jpg)
Advantages of Hosted Airflow for Your ETL Workflows
Maximize ETL efficiency with hosted Apache Airflow on Astro, not self-hosting open-source. Benefit from simplified infrastructure management, scalable elasticity, and dedicated support for your workloads.
