Astronomer Adopts DAG Factory to Democratize Writing Data Pipelines
We are thrilled to announce that Astronomer is officially taking over Adam Boscarino’s DAG Factory, an open source project that allows DAGs to be generated from YAML files.
Astronomer Blog
The latest insights from our team of Apache Airflow® experts.
Introducing Cohort 3 of the Astronomer Champions Program for Apache Airflow®!
We are thrilled to introduce Cohort 3 of our Champions Program for Apache Airflow! Launched at the start of 2024, our diverse and experienced champions have paved the way for this latest group to make a lasting impact on the Apache Airflow® community.
Exploring Airflow 2.9 Features with Astronomer's 29 Days of Airflow 2.9 Series
Join Astronomer in a comprehensive review of Airflow 2.9's capabilities across 29 days. Discover significant updates from dynamic data scheduling to advanced task management, and see how they can enhance your data engineering strategies.
Day 2 Operations for LLMs with Apache Airflow®: Going Beyond the Prototype, Part 1
We discuss the challenges of LLM application development beyond the prototype and why Apache Airflow® was highlighted for its ability to enable day-2 operations for LLM applications.
Change Data Capture with Airflow - Part 2
We look at some examples to implement CDC in Airflow using a legacy data pipeline with batch processing, and look at advanced use cases for near-real-time processing.
Comparing Data Orchestration: Databricks Workflows vs. Apache Airflow®, Part 1
A detailed guide to building an ETL workflow in Databricks Workflows and Apache Airflow®
Debugging Airflow Made Easy: 3 Key Steps to Debug your Airflow DAGs
In this post, we’ll walk through some high level methods for easily debugging your DAGs
Orchestrating Machine Learning Pipelines with Airflow
In the realm of machine learning, managing workflows efficiently is paramount. One tool that has emerged as a game-changer in this space is Apache Airflow®.
Change Data Capture in Apache Airflow® - Part 1
A complete guide to handling CDC in Airflow with ease.
Introducing Airflow 2.7
The latest minor release includes several new features, such as automatic setup/teardown of tasks, built-in OpenLineage support, cluster activity view, fail-stop functionality, and more.
Leveraging Apache Airflow® and Kubernetes for Data Processing
Get insights on how to use Apache Airflow® and the Kubernetes Executor for data processing, along with proven best practices and tips for scaling your workloads.
Test Airflow Upgrades with the Astro CLI
The Local Upgrade Test command in the Astro CLI eliminates upgrade pains and ensures safe upgrades, allowing users to confidently identify and resolve compatibility issues, and DAG import errors.
