Astronomer Blog

The latest insights from our team of experts

September 22, 2023

Ask Astro: An open source LLM Application with Apache Airflow

Michael Gregory Staff Ecosystem Engineer

A reference implementation and community dialogue for building production-grade day-2 operations for LLM applications

September 8, 2023

Change Data Capture with Airflow - Part 2

We look at some examples to implement CDC in Airflow using a legacy data pipeline with batch processing, and look at advanced use cases for near-real-time processing.

August 30, 2023

Comparing Data Orchestration: Databricks Workflows vs. Apache Airflow, Part 1

A detailed guide to building an ETL workflow in Databricks Workflows and Apache Airflow

October 17, 2022

Orchestration, or How to Become a Data-Driven Company

Learn how Astronomer’s ambitious vision for data orchestration has led to a fully coordinated ecosystem that runs consistently and reliably, with minimal overhead and with full control, visibility, and governance of data operations.

October 11, 2022

Micropipelines: A Microservice Approach for DAG Authoring in Apache Airflow

Airflow 2.4 lets you break down big, monolithic pipelines — in which long-running tasks can delay time-sensitive ones — into “micropipelines” that let you tune your data ecosystem and make critical data products available on time.

October 4, 2022

Expanding Data Access and Exchange Inside a Company

How a technical ecosystem built by Astronomer’s data science team is empowering internal groups to own and share data.

September 29, 2022

Airflow 2.4 and Data-Driven Scheduling: How a New Feature Is Saving Time at Astronomer

The new data-driven scheduling functionality in Airflow 2.4 eliminates a lot of complexity and toil. See how Astronomer is applying this useful feature.

September 22, 2022

Astro CLI: The Easiest Way to Install Apache Airflow

Learn all about the Astro CLI — the free, open source tool that makes it easy to install, run, and test Apache Airflow from your command line in under five minutes.

September 19, 2022

Apache Airflow 2.4 — Everything You Need to Know

Airflow 2.4 is here, with a new datasets feature that augments Airflow with powerful data-driven scheduling capabilities. And much more!

September 14, 2022

Podcast Spotlight: What Observability Brings to Data Orchestration

Astronomer’s SVP of R&D talks about the “virtuous circle” that combining them creates for business.

September 9, 2022

Announcing Astro’s HIPAA and PCI-DSS Compliance

Two more reasons that users who need high levels of data security and protection can count on Astro.

September 1, 2022

How We Track the Growth of Apache Airflow

Find out how we use data to keep track of what’s happening in the Airflow project.

Astronomer Blog

Try Astro Today