Astronomer Blog

The latest insights from our team of experts

September 22, 2023

Ask Astro: An open source LLM Application with Apache Airflow

Michael Gregory Staff Ecosystem Engineer

A reference implementation and community dialogue for building production-grade day-2 operations for LLM applications

September 8, 2023

Change Data Capture with Airflow - Part 2

We look at some examples to implement CDC in Airflow using a legacy data pipeline with batch processing, and look at advanced use cases for near-real-time processing.

August 30, 2023

Comparing Data Orchestration: Databricks Workflows vs. Apache Airflow, Part 1

A detailed guide to building an ETL workflow in Databricks Workflows and Apache Airflow

November 23, 2021

Machine Learning Pipeline Orchestration

A complete guide to orchestrating machine learning in production.

November 19, 2021

How to Build a Modern Data Stack

Breaking down what a modern data stack means in practice. We discuss four core components, five reasons to set it up, and how to orchestrate it.

November 17, 2021

Apache Airflow vs. Apache Beam

Apache Airflow or Apache Beam? Or both, working together? Let's have a closer look at two popular data management open source tools.

October 28, 2021

Democratizing the Data Stack—Airflow for Business Workflows

Learn how Hightouch drives action in marketing & sales teams with Reverse ETL, SQL, and Apache Airflow

October 26, 2021

Machine Learning Pipelines: Everything You Need to Know

Learn what is the process of building a ML pipeline, what are the steps, and how to do it with Airflow and Astronomer.

October 19, 2021

What is Reverse ETL and How Can It Improve Data Flow?

Find out what is reverse ETL and how to use Census and Airflow together to improve data orchestration.

October 14, 2021

Airflow at BBC—Data Orchestration Solution in Media

A conversation with the BBC's Principal Data Engineer about how Apache Airflow helps them deliver personalized experiences to the audience.

October 12, 2021

Everything You Need to Know About Apache Airflow 2.2.0

It's alive! Discover the major Airflow 2.2.0 features including customizable timetables, deferrable tasks, Airflow standalone CLI command, and many more.

October 5, 2021

Big Data Architecture: Core Components, Use Cases, and Limitations

Is Big Data Architecture the answer to major business problems, or just a crucial piece of a bigger puzzle? Discover our insights on the topic in this short blog post!

Astronomer Blog

Try Astro Today