Astronomer Blog

The latest insights from our team of experts

September 22, 2023

Ask Astro: An open source LLM Application with Apache Airflow

Michael Gregory Staff Ecosystem Engineer

A reference implementation and community dialogue for building production-grade day-2 operations for LLM applications

September 8, 2023

Change Data Capture with Airflow - Part 2

We look at some examples to implement CDC in Airflow using a legacy data pipeline with batch processing, and look at advanced use cases for near-real-time processing.

August 30, 2023

Comparing Data Orchestration: Databricks Workflows vs. Apache Airflow, Part 1

A detailed guide to building an ETL workflow in Databricks Workflows and Apache Airflow

January 25, 2018

Improving Government Services With Apache Airflow: a Q&A With San Diego’s Chief Data Officer

Applying Airflow in the public sector to operationalize public data.

October 24, 2017

From Behavioral Analytics to Data Science With Astronomer

Being a stand-out company requires elevating product innovation within the organization and making sure that innovation isn’t reactive, but predictive.

October 10, 2017

Using Apache Airflow to Create Data Infrastructure in the Public Sector

When ARGO began exploring the technology required to build, operate, and maintain data infrastructure in the public sector, it’s no surprise they landed on Apache Airflow.

August 3, 2017

Data Formats 101

Business analysts generally encounter four main formats of data: JSON, XML, CSV, and TSV. So what are these types and why would we use them?

July 18, 2017

Why Every Data Scientist Needs a Data Engineer

The data scientist, the sexiest role of the 21st century, isn't actually very sexy. But it could be.

July 17, 2017

What Exactly Is a DAG?

What exactly is a DAG and what does it tell us that the term “data pipeline” can't?

June 28, 2017

Data Engineering Platform Astronomer Closes $3.5M Financing

The Astronomer platform collects, processes and unifies data, allowing organizations to scale analytics, data science and insights.

June 20, 2017

Normalizing Data for Warehouse Centralization

Knowing the options for storing data will help you make the right decisions for your company when you’re ready to take this step.

February 28, 2017

Apache Airflow and the Future of Data Engineering: A Q&A with Maxime Beauchemin

I reached out to Max about doing an interview post, and to my delight, he agreed. Here are thoughtful answers to questions about Airflow and data engineering.

Astronomer Blog

Try Astro Today