NEW WEBINAR: 4 Things To Consider When Deciding How to Run Airflow|Register Today →

Automating Airflow Backfills with Marquez

When: June 21, 10:10 — 10:50 AM (EDT) at Open Source Summit North America

Hosted By

  • Willy Lulciuc Willy Lulciuc Senior Software Engineer


<blockquote> <em> Open Source Summit is a collection of events covering the most important technologies, topics, and issues affecting open source today. </em> </blockquote> <br>

As a data engineer, backfilling data is an important part of your day-to-day work. But backfilling interdependent DAGs is a time-consuming and often unpleasant experience.

Willy Lulciuc, Senior Software Engineer at Astronomer, will briefly introduce you to how backfills are handled in Airflow, then discuss how DAG lineage metadata stored in Marquez can be used to automate backfilling DAGs with complex upstream and downstream dependencies.

RSVP Today

By proceeding you agree to our Privacy Policy , our Website Terms and to receive emails from Astronomer.