WEBINAR

Efficient data quality checks with Airflow 2.7

Recorded On September 14, 2023

  • Tamara Fingerlin
  • Kenten Danas

Data quality is key to the health of an organization’s data ecosystem. With in-DAG quality checks, you can halt pipelines before bad data makes its way into production.

Implementing data quality checks efficiently can be challenging, but has recently gotten a lot easier with the addition of automatic setup/teardown tasks in Airflow 2.7. Setup/teardown tasks support the pattern of creating resources to run data quality checks and tearing them down once the checks are completed.

This webinar covers everything you need to know about setup/teardown tasks and how to use them to support your data quality checks, including:

  • Basic setup/teardown task concepts and usage.
  • Efficient DAG design for data quality checks using setup/teardown tasks.
  • A real world example showing how Astronomer uses this pattern in our own DAGs.

See More Resources

Airflow 3.0 Security Enhancements: Remote Execution and Task Isolation Deep Dive

The New DAG Schedule Parameter

How to build reliable data products with Astro

The State of Airflow 2025

Try Astro for Free for 14 Days

Sign up with your business email and get up to $500 in free credits.

Get Started

Build, run, & observe your data workflows. All in one place.

Build, run, & observe
your data workflows.
All in one place.

Try Astro today and get up to $500 in free credits during your 14-day trial.