WEBINAR

Implementing Data Quality Checks in Airflow

Recorded On September 6, 2022

  • Kenten Danas
  • Benji Lampel

Please note: The webinar stream ended prematurely — about 40 minutes into the webinar, during the Q&A session — due to a technical error. We apologize for the inconvenience.

In this webinar, we discuss the benefits of the Common SQL provider package, a consistent, easy-to-use, and versatile set of operators for implementing data quality checks in your pipelines. In particular, we focus on the SQLColumnCheckOperator and SQLTableCheckOperator, both part of the provider package, and how they can work with OpenLinage.

The webinar shows you how to effectively use SQL for data quality checks, and answers questions like:

  • Why does the Common SQL provider exist and how does it work?
  • How do I implement column-level and table-level checks in my DAGs?
  • How does the Common SQL provider operate with OpenLineage?

All of the sample code shown in this webinar can be found in this repo.

See More Resources

Deliver Data with Precision: Supercharge your Airflows with Apache Airflow® on Astro - An Azure Native ISV Service

Dynamic Task Mapping on Multiple Parameters

DAG Writing Best Practices in Apache Airflow®

Best practices for managing Airflow across teams

Try Astro for Free for 14 Days

Sign up with your business email and get up to $500 in free credits.

Get Started

Build, run, & observe your data workflows. All in one place.

Build, run, & observe
your data workflows.
All in one place.

Try Astro today and get up to $500 in free credits during your 14-day trial.