Introduction to Observability for Data Pipelines

Whether your data pipelines power an AI-powered support chatbot, an eCommerce recommendation engine, or internal analytics dashboards, robust observability is critical. Knowing how to intervene if something goes wrong, whether your pipelines are running on time, and whether your data passes your quality requirements is crucial for getting value out of your data. Observability tools and features allow you to gain insights into the health and performance of your data pipelines, ensuring your data is accurate, fresh, and on time.

In this webinar, we’ll give an overview of data observability for Airflow, including topics like tracking data lineage, detecting anomalies, monitoring data quality, and getting actionable insights into the performance of your data operations. We’ll cover relevant OSS Airflow features like alerts, lineage, and data quality providers, as well as how to get comprehensive data observability on Astro.