Astro Observe delivers pipeline-aware data observability purpose-built for Airflow, giving your team visibility into the health, quality, and performance of your business critical data products across the modern data stack. This guide outlines the fundamental requirements and steps to onboard.
apache-airflow>=2.7.0)apache-airflow-providers-openlineage>=1.12.1) and OpenLineage client (openlineage-python>=1.38.0) specified in project dependencies (for example, in requirements.txt)Astronomer recommends using the latest possible OpenLineage provider version and the latest OpenLineage client version. The client can be updated regardless of your Airflow version. See Configure OpenLineage on Astro for more information and upgrade instructions.
Confirm your Deployments are running the recommended OpenLineage client and provider versions by checking your Astro project’s requirements.txt. See Configure OpenLineage on Astro for instructions on upgrading to the recommended client and provider versions.
A data product is a composition of assets that, taken together, deliver a result with business relevance.
For guidance on identifying data products in your organization, see how to identify data products in your organization.
In the Astro UI, navigate to Observe > Data Products. Select + Data Product and follow the prompts to define your product and choose assets.
See Create a data product for more information.
Add an SLA to your data product to monitor freshness or timeliness of your data product. See Create an SLA.
Set up alerts to be notified of SLA violations or potential failures. To monitor the SLA you just created, click the SLAs tab in your data product, and select the SLA you want to monitor. Click + Alert to set up a Data Product SLA Violation Alert. See Create an alert for details on alert configuration settings.
To allow others to create data products, SLAs, and monitors in Observe, grant the Observe Member or Observe Admin role to any teammates who need to set up and manage these features. See user permissions reference for instructions on managing roles.