Add custom metadata to OpenLineage assets

Astro Observe supports a custom OpenLineage facet, astroCustomMetadata, that you can use to attach links to assets sourced from OpenLineage, such as datasets and tasks. The links appear on the asset in the Details tab and in the Asset Overview in Lineage. Use them to surface contextual resources, such as documentation, dashboards, or runbooks, alongside your assets.

Astro Observe supports up to five links per asset.

Prerequisites

• A Deployment that emits OpenLineage events to Astro Observe. See Configure OpenLineage on Astro.
• A Dag that emits OpenLineage events for an Observe asset (supported for now: Airflow Task, OpenLineage Dataset).

Facet structure

Attach the facet using the facets keyword argument. The facet body contains an observeAssetMetadata array, where each item describes a single link.

Example

The following custom facet attaches one link to an OpenLineage dataset:

1
from airflow.providers.common.compat.openlineage.facet import Dataset as OLDataset
2
3
4
def observe_facet():
5
    return {
6
        "astroCustomMetadata": {
7
            "observeAssetMetadata": [
8
                {
9
                    "value": "https://www.astronomer.io/docs/astro/astro-observe",
10
                    "type": "url",
11
                    "typeParams": {"displayText": "Go to Astro Observe documentation"},
12
                },
13
            ],
14
        }
15
    }
16
17
18
# See https://openlineage.io/docs/spec/naming#dataset-naming
19
ol_dataset = OLDataset(
20
    name="<dataset-name>",
21
    namespace="<dataset-namespace>",
22
    facets=observe_facet(),
23
)
24
task = BashOperator(
25
    task_id="task",
26
    bash_command="exit 0;",
27
    outlets=[ol_dataset],
28
)

Use this dataset as an outlets value on any task that writes to it. The OpenLineage event for that task includes the dataset with this facet, and Astro Observe attaches the link to the corresponding dataset asset.

To attach the facet to a task run instead of a dataset, emit it as a custom run facet. See Custom facets in the Apache Airflow documentation.