Develop DAGsKubernetesPodOperator

Run your Deployment's current Airflow image

Run the current Airflow image

You can run your Deployment’s current Airflow image with the KubernetesPodOperator by using the environment variable, ASTRONOMER_AIRFLOW_IMAGE. This environment variable allows you to run KPO tasks using the same Runtime image that your Deployment uses, in a way that won’t be affected by underlying cluster infrastructure changes that might change the base image repository URL. The ASTRONOMER_AIRFLOW_IMAGE environment variable allows you to ensure that your Deployment retrieves the correct image URL.

Setup

1

Add current Airflow image configuration

Add the ASTRONOMER_AIRFLOW_IMAGE environment variable to your KPO configuration. For example:

1import os
2from airflow.configuration import conf
3from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import KubernetesPodOperator
4from kubernetes.client import models as k8s
5KubernetesPodOperator(
6    namespace=conf.get("kubernetes", "NAMESPACE"),
7    image=os.getenv("ASTRONOMER_AIRFLOW_IMAGE"),
8    image_pull_secrets=[
9        k8s.V1LocalObjectReference("image-pull-secret"),
10        k8s.V1LocalObjectReference("per-dp-registry-pull-secret"),
11    ],
12    cmds=["<commands-for-image>"],
13    arguments=["<arguments-for-image>"],
14    labels={"<pod-label>": "<label-name>"},
15    name="<pod-name>",
16    task_id="<task-name>",
17    get_logs=True,
18    in_cluster=True,
19)