> ## Documentation Index
> Fetch the complete documentation index at: https://astronomer.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Launch a Pod in a GKE cluster on GCP

If some of your tasks require specific resources such as a GPU, you might want to run them in a different cluster than your Airflow instance. In setups where both clusters belong to the same Google Cloud project, you can manage separate clusters with roles and permissions.

This document shows how to configure a Google Kubernetes Engine (GKE) cluster on Google Cloud and run a Pod on it from an Airflow instance where cross-project access isn't available.

<Info>
  To launch Pods in external clusters from a local Airflow environment, you must have valid authentication for the external cluster. For managed Kubernetes services from public cloud providers, authentication is federated through the native IAM service. To grant the Astro role permissions to launch Pods on your cluster, you can either include static credentials or use workload identity to authorize the Astro role to your cluster.
</Info>

## Prerequisites

* Network connectivity between your Airflow execution environment and the external Kubernetes cluster:
  * **Hosted execution mode**: A [network connection](/docs/astro/networking-overview) between your Astro Deployment and the external cluster.
  * **Remote execution mode**: Network connectivity between the environment where your [Remote Execution Agent](/docs/astro/remote-execution-overview) runs and the external cluster. You are responsible for managing this connectivity. A direct network connection between Astro and the external cluster isn't required.

## Setup

<Steps>
  <Step title="Set up your external GKE cluster">
    Follow Google Cloud's documentation to prepare a GKE cluster that your Astro Deployment can authenticate to:

    1. [Create a GKE cluster](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster) if you don't already have one.
    2. Authorize your Astro Deployment to Google Cloud by following the [Deployment workload identity setup](/docs/astro/authorize-deployments-to-your-cloud?tab=gcp).
    3. Grant the service account [IAM and Kubernetes RBAC permissions](https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control) in the namespace where your `KubernetesPodOperator` tasks run.

       At a minimum, provision the following permissions for your service account in your specified namespace:

       * `container.clusters.get`
       * `container.events.list`
       * `container.pods.get`
       * `container.pods.getLogs`
       * `container.pods.list`
       * `container.pods.create`
       * `container.pods.delete`
       * `container.pods.update`

       If your Dag uses `do_xcom_push=True`, also grant the `container.pods.exec` permission.
  </Step>

  <Step title="Install dependencies in your Astro Runtime Docker image">
    To connect to your external GKE cluster, the [`gcloud` CLI](https://cloud.google.com/sdk/docs/install) and the [`gke-gcloud-auth-plugin`](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl#install_plugin) must be available inside your Astro Runtime image.

    Add the following to your `Dockerfile`:

    ```dockerfile title="Dockerfile" wrap theme={null}
    USER root

    RUN apt-get update && apt-get install -y apt-transport-https ca-certificates gnupg curl \
        && curl -sL https://packages.cloud.google.com/apt/doc/apt-key.gpg \
            | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg \
        && echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" \
            > /etc/apt/sources.list.d/google-cloud-sdk.list \
        && apt-get update \
        && apt-get install -y google-cloud-cli google-cloud-cli-gke-gcloud-auth-plugin \
        && rm -rf /var/lib/apt/lists/*

    USER astro
    ```

    For production deployments, consider pinning the `google-cloud-cli-gke-gcloud-auth-plugin` version for build reproducibility, or using a multi-stage build with the `google/cloud-sdk:slim` image to copy only the plugin binary into your final image and reduce its size.

    Add the following line to your `requirements.txt` to include the CNCF Kubernetes provider:

    ```text title="requirements.txt" wrap theme={null}
    apache-airflow-providers-cncf-kubernetes
    ```
  </Step>

  <Step title="Configure your `kubeconfig` file">
    The following sample Kubernetes `kubeconfig` file allows the Kubernetes command-line tool, `kubectl`, or other clients to connect to a remote Kubernetes cluster using Google Cloud workload identity for authentication.

    ```yaml title="kubeconfig" expandable wrap theme={null}
    # Specifies the version of the Kubernetes API for this configuration file.
    # v1 is the standard version used for kubeconfig files.
    apiVersion: v1
    # List of Kubernetes clusters that the configuration can connect to.
    clusters:
    - cluster:
        # base64-encoded certificate for the Kubernetes API server to verify SSL communication.
        certificate-authority-data: <base64-public-certificate>
        # Endpoint of the remote cluster you want to interact with.
        server: https://<cluster-endpoint>
      # Name of the cluster, which is referenced in the contexts section.
      name: <GKE cluster>
    # List of contexts that define which cluster and user combination to use when interacting with Kubernetes.
    contexts:
    # Describes the context for connecting to the cluster.
    - context:
        # References the cluster from the clusters section.
        cluster: <GKE cluster>
        # Associates the user configuration to be used for authentication with the cluster.
        user: <user>
      # The name of the context, which is referenced by current-context.
      name: <GKE cluster>
    # Specifies the active context that will be used by default when running kubectl commands.
    current-context: <GKE cluster>
    # Identifies the file type as a Kubernetes Config.
    kind: Config
    preferences: {}
    # List of users and the method they use for authentication.
    users:
    # Defines the user that is being used in the context.
    # This user is responsible for authenticating with the Kubernetes cluster.
    - name: <user>
      user:
        exec:
          apiVersion: client.authentication.k8s.io/v1beta1
          command: gke-gcloud-auth-plugin
          provideClusterInfo: true
    ```

    Fetch the `certificate-authority-data` and `cluster-endpoint` fields from the GKE cluster details page or using the Google Cloud SDK.

    <Note>
      **Scaling considerations**

      If you run a high number of concurrently deferred tasks against this connection, consider a static bearer token instead of the exec-based plugin shown in the preceding `kubeconfig`. At high triggerer concurrency, the plugin's subprocess invocation on each credential refresh can block the triggerer's shared event loop.
    </Note>
  </Step>

  <Step title="Create an Airflow connection to use the `kubeconfig` file">
    To use the `kubeconfig` file, create a new Kubernetes Airflow connection.

    There are multiple ways to pass the `kubeconfig` file to your Airflow connection. If your `kubeconfig` file contains any sensitive information, Astronomer recommends storing it as JSON inside the connection, as described in the JSON format tab.

    <Tabs>
      <Tab title="JSON format (Recommended)">
        Convert the `kubeconfig` file to JSON format and paste it into the **Kube config (JSON format)** field in the connection configuration.
      </Tab>

      <Tab title="External file in the default location">
        If the `kubeconfig` file resides in the default location on the machine (`~/.kube/config`), you can leave all fields empty in the connection configuration. Airflow automatically uses the `kubeconfig` from the default location. Add the following `COPY` command at the end of your Dockerfile to add your `kubeconfig` file inside your Astro Runtime Docker image:

        ```dockerfile wrap theme={null}
        COPY kubeconfig ~/.kube/config/airflow/kubeconfig
        ```
      </Tab>

      <Tab title="External file with a custom path">
        You can specify a custom path to the `kubeconfig` file by inserting the path into the **Kube config path** field of your Airflow connection. Add the following `COPY` command at the end of your Dockerfile to add your `kubeconfig` file inside your Astro Runtime Docker image:

        ```dockerfile wrap theme={null}
        COPY kubeconfig /usr/local/airflow/kubeconfig
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Configure your task">
    In your `KubernetesPodOperator` task, set `kubernetes_conn_id` to the connection you created, `namespace` to the namespace in your GKE cluster where the Pod should run, and `in_cluster=False` so that the operator uses the connection's `kubeconfig` instead of looking for an in-cluster service account.

    ```python wrap theme={null}
    from airflow.decorators import dag
    from airflow.providers.cncf.kubernetes.operators.pod import KubernetesPodOperator
    from pendulum import datetime


    @dag(
        dag_id="remote_kpo",
        start_date=datetime(2024, 1, 1),
        schedule=None,
        tags=["kubernetes", "gke"],
    )
    def remote_kpo():
        KubernetesPodOperator(
            task_id="run_on_gke",
            kubernetes_conn_id="<my-gke-connection>",
            namespace="<my-namespace>",
            image="ubuntu:latest",
            cmds=["echo"],
            arguments=["External KPO is working!"],
            name="example-pod",
            get_logs=True,
            in_cluster=False,
        )


    remote_kpo()
    ```
  </Step>
</Steps>