Manage DeploymentsRemote ExecutionCore configuration

Register and configure agents

Airflow 3
This feature is only available for Airflow 3.x Deployments.

Remote Execution Agents execute Airflow tasks in your Kubernetes infrastructure. This guide covers registering agents with your Astro Deployment and installing the Helm chart.

Prerequisites

  • Astro Deployment configured for Remote Execution mode. See Create a Deployment.
  • Kubernetes 1.30 or later
  • Helm 3 or later
  • Deployment API token with Deployment Admin role to pull the base Astro Remote Execution Agent Image

Step 1: Create agent token

The agent token authenticates your agent to the Astro orchestration plane. Create this token before installing the Helm chart.

Save the token value in a secure location immediately after creation. You cannot retrieve it again. The limit is 50 agent tokens per Deployment.

2

Open tokens view

Select the Remote Agents tab and toggle to the Tokens view.

3

Create token

  1. Click +Agent Token
  2. Enter a Name and Expiration period
  3. Optionally add a Description
  4. Click Create
4

Save token

Copy the agent token and save it securely. You will use this token in the Helm chart configuration.

Step 2: Install Helm chart

Astronomer recommends pulling both the Remote Execution Agent image and the Sentinel image and storing them in your private registry. Sentinel provides advanced monitoring and reporting for Remote Execution Agents, starting from version 1.2.0. The Agent base images are minimal, so you might need to add packages for your pipelines to function properly. Use either an Organization API token with the Org Owner role or a Deployment API token with the Deployment Admin role to authenticate.

1

Download values file

  1. In the Astro UI, go to the Remote Agents tab
  2. Toggle to the Agents view
  3. Click Register a Remote Agent
  4. Click Download to get the values.yaml file
2

Configure required values

Update the following values in values.yaml. All other values have working defaults.

You must configure these values before installing the Helm chart:

See the Helm chart comments and Helm chart configuration reference for descriptions of values.

3

Pull agent image for private registries

If self-hosting the image, log in to the image registry with your Deployment API token:

docker login images.astronomer.cloud -u cli -p <your-token>
Sentinel image available with 1.2.0 and later

Starting with Remote Execution Agent 1.2.0, a Sentinel image is published alongside the agent images to provide enhanced monitoring for Remote Execution Agents. The Sentinel image must be pulled separately and is not deployed by default. To use Sentinel, explicitly enable and configure the service in your values.yaml file.

After you log in, you can pull the Remote Execution Agent and optionally the Sentinel image directly. To find the latest version and image path, refer to the Remote Execution Agent release notes for all currently hosted images and Remote Execution Agent image reference for their full URLs. For example:

$docker pull images.astronomer.cloud/baseimages/astro-remote-execution-agent:3.1-3-python-3.12-astro-agent-1.2.0
$docker pull images.astronomer.cloud/baseimages/astro-remote-execution-sentinel:1.2.0
Configure scope for registry proxies

If you use JFrog Artifactory or a similar registry management tool to mirror or proxy images.astronomer.cloud, you need to configure specific include patterns instead of using the default **/* pattern.

The Deployment API token has limited scope and cannot fetch manifests for all repositories. Configure your remote registry to include only these specific paths:

  • baseimages/astro-remote-execution-agent
  • baseimages/astro-remote-execution-sentinel

Without these specific patterns, you might encounter 403 Forbidden errors when JFrog attempts to crawl all repositories in the registry.

Pull the Remote Execution Agent image, apply customizations that your dags require, and push it to your private registry. Then update the values.yaml file to reference your customized image.

4

Install Helm chart

Run the following commands to install the agent:

$helm repo add astronomer https://helm.astronomer.io
$helm repo update
$helm install astro-agent astronomer/astro-remote-execution-agent -f values.yaml

Step 3: Optionally set allowed IP ranges

Restrict Deployment access to specific IP address ranges for additional security or network isolation between environments.

1

Open Deployment settings

In the Astro UI, click the options menu for your Deployment and select Edit.

2

Add IP ranges

  1. In the Advanced section, click +Add IP.
  2. Enter an IP address range in CIDR format.
  3. Click Add.
  4. Repeat to add multiple ranges.

Step 4: Verify agent heartbeat

Confirm the agent is connected and healthy.

1

Check agent status

In the Astro UI, go to the Remote Agents tab. A healthy agent shows:

  • Health status: Healthy
  • Last heartbeat: Within the past minute

You can also verify locally that all agent client deployment Pods are running with kubectl get pods -n <namespace>. For more in-depth validation, check pod logs for heartbeat activity.

Temporarily remove any configured allowed IP ranges if the agent is not starting up and reporting Healthy.

2

Configure dag bundles

After verifying agent health, configure how agents access DAG code. See Configure DAG sources.

3

Run test dag

Trigger a test DAG run to verify the agent executes tasks successfully.

HTTP/HTTPS proxy server support

Starting with Remote Execution Agent 1.3.2, the agents support running behind an HTTP(S) proxy server. Configure proxy settings using the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables.

For Remote Execution Agent versions earlier than 1.3.2, proxy servers are not supported. If your Kubernetes environment automatically adds a proxy configuration to Pods, the agents will fail to establish an outbound connection to the orchestration plane. You might see errors similar to these in worker logs:

  • "exc_type":"ReadError","exc_value":"[Errno 104] Connection reset by peer"
  • "exc_type":"HTTPStatusError","exc_value":"Client error '400 Bad Request' for url ...

Workaround: Remove the proxy configuration from the agent Pods, or upgrade to Agent 1.3.2 or later.

Agent token configuration

Provide the agent token using one of these methods:

agentToken

Store the token directly in values.yaml:

1agentToken: "<your-token-value>"

Storing tokens directly in values files exposes them in version control. Use agentTokenSecretName or agentTokenFile for better security.

agentTokenSecretName

Reference an existing Kubernetes secret containing the token:

$kubectl -n <namespace> create secret generic agent-token \
>  --from-file=token=token.txt

In values.yaml:

1agentTokenSecretName: "agent-token"

agentTokenFile

Mount a file containing the token. The agent reads the token at runtime:

1agentTokenFile: "/path/to/token/file"

Image pull secret configuration

Configure image pull secrets to authenticate with your container registry. The configuration differs depending on whether you pull images directly from Astronomer’s registry or from a self-hosted registry.

The image pull secret requires an Astro API token, not an agent token. Use either an Organization API token with the Org Owner role or a Deployment API token with the Deployment Admin role. The agent token created in Step 1 authenticates the agent to the Astro orchestration plane and cannot be used for pulling images.

Use this configuration when pulling images directly from images.astronomer.cloud.

imagePullSecretName

Reference an existing Kubernetes secret in your namespace:

$kubectl create secret docker-registry -n <namespace> <secretName> \
>  --docker-server=images.astronomer.cloud \
>  --docker-username=cli \
>  --docker-password=<your-astro-api-token>

In values.yaml:

1imagePullSecretName: "<secretName>"

imagePullSecretData

Alternatively, provide Docker config JSON directly. The Helm chart creates a secret named image-pull-secret:

1imagePullSecretData: |
2  {
3    "auths": {
4      "images.astronomer.cloud": {
5        "auth": "<base64-encoded-credentials>",
6        "email": "<email>"
7      }
8    }
9  }

Manage Remote Execution Agents

You can take the following actions on your registered Remote Execution Agents:

  • Cordon: Cordoning a Remote Execution Agent marks it as unavailable for scheduling new tasks, while allowing it to continue running and complete any tasks already in progress.

This allows you to gracefully remove the Agent from service without interrupting current workloads. For example, you can cordon an Agent to delete or perform maintenance, such as an upgrade, on the Agent or underlying infrastructure.

A cordoned Agent will not receive new work, but it remains active until all running tasks have finished. Once ready to reintroduce the Agent to the task pool, it can be uncordoned to resume normal operation.

  • Uncordon: Uncordoning a Remote Execution Agent re-enables it to receive new tasks and resume normal scheduling.

  • Delete: Deletes the Remote Execution Agent from the Deployment.

Remote Execution Agent maintenance policy

Each Remote Execution Agent minor version is maintained for 6 months from the release month.

See Agent maintenance policy for more details about versioning, support, and upgrade recommendations.

Next steps

After registering agents, configure the required components: