> ## Documentation Index
> Fetch the complete documentation index at: https://astronomer.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Helm chart configuration reference
This reference describes configuration values for the Remote Execution Agent Helm chart. For complete configuration options, see the `values.yaml` file downloaded from the Astro UI.
## Required configuration values
The following values must be configured before installing the Helm chart:
### Agent authentication
**agentToken** / **agentTokenSecretName** / **agentTokenFile**
You must specify exactly one of these to provide the agent token generated in the Astro UI.
* **agentToken**: Token value as plain text in `values.yaml` (not recommended for production)
* **agentTokenSecretName**: Name of existing Kubernetes secret containing the token
* **agentTokenFile**: Path to file containing the token (agent reads at runtime)
See [Agent token configuration](/docs/astro/remote-execution-configure-agents#agent-token-configuration) for detailed instructions.
### Image registry access
**imagePullSecretName** / **imagePullSecretData**
You must specify exactly one of these to allow agents to pull images from the registry.
* **imagePullSecretName**: Name of existing Kubernetes secret with Docker credentials
* **imagePullSecretData**: Docker config JSON as string (Helm creates secret named `image-pull-secret`)
See [Image pull secret configuration](/docs/astro/remote-execution-configure-agents#image-pull-secret-configuration) for detailed instructions.
### Kubernetes namespace
**namespace**
Kubernetes namespace where the agent will be deployed.
* If `createNamespace: true`, Helm creates the namespace
* If `createNamespace: false`, namespace must exist before installation
If using `agentTokenSecretName` and `imagePullSecretName`, set `createNamespace: false` and create the namespace manually with secrets already present.
See [Install in restricted Kubernetes namespace](/docs/astro/remote-agents-restricted-kubernetes) for restricted namespace configuration.
### Resource name prefix
**resourceNamePrefix**
Name prefix for all Kubernetes resources (Deployments, ConfigMaps, Secrets) created by the Helm chart.
### Secrets backend
**secretBackend**
Airflow secrets backend class for accessing connections and variables. Required for agent operation.
Supported backends:
* `airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend`
* `airflow.providers.microsoft.azure.secrets.key_vault.AzureKeyVaultBackend`
* `airflow.providers.google.cloud.secrets.secret_manager.CloudSecretManagerBackend`
* `airflow.providers.hashicorp.secrets.vault.VaultBackend`
* `airflow.secrets.local_filesystem.LocalFilesystemBackend` (not recommended for production)
See [Configure secrets backend](/docs/astro/secrets-backend) for detailed configuration instructions.
### XCom backend
**xcomBackend**
Airflow XCom backend class for passing data between tasks. Required for agent operation.
Typically set to: `airflow.providers.common.io.xcom.backend.XComObjectStorageBackend`
See [Configure XCom backend](/docs/astro/remote-execution-configure-xcom-backend) for detailed configuration instructions.
### State store backend
**stateStoreBackend**
Airflow worker-side state store backend class for persisting task and asset state on Astro Runtime 3.3 and later. Available in Helm chart 2.3.0 and later.
Defaults to `airflow.providers.common.io.state_store.backend.StateStoreObjectStorageBackend` with a local file path. For production, point the path at shared object storage.
See [Configure state store backend](/docs/astro/remote-execution-configure-state-store-backend) for detailed configuration instructions.
### DAG bundles
**dagBundleConfigList**
JSON string defining how agents access dag code. Required for running dags.
See [Configure DAG sources](/docs/astro/remote-execution-configure-dag-sources) for detailed configuration instructions.
## Common environment variables
**commonEnv**
Environment variables applied to all agent components (worker, DAG processor, triggerer). Used to configure secrets backend parameters, XCom paths, logging settings, and other Airflow configuration.
Example:
```yaml title="values.yaml" wrap theme={null}
commonEnv:
- name: AIRFLOW__SECRETS__BACKEND_KWARGS
value: '{"connections_prefix": "airflow/connections", "variables_prefix": "airflow/variables"}'
- name: AIRFLOW__COMMON_IO__XCOM_OBJECTSTORAGE_PATH
value: "s3://bucket/xcom"
```
## Worker resource configuration
**workers**
Workers are configured as a list in `values.yaml`. Each entry defines a worker Deployment with its own name, resource allocation, replica count, and optional queue assignment.
| Parameter | Description | Default |
| ------------------------------- | ------------------------------------------------------------------------------------------------------------- | ---------------- |
| `name` | Unique identifier for the worker Deployment. Used in Kubernetes resource names. | `default-worker` |
| `replicas` | Number of worker Pod replicas. Ignored when `hpa.enabled` is `true`. | `1` |
| `queues` | Comma-separated list of Airflow queues this worker listens on. | `default` |
| `resources.requests.cpu` | Minimum CPU allocated to the worker Pod. | — |
| `resources.requests.memory` | Minimum memory allocated to the worker Pod. | — |
| `resources.limits.cpu` | Maximum CPU the worker Pod can use. | — |
| `resources.limits.memory` | Maximum memory the worker Pod can use. | — |
| `env` | List of environment variables specific to this worker. | `[]` |
| `volumes` | Additional volumes to mount on the worker Pod. | `[]` |
| `volumeMounts` | Mount paths for the additional volumes. | `[]` |
| `nodeSelector` | Kubernetes node selector for scheduling worker Pods. | `{}` |
| `tolerations` | Kubernetes tolerations for scheduling worker Pods. | `[]` |
| `serviceAccount.name` | Custom service account name. Overrides the default `{{ resourceNamePrefix }}-worker-{{ worker.name }}`. | — |
| `serviceAccount.create` | Whether the Helm chart creates the service account. Set to `false` when using a pre-existing service account. | `true` |
| `terminationGracePeriodSeconds` | The grace period for the worker Pod to finish existing tasks before terminating. | `600` |
Example with two workers:
```yaml title="values.yaml" wrap theme={null}
workers:
- name: default-worker
replicas: 2
queues: "default"
resources:
requests:
cpu: "500m"
memory: "1Gi"
limits:
cpu: "2"
memory: "4Gi"
- name: high-memory-worker
replicas: 1
queues: "high-memory"
resources:
requests:
cpu: "1"
memory: "4Gi"
limits:
cpu: "4"
memory: "16Gi"
```
When you configure multiple workers, each worker creates a separate Kubernetes Deployment. The service account name for each worker defaults to `{{ resourceNamePrefix }}-worker-{{ worker.name }}`. If you use IRSA (AWS), Workload Identity (GCP), or managed identity (Azure), annotate each worker's service account.
## Horizontal Pod Autoscaler
**workers\[].hpa**
Each worker supports a Horizontal Pod Autoscaler (HPA) configuration to automatically scale the number of worker Pod replicas based on resource utilization or custom metrics.
When `hpa.enabled` is `true`, the Helm chart creates a `HorizontalPodAutoscaler` resource for the worker Deployment. The `replicas` value is ignored because the HPA controls replica count.
| Parameter | Description | Default |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| `hpa.enabled` | Enable the Horizontal Pod Autoscaler for this worker. | `false` |
| `hpa.minReplicas` | Minimum number of worker Pod replicas. | `1` |
| `hpa.maxReplicas` | Maximum number of worker Pod replicas. | `10` |
| `hpa.metrics` | List of metric targets that the HPA uses to make scaling decisions. Follows the Kubernetes [HPA metrics spec](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/). | — |
Example with CPU-based autoscaling:
```yaml title="values.yaml" wrap theme={null}
workers:
- name: default-worker
queues: "default"
resources:
requests:
cpu: "500m"
memory: "1Gi"
limits:
cpu: "2"
memory: "4Gi"
hpa:
enabled: true
minReplicas: 1
maxReplicas: 5
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 80
```
You must set `resources.requests` for the metrics you use in HPA targets. For example, CPU-based autoscaling requires `resources.requests.cpu` to be set. Without resource requests, the HPA cannot calculate utilization percentages.
You can combine multiple metrics to define more sophisticated scaling behavior. The HPA evaluates all specified metrics and scales to the highest recommended replica count.
Example with CPU and memory metrics:
```yaml wrap theme={null}
hpa:
enabled: true
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 75
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 80
```
## Triggerer resource configuration
**triggerer**
The triggerer runs deferred tasks asynchronously. Configure the triggerer to control replica count, async capacity, resource allocation, and Pod-level settings.
| Parameter | Description | Default |
| -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | --------------------- |
| `replicas` | Number of triggerer Pod replicas. | `1` |
| `asyncSlots` | Number of concurrent triggers the triggerer Pod can compute. | `1000` |
| `image` | Docker image for the triggerer. Defaults to the top-level `image` if not set. | — |
| `imagePullPolicy` | Image pull policy for the triggerer. Defaults to the top-level `imagePullPolicy` if not set. | — |
| `resources.limits.cpu` | Maximum CPU the triggerer Pod can use. | `1` |
| `resources.limits.ephemeral-storage` | Maximum ephemeral storage the triggerer Pod can use. | `1Gi` |
| `resources.limits.memory` | Maximum memory the triggerer Pod can use. | `2Gi` |
| `resources.requests.cpu` | Minimum CPU allocated to the triggerer Pod. | `1` |
| `resources.requests.ephemeral-storage` | Minimum ephemeral storage allocated to the triggerer Pod. | `1Gi` |
| `resources.requests.memory` | Minimum memory allocated to the triggerer Pod. | `2Gi` |
| `env` | List of environment variables specific to the triggerer. | `[]` |
| `livenessProbe` | Liveness probe configuration for the triggerer Pod. | See following example |
| `readinessProbe` | Readiness probe configuration for the triggerer Pod. | See following example |
| `podSecurityContext` | Pod security context for the triggerer Pod. By default, the agent runs as a non-root user with UID 50000 and group ID 50000. | `~` |
| `containerSecurityContext` | Security context for the triggerer container. | `{}` |
| `initContainers` | Init containers to add to the triggerer Pod. | `[]` |
| `extraContainers` | Sidecar containers to add to the triggerer Pod. | `[]` |
| `volumes` | Additional volumes to mount on the triggerer Pod. | `[]` |
| `volumeMounts` | Mount paths for the additional volumes. | `[]` |
| `nodeSelector` | Kubernetes node selector for scheduling triggerer Pods. | `~` |
| `affinity` | Affinity rules for the triggerer Pod. | `{}` |
| `tolerations` | Kubernetes tolerations for scheduling triggerer Pods. | `[]` |
Example:
```yaml title="values.yaml" expandable wrap theme={null}
triggerer:
replicas: 2
asyncSlots: 500
resources:
limits:
cpu: "1"
ephemeral-storage: "1Gi"
memory: "2Gi"
requests:
cpu: "1"
ephemeral-storage: "1Gi"
memory: "2Gi"
livenessProbe:
httpGet:
path: healthz
port: 39091
initialDelaySeconds: 30
periodSeconds: 5
failureThreshold: 3
successThreshold: 1
timeoutSeconds: 5
readinessProbe:
httpGet:
path: healthz
port: 39091
initialDelaySeconds: 30
periodSeconds: 5
failureThreshold: 3
successThreshold: 1
timeoutSeconds: 5
podSecurityContext:
runAsUser: 50000
fsGroup: 50000
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containerSecurityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
initContainers:
- name: vault-agent
image: vault:1.13.0
command: ["vault", "agent", "-config=/etc/vault/config.hcl"]
volumeMounts:
- name: vault-config
mountPath: /etc/vault
env:
- name: VAULT_ADDR
value: "https://vault.example.com"
- name: VAULT_TOKEN
valueFrom:
secretKeyRef:
key: token
name: vault-token
extraContainers:
- name: logging-sidecar
image: timberio/vector:0.45.0-debian
env:
- name: VECTOR_CONFIG
value: /etc/vector/vector.yaml
volumeMounts:
- name: vector-config
mountPath: /etc/vector
resources:
limits:
cpu: "0.5"
memory: "256Mi"
requests:
cpu: "0.5"
memory: "256Mi"
volumes:
- name: task-logs
emptyDir: {}
- name: vector-config
configMap:
name: vector-config
volumeMounts:
- name: task-logs
mountPath: /var/log/airflow
readOnly: true
nodeSelector:
node.kubernetes.io/instance-type: c5.large
```
To run multiple triggerers, increase `replicas`. The configured number of replicas runs continuously. For restricted namespaces with Pod security standards set to restricted, configure `podSecurityContext` and `containerSecurityContext` to meet your cluster's requirements.
## Optional configuration
### Logging sidecar
**loggingSidecar**
Optional sidecar for exporting task logs to external platforms or viewing logs in the Airflow UI before task completion.
See [Configure logging sidecar](/docs/astro/remote-agents-logging-sidecar) for configuration instructions.
### OpenLineage
**openLineage**
Optional configuration for data lineage collection.
You must configure OpenLineage to use [Astro Observe](/docs/astro/astro-observe) with Remote Execution Deployments.
See [Configure OpenLineage](/docs/astro/remote-execution-configure-openlineage) for configuration instructions.
### Sentinel monitoring
**sentinel**
Monitoring service for agent health reporting (agent version 1.2.0+). Astronomer recommends enabling Sentinel for all deployments.
See [Sentinel for Remote Execution Agents](/docs/astro/remote-agents-sentinel) for configuration instructions.
### Cloud provider annotations
**annotations** and **labels**
Kubernetes annotations and labels to configure Pods to run using a specific IAM role (AWS), workload identity (GCP) or managed identity (Azure).
## Helm commands
After the Remote Execution Agent is installed, any updates to the agent use the `helm upgrade` command.
### Install agent
```sh wrap theme={null}
helm repo add astronomer https://helm.astronomer.io
helm repo update
helm install astro-agent astronomer/astro-remote-execution-agent -f values.yaml
```
### Update agent
```sh wrap theme={null}
helm upgrade astro-agent astronomer/astro-remote-execution-agent -f values.yaml
```
### View current configuration
```sh wrap theme={null}
helm get values astro-agent
```