Manage DeploymentsRemote ExecutionAdvanced configuration

Set up custom timetable support for Remote Execution

Airflow 3
This feature is only available for Airflow 3.x Deployments.

When Dags use custom timetables that connect to external data sources, such as querying Snowflake for scheduling metadata, the scheduler must retrieve connection credentials from your secrets backend at schedule time.

When you configure Customer Managed Identity for a Remote Execution Deployment, the setup only authorizes the apiserver to access your cloud resources for reading task logs. To support custom timetables, the scheduler also needs authorization to access your secrets backend.

Without this configuration, the scheduler can’t retrieve connections from the secrets backend, causing errors like:

airflow.exceptions.AirflowNotFoundException: The conn_id `<connection-id>` isn't defined

Prerequisites

Authorize the scheduler

Extend your existing Customer Managed Identity configuration to include the scheduler service account. This is the same process used when you first configured workload identity for the apiserver.

No additional configuration is required. The default Customer Managed Identity setup for AWS uses a wildcard pattern in the IAM trust policy that authorizes all service accounts in the Deployment namespace, including the scheduler:

1"<clusterOIDCIssuerUrl>:sub": "system:serviceaccount:<deployment-namespace>:*"

If you specified individual service accounts instead of using a wildcard, add the scheduler service account to your IAM trust policy:

1"<clusterOIDCIssuerUrl>:sub": "system:serviceaccount:<deployment-namespace>:<deployment-namespace>-scheduler-serviceaccount"

Verify the configuration

After updating your workload identity configuration, verify that the scheduler can retrieve connections:

  1. Check the scheduler logs for authentication errors. Cloud-specific errors such as AADSTS700213: No matching federated identity record found (Azure) or AccessDenied (AWS/GCP) should no longer appear.
  2. Trigger a Dag that uses a custom timetable dependent on a connection from your secrets backend. The Dag should schedule without AirflowNotFoundException errors.

See also