Trigger failover and failback | Astronomer Documentation

Trigger failover

Failover is also supported using the Astro API.

Running, scheduled, and event-triggered tasks may be impacted during the failover window. Tasks may fail and require a retry.

Open the DR tab

In the Astro UI, go to Settings > Clusters (Organization Settings > Clusters in the legacy UI), select your primary cluster, and open the Disaster Recovery tab. Confirm the Status indicator shows the primary region is active.

Initiate failover

On the Disaster Recovery tab, click Failover. Alternatively, open the cluster’s actions menu (⋯) at the top right of the page and select Failover to Secondary…. Follow the prompts to confirm.

The secondary cluster is promoted to active, and all Deployments and data become available in the secondary cluster.

Validate Deployments

After failover completes, check the health and status of your Deployments, especially mission-critical ones. Validate that your Dags and tasks are running as expected and retry any failures if necessary.

Trigger failback

After the primary region recovers, you can fail back to the original primary cluster.

Failback is also supported using the Astro API.

Running, scheduled, and event-triggered tasks may be impacted during the failback window. Tasks may fail and require a retry.

Open the DR tab

In the Astro UI, go to Settings > Clusters (Organization Settings > Clusters in the legacy UI), select your original primary cluster, and open the Disaster Recovery tab. Confirm the Status indicator shows the cluster has failed over to the secondary region.

Initiate failback

On the Disaster Recovery tab, click Failback. Alternatively, open the cluster’s actions menu (⋯) at the top right of the page and select Failback to Primary…. Follow the prompts to confirm.

Remote Execution in DR pairs

As with other Deployments, running, scheduled, and event-triggered tasks may be impacted during the failover window and might fail and require a retry.

Deployments that use Remote Execution are supported in DR cluster pairs. During failover, Astro promotes the orchestration plane in the secondary cluster and the Remote Execution API URL remains the same, so your Remote Execution Agents reconnect to the secondary cluster without changing their configuration.

Only Remote Execution Agents running version 1.8.0 and later detect that a failover occurred and reconnect automatically to the secondary cluster. Agents on earlier versions might need to be restarted after a failover.

The execution plane runs in your own infrastructure and isn’t part of Astro’s region failover. To keep tasks running after failover, plan for the following:

Run your Remote Execution Agents in a location that remains available during an outage of the primary Astro region, or have standby Remote Execution Agents in the same secondary region as your Astro orchestration plane. Agent infrastructure and its availability are your responsibility. See the Remote Execution shared responsibility model.
If you use AWS PrivateLink or Azure Private Link for agent connectivity, configure private connectivity to the secondary cluster separately. Private endpoints are region-specific and don’t transfer automatically during failover.

Universal Metrics Export in DR pairs

If you have Universal Metrics Export (UME) configured, the same UME configuration applies to both the primary and secondary clusters. Metrics exported from each cluster include a cloud_region attribute so you can distinguish data from each cluster in your metrics system.

After failover, update your UME settings if needed to reflect the new active cluster.