Skip to main content

​
Trigger failover

Running, scheduled, and event-triggered tasks may be impacted during the failover window. Tasks may fail and require a retry. Failover can take up to one hour for all workloads to resume.
Failover is also supported using the Astro API and the Astro Terraform provider.
1

Open the DR tab

In the Astro UI, go to Settings > Clusters (Organization Settings > Clusters in the legacy UI), select your primary cluster, and open the Disaster Recovery tab. Confirm the Status indicator shows the primary region is active.
On Azure, check the replication lag on this tab before you fail over. If the Deployment image registry backlog isn’t near zero, the secondary cluster can be missing Deployment images and your tasks fail to start. Failover reverses replication, so the secondary cluster can’t recover that data afterward. Wait for the backlog to clear unless you’re responding to a real region outage.
2

Initiate failover

On the Disaster Recovery tab, click Failover. Alternatively, open the cluster’s actions menu (⋯) at the top right of the page and select Failover to Secondary…. Enter the name of the failover region to confirm, then click Initiate Failover.The secondary cluster is promoted to active, and all Deployments and data become available in the secondary cluster.
3

Validate Deployments

After failover completes, check the health and status of your Deployments, especially mission-critical ones. Validate that your Dags and tasks are running as expected and retry any failures if necessary.

​
Trigger failback

After the primary region recovers, you can fail back to the original primary cluster. Unlike failover, failback stops writes to the secondary cluster before promoting the original primary, which guarantees zero data loss.
Running, scheduled, and event-triggered tasks may be impacted during the failback window. Tasks may fail and require a retry. Failback can take up to one hour for all workloads to resume.
Failback is also supported using the Astro API and the Astro Terraform provider.
1

Open the DR tab

In the Astro UI, go to Settings > Clusters (Organization Settings > Clusters in the legacy UI), select your original primary cluster, and open the Disaster Recovery tab. Confirm the Status indicator shows the cluster has failed over to the secondary region.
2

Initiate failback

On the Disaster Recovery tab, click Failback. Alternatively, open the cluster’s actions menu (⋯) at the top right of the page and select Failback to Primary…. Enter the name of the primary region to confirm, then click Initiate Failback.

​
Disaster recovery status

The Status indicator on the Disaster Recovery tab reports which region is active and whether a transition is in progress: Failover Failed is a final state. Astro doesn’t retry automatically. Trigger the operation again, or contact Astronomer support if it continues to fail.

​
Remote Execution in DR pairs

As with other Deployments, running, scheduled, and event-triggered tasks may be impacted during the failover window and might fail and require a retry.
Deployments that use Remote Execution are supported in DR cluster pairs. During failover, Astro promotes the orchestration plane in the secondary cluster and the Remote Execution API URL remains the same, so your Remote Execution Agents reconnect to the secondary cluster without changing their configuration.
Only Remote Execution Agents running version 1.8.0 and later detect that a failover occurred and reconnect automatically to the secondary cluster. Agents on earlier versions might need to be restarted after a failover.
The execution plane runs in your own infrastructure and isn’t part of Astro’s region failover. To keep tasks running after failover, plan for the following:
  • Run your Remote Execution Agents in a location that remains available during an outage of the primary Astro region, or have standby Remote Execution Agents in the same secondary region as your Astro orchestration plane. Agent infrastructure and its availability are your responsibility. See the Remote Execution shared responsibility model.
  • If you use AWS PrivateLink or Azure Private Link for agent connectivity, configure private connectivity to the secondary cluster separately. Private endpoints are region-specific and don’t transfer automatically during failover.

​
Universal Metrics Export in DR pairs

If you have Universal Metrics Export (UME) configured, the same UME configuration applies to both the primary and secondary clusters. Metrics exported from each cluster include a cloud_region attribute so you can distinguish data from each cluster in your metrics system. After failover, update your UME settings if needed to reflect the new active cluster.