Migrate to Astro Private Cloud 1.0 from 0.37

This guide provides instructions to migrate from Astro Private Cloud (APC) 0.37.6 to 1.0.0, including transitioning from STAN to JetStream and preparing your environment for Airflow 3 Deployments.

Prerequisites

  • Upgrade to APC 0.37.6 if you are on an earlier version.
  • Back up your platform database and configuration.
  • Verify that all platform and Airflow deployment Pods are healthy.
  • (Optional) Capture logs or metrics from your NATS and STAN instances.

If you experience any issues during your migration to 1.0, refer to the Debug migration guide.

Manual DNS and load balancer update required

When upgrading from Astro Private Cloud (APC) 0.x.x to 1.0.0, APC creates a new control plane NGINX ingress Service (astronomer-cp-nginx) and a new LoadBalancer. The previous ingress and LoadBalancer are replaced.

You must:

  • Update all DNS records to the new load balancer IP address.
  • Update firewall, allowlist, and security rules to point to the new public endpoint.
  • Re-issue or update any TLS/SSL certificates that reference the previous LoadBalancer hostname, if applicable.

This change occurs because the control plane ingress Service name changes from astronomer-nginx (0.x) to astronomer-cp-nginx (1.0), which causes Kubernetes to provision a new external LoadBalancer with a new public IP/hostname. Prepare these updates before performing the upgrade.

Example (your output will vary by cloud/provider):

1kubectl -n astronomer get svc astronomer-cp-nginx
2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
3astronomer-cp-nginx LoadBalancer 10.100.223.78 7666ac61ef6-1683718677.us-east-2.elb.amazonaws.com 80:32255/TCP,443:32647/TCP 25d

Step 1: Delete existing STAN and NATS StatefulSets

Before upgrading to version 1.0.0, you must migrate from STAN to JetStream. In your cluster, run the following commands to remove legacy STAN components before JetStream is initialized:

1kubectl delete sts <release-name>-stan
2kubectl delete sts <release-name>-nats --cascade=orphan

Step 2: Restart NATS and Houston Worker

After you complete the platform upgrade and all Pods are running, restart your workers to ensure the new JetStream components and Houston workers are connected and synchronized:

1kubectl rollout restart sts/<release-name>-nats
2kubectl rollout restart deploy/<release-name>-houston-worker

Step 3: Upgrade to APC 1.0.0 from 0.37.6

To upgrade to Astro Private Cloud (APC) 1.0.0 from version 0.37.6, you must first configure your deployment to run in unified plane mode. In 1.0.0, the control plane and data plane can run separately, but unified mode is recommended for upgrades and simplified operation. Unified mode does not prevent separate data planes from being added later.

Update your values.yaml with the following configuration:

1global:
2 plane:
3 mode: "unified" # Options: control, data, unified
4 domainPrefix: "" # Required only if running in data plane mode

Once this is set, perform a standard Helm upgrade using your existing release name and namespace:

$helm upgrade -f values.yaml -n astronomer astronomer astronomer/astronomer --version 1.0.0

This will upgrade your existing 0.37.6 installation to 1.0.0 on the same cluster. Make sure you’ve completed the pre-upgrade checks (backups, JetStream transition, etc.) before running the upgrade command.

Step 4: Airflow 3 configuration

Before you can create Airflow 3 Deployments, update your cluster’s override configuration to include the latest airflowLocalSettings snippet.

1

Update cluster configuration in the Astro UI

  1. Navigate to Clusters > [your cluster name] > Edit > Override Config.
  2. Add a JSON snippet defining the airflowLocalSettings and minimumAstroRuntimeVersion in the Override Config section.
1{
2 "helm": {
3 "airflow": {
4 "airflowLocalSettings":"<Pod mutation hook function>"
5 }
6 },
7 "airflowV3": {
8 "enabled": true,
9 "minimumAstroRuntimeVersion": "3.1-2"
10 }
11}
# This Pod mutation hook runs for all Pods dynamically created by Airflow in Kubernetes (K8s).
This includes K8s executor Airflow workers, which is an alternative executor to Celery, and K8s Pod Operator (KPO) Pods, which is an Airflow operator that launches a task in a K8s cluster.
This function is responsible for two things. It assigns labels to disambiguate between KPO and K8s executor Pods and it handles writing the entrypoint for K8s executor Pods.
# This function assigns labels to disambiguate between KPO and K8s executor Pods.
import warnings
from airflow.configuration import conf
from airflow.version import version
xcom_sidecar_image = "quay.io/astronomer/ap-init:3.21.3-5"
try:
from airflow.providers.cncf.kubernetes.utils.xcom_sidecar import PodDefaults
PodDefaults.SIDECAR_CONTAINER.image = xcom_sidecar_image
except ImportError:
warnings.warn(
"""
Airflow CNCF Provider not installed by default.
This will bypass custom images for xcom and defaults to alpine.
Refer https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/index.html
to install the provider.
"""
)
def pod_mutation_hook_deprecated(pod):
from airflow.contrib.kubernetes.pod import Pod
extra_labels = {
"kubernetes-pod-operator": "False"
}
if 'airflow-worker' in pod.labels.keys() and \
conf.get('core', 'EXECUTOR') == "KubernetesExecutor":
# extra_labels["kubernetes-executor"] = "True"
# By default, Airflow overwrites the entrypoint
# of KPO and K8s executor Airflow-worker pods.
# K8s worker pods are Airflow containers, and we can assume these
# Airflow containers are vendored by Astronomer. Astronomer provides
# an entrypoint in these containers to handle waiting for the network
# and the database to be ready, so we do not want this entrypoint
# to be overwritten. This helps with the stability of the K8s executor.
if "1.10" in version:
if not pod.args:
pod.args = []
pod.args = pod.cmds + pod.args
pod.cmds = ["tini", "--", "/entrypoint"]
else:
# In the case of KPO, we allow Airflow to overwrite the entrypoint
# because we do not know what the container will be (and it's probably
# not a container vendored by Astronomer, and it's definitely not
# an Airflow container).
extra_labels["kubernetes-pod-operator"] = "True"
extra_labels["kubernetes-executor"] = "False"
pod.labels.update(extra_labels)
def pod_mutation_hook_new(pod):
extra_labels = {
"kubernetes-pod-operator": "False"
}
if 'airflow-worker' in pod.metadata.labels.keys() and \
conf.get('core', 'EXECUTOR') == "KubernetesExecutor":
# extra_labels["kubernetes-executor"] = "True"
# By default, Airflow overwrites the entrypoint
# of KPO and K8s executor Airflow-worker pods.
# K8s worker pods are Airflow containers, and we can assume these
# Airflow containers are vendored by Astronomer. Astronomer provides
# an entrypoint in these containers to handle waiting for the network
# and the database to be ready, so we do not want this entrypoint
# to be overwritten. This helps with the stability of the K8s executor.
container = pod.spec.containers[0]
if "1.10" in version:
if not container.args:
container.args = []
container.args = container.command + container.args
container.command = ["tini", "--", "/entrypoint"]
pod.spec.containers[0] = container
else:
# In the case of KPO, we allow Airflow to overwrite the entrypoint
# because we do not know what the container will be (and it's probably
# not a container vendored by Astronomer, and it's definitely not
# an Airflow container).
extra_labels["kubernetes-pod-operator"] = "True"
extra_labels["kubernetes-executor"] = "False"
pod.metadata.labels.update(extra_labels)
def pod_mutation_hook(pod):
try:
pod_mutation_hook_new(pod)
except Exception:
pod_mutation_hook_deprecated(pod)
2

Apply and save configuration

Click Apply Changes to save your configuration.

3

Wait for configuration to propagate

Wait for configuration changes to propagate to all Pods before you proceed to make new Airflow 3 Deployments.

If you skip creating the cluster configuration override, your Airflow 3 Deployment creation will fail with the following error:

1File "/usr/local/airflow/config/airflow_local_settings.py", line 9, in <module>
2 from airflow.providers.cncf.kubernetes.utils.xcom_sidecar import PodDefaults
3ModuleNotFoundError: No module named 'airflow.providers.cncf'

Step 5: Upgrade all Airflow deployments

Once you have validated that all platform Pods in APC 1.0 are healthy and running, you must upgrade every Airflow deployment to ensure compatibility with 1.0.

There are multiple ways to trigger these upgrades:

  • If you only have a few deployments, set the astronomer.houston.upgradeDeployments.enabled=true Helm flag to automatically upgrade all Airflow deployments:

    $helm upgrade -f values.yaml -n astronomer astronomer astronomer/astronomer --set astronomer.houston.upgradeDeployments.enabled=true
  • If you have many Airflow deployments, you can upgrade them using one of the following approaches:

    • Upgrade each deployment manually from the Astro UI.
    • Use an automated mechanism such as the Houston API or the Astro CLI’s astro deploy command for programmatic or bulk upgrades.

Step 6: Post-upgrade validation

After you complete your upgrade, validate your upgrade works as expected by completing the following steps:

1

Confirm that NATS pods are running with JetStream enabled.

$kubectl -n astronomer get jobs | grep jetstream

Checks if jetstream job is created.

$kubectl -n astronomer get pods -lcomponent=nats

Checks if nats Pods are running

2

Verify Houston Worker pods are healthy and processing events.

$kubectl -n astronomer get pods -l component=houston-worker

Checks if Houston Worker Pods are in running state.

3

Create a test Airflow 3 Deployment to confirm initialization.

See Create a Deployment in the APC UI.

4

Check logs — there should be no remaining references to STAN.

$kubectl -n astronomer get statefulsets | grep stan

Command should return no results.