You can create Astro Private Cloud Deployments with the APC API that use pre-created databases, external to the Airflow Deployment, as both a metadata storage and result storage backend.
In Astro Private Cloud 2.x, databaseManagement.manualConnectionStrings.enabled is a deployments.* setting. Cluster, Workspace, and Deployment overrides take precedence over values.yaml for deployments.* keys. See Configure Astro Private Cloud for the precedence rules.
Choose one of the following options based on the scope you need:
values.yaml and run a Helm upgrade.Cluster overrides apply to deployments.* values for the selected data plane cluster, so don’t include the deployments. prefix in the override.
In the Astro UI, open Clusters, select your data plane cluster, then click Edit on Configuration Override. See Update data plane cluster configurations.
Add the following to Configuration Override:
Click Update cluster to apply the change. The override is deep-merged with the cluster’s existing configuration.
Use this option only when no cluster has saved an override for deployments.databaseManagement.manualConnectionStrings.enabled. If a cluster already has a saved value for this key, you must update the cluster configuration as in Option A.
Open your values.yaml file.
Add the following under astronomer.houston.config:
Push the configuration change. See Apply a config change. If the astronomer-houston pods don’t roll automatically after the Helm upgrade, restart them manually so they pick up the new configuration.
The values.yaml settings in this step only take effect during the initial cluster installation. For clusters that are already registered, Houston resolves Deployment configuration directly from the cluster’s database record (Cluster.config.deployments) and ignores the Helm-derived ConfigMap. To enable manual connection strings on an existing cluster, apply the change through System Admin → Clusters → Edit → Cluster Deployment Configuration in Astronomer instead.
Substitute astro-db-name with your own database name, if you need to create a new database.
Substitute astro-user-name and astro-user-password with your information. You can use an existing database for this step.
USAGE and CREATE privileges on the public schema to astro-user-name:Now, go into the database you created, which is astro-db-name in this example, and run the following queries
Retrieve the connection information for your external database. For example, with AWS, you can retrieve your endpoint information by Finding the connection information for an RDS for MySQL DB instance.
You need connection strings that define how Astro Private Cloud configures the connection to your external databases from your Airflow Deployment. The values of these strings are used when you define your metadataConnection or resultBackendConnection when you create, update, or upsert your Deployment.
Use the values for your astro-user-name, astro-user-password, astro-db-name, and the host information you retrieved to compose the connection strings in the following format, depending on whether you want to define a result backend connection or a metadata database connection.
For PostgreSQL Deployments, PgBouncer is enabled by default in Astro Private Cloud. When PgBouncer is enabled, URI-style connection strings (postgresql://...) are rejected during upsert and you must use the JSON format (metadataConnectionJson and resultBackendConnectionJson). Use the URI tabs below only if you have explicitly disabled PgBouncer for your Deployment.
metadataConnection:
resultBackendConnection:
The connection string format validation regex don’t cover the resultbackend connection string format, which includes db+. This is specifically required for the Celery executor worker. If the connection string doesn’t include db+, then Celery worker pod fails. The regex validation isn’t implemented because it adds the complications on format validation logic in different scenarios.
If you have PgBouncer enabled, and are using Postgres, you must configure metadataConnectionJson and resultBackendConnectionJson instead. PgBouncer is enabled by default in Astro Private Cloud, so this is the typical path.
Use the values for your astro-user-name, astro-user-password, astro-db-name, and the host information you retrieved to compose the connection strings in the following format, depending on whether you want to define a result backend connection or a metadata database connection.
metadataConnectionJson:
resultBackendConnectionJson:
Use the APC API to create your Deployment configuration.
When you point a Deployment at an external database, you must set skipAirflowDatabaseProvisioning: true in the upsertDeployment mutation. Without it, Commander overwrites the host you passed with the platform database host before running helm install, and the Deployment silently lands on the wrong database instance with no error.
The following example shows the mutation and queries for using upsertDeployment. See APC API code examples for examples on how to use the update and upsert options for configuring your Deployment.
When you provide manual connection input, also set skipAirflowDatabaseProvisioning: true in your upsert payload. Without this flag, Commander overwrites the host value in your metadataConnection (or metadataConnectionJson) with the data plane database URL before running helm install, regardless of the host you submitted, and your Deployment silently lands on the platform database instead of your external one. Setting skipAirflowDatabaseProvisioning: true skips automatic database provisioning and preserves the host you provided.