> ## Documentation Index
> Fetch the complete documentation index at: https://astronomer.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Troubleshoot data plane registration

Use this guide to diagnose and resolve errors that occur when you register a data plane cluster in Astro Private Cloud (APC) 1.x.

## "Commander Metadata service unavailable"

```text wrap theme={null}
Registration Failed
Cannot fetch dataplane metadata details. Reason: Commander Metadata service unavailable.
```

This error occurs when the control plane Houston API cannot reach Commander's `/metadata` HTTP endpoint on the data plane during the registration handshake. Registration cannot complete until Houston successfully fetches and validates this metadata.

### How registration works

When you register a data plane, Houston makes an outbound HTTPS `GET` request to Commander's metadata endpoint and validates the JSON response before creating the cluster record. The failure can occur at any point along that path: DNS resolution, network connectivity, TLS trust, or ingress routing.

### Correct metadata URL format

In APC 1.x, the chart creates two separate ingresses for Commander on the data plane:

| Ingress                      | Hostname                                | Port | Protocol | Purpose                           |
| ---------------------------- | --------------------------------------- | ---- | -------- | --------------------------------- |
| `commander-api-ingress`      | `commander.<domainPrefix>.<baseDomain>` | 443  | gRPC     | Houston-Commander control channel |
| `commander-metadata-ingress` | `<domainPrefix>.<baseDomain>`           | 443  | HTTPS    | Registration metadata endpoint    |

The metadata URL you enter during registration must point to the metadata ingress (the bare data plane domain). Houston appends `/metadata` to this URL internally when it makes the fetch call, so don't include the path in the registration form:

```text wrap theme={null}
https://<domainPrefix>.<baseDomain>
```

**Example**: If your control plane base domain is `apc.example.com` and your data plane `domainPrefix` is `dp-01`:

* **Correct**: `https://dp-01.apc.example.com`
* **Incorrect**: `https://commander.dp-01.apc.example.com` (this is the gRPC API ingress and returns a 404)

### Diagnose the error

Work through the following steps in order to isolate the cause.

#### Step 1: Verify Commander is running

On the data plane cluster, confirm Commander is healthy:

```bash wrap theme={null}
kubectl get pods -n <data-plane-namespace> -l component=commander
```

Expected output: `1/1 Running`. If Commander is crashlooping, check its logs:

```bash wrap theme={null}
kubectl logs -n <data-plane-namespace> -l component=commander
```

#### Step 2: Verify DNS resolves to the correct IP

The data plane has its own NGINX ingress controller with its own load balancer IP, separate from the control plane's load balancer. Confirm that the DNS for the data plane domain resolves to the data plane's load balancer, not the control plane's.

Find the data plane's load balancer IP:

```bash wrap theme={null}
kubectl get svc -n <data-plane-namespace> -l component=nginx
```

Look for the `EXTERNAL-IP` on the `LoadBalancer`-type service. In a co-located setup (control plane and data plane on the same cluster), the service name typically includes `-dp-nginx` and has a different IP from the control plane's NGINX service.

Then verify DNS:

```bash wrap theme={null}
dig +short <domainPrefix>.<baseDomain>
```

The returned IP must match the data plane's load balancer `EXTERNAL-IP`. A mismatch (for example, the data plane subdomain pointing to the control plane load balancer IP) causes all requests to return 404 because the control plane's NGINX has no ingress rules for the data plane's hostnames.

<Note>
  **Co-located deployments**

  When the control plane and data plane share a single cluster, they use separate namespaces and separate NGINX load balancers. DNS for `*.<domainPrefix>.<baseDomain>` must be a distinct wildcard record pointing to the data plane's load balancer IP. Verify this record exists and doesn't share the control plane's IP.
</Note>

#### Step 3: Test the metadata endpoint directly

After DNS resolves correctly, confirm the endpoint returns a valid 200 response:

```bash wrap theme={null}
curl -sk -w "\nHTTP Status: %{http_code}\n" \
  "https://<domainPrefix>.<baseDomain>/metadata"
```

A healthy response returns JSON similar to the following:

```json wrap theme={null}
{
  "kubernetesVersion": "v1.32.x",
  "mode": "data",
  "dataplaneChartVersion": "1.x.x",
  "cloudProvider": "GCP",
  "healthStatus": "HEALTHY",
  "baseDomain": "dp-01.apc.example.com",
  "commander": {
    "version": "1.x.x",
    "url": "commander.dp-01.apc.example.com:443",
    "status": "HEALTHY",
    "airflowChartVersion": "1.x.x"
  }
}
```

<Note>
  The `cloudProvider` field may return `"local"` on Kubernetes clusters and certain co-located environments. This is expected and doesn't affect registration.
</Note>

If this request returns a 404, continue to Step 4. If it times out or the connection is refused, skip to Step 5.

#### Step 4: Verify the metadata ingress exists and is configured correctly

Check that the metadata ingress exists in the data plane namespace:

```bash wrap theme={null}
kubectl get ingress -n <data-plane-namespace> | grep metadata
```

Describe it to verify the hostname and path:

```bash wrap theme={null}
kubectl describe ingress -n <data-plane-namespace> <release-name>-commander-metadata-ingress
```

Confirm the following:

* **Host** matches `<domainPrefix>.<baseDomain>` exactly.
* **Path** is `/metadata`.
* **Backend** points to the Commander service on port `8880` (the HTTP port, not the gRPC port `50051`).
* **Ingress class** annotation (`kubernetes.io/ingress.class`) matches the data plane's NGINX class name.

If the ingress is missing or misconfigured, re-run `helm upgrade` on the data plane release with the correct `global.plane.domainPrefix` and `global.baseDomain` values.

#### Step 5: Verify network connectivity from the control plane

Even if the endpoint works from your workstation, Houston on the control plane must also reach it. Test from inside the Houston Pod:

```bash wrap theme={null}
kubectl exec -it -n <cp-namespace> deploy/<cp-release>-houston -- sh
```

Inside the Pod, run:

```bash wrap theme={null}
wget -qO- https://<domainPrefix>.<baseDomain>/metadata
```

If this fails, check for firewall rules or network policies blocking HTTPS egress from the control plane to the data plane's load balancer IP.

#### Step 6: Verify TLS certificate trust

If the data plane uses a private certificate authority (CA) or a certificate chain that Houston doesn't trust, the HTTPS request fails with a TLS error. Test with certificate verification disabled:

```bash wrap theme={null}
# Inside the Houston Pod
wget --no-check-certificate -qO- https://<domainPrefix>.<baseDomain>/metadata
```

If this succeeds but the unmodified request fails, add the data plane's CA certificate to the control plane's `global.privateCaCerts` Helm values and re-deploy.

### Summary checklist

Before retrying registration, verify all of the following:

* Commander Pod is `1/1 Running` in the data plane namespace.
* `dig <domainPrefix>.<baseDomain>` returns the data plane's NGINX load balancer IP (not the control plane's).
* `curl https://<domainPrefix>.<baseDomain>/metadata` returns HTTP 200 with valid JSON.
* The metadata URL entered in the registration form is `https://<domainPrefix>.<baseDomain>` (no `/metadata` suffix, Houston appends that internally).
* The metadata ingress exists and its backend targets Commander's HTTP port (`8880`).
* Houston on the control plane can reach the data plane's load balancer over HTTPS.

## Related

* [Register a data plane cluster](/docs/astro-private-cloud/v-1-x/register-data-plane)
* [Data plane cluster overview](/docs/astro-private-cloud/v-1-x/overview-data-plane-cluster)
* [Debug an installation](/docs/astro-private-cloud/v-1-x/debug-install)