> ## Documentation Index > Fetch the complete documentation index at: https://astronomer.io/docs/llms.txt > Use this file to discover all available pages before exploring further. # Configure node pools for Astro Private Cloud worker queues > Declare the node groups in a data plane cluster so Deployment users can place worker queues on specific nodes. A *node pool* is a named description of a node group that already exists in one of your data plane clusters. It records the node selector and tolerations that place a Pod on those nodes, and the largest worker those nodes can hold. Astro Private Cloud doesn't create or discover node groups. A cluster administrator declares each group once as a node pool, and Deployment users select a pool by name when they configure a [worker queue](https://astronomer.io/docs/astro-private-cloud/v-2-x/configure-worker-queues.md). You provision and maintain the underlying node group, including keeping enough nodes of the declared size and labels available. Node pools place worker queues only, not other Airflow components. > [!NOTE] > > **Astro Private Cloud 2.2** > > This feature was introduced in Astro Private Cloud 2.2. To access this feature, upgrade your Astro Private Cloud installation to 2.2 or later. ## Prerequisites * Astro Private Cloud 2.2.0 or later. See [Upgrade to a minor version](https://astronomer.io/docs/astro-private-cloud/v-2-x/upgrade-apc-minor-version.md). * A registered data plane cluster. See [Register a data plane](https://astronomer.io/docs/astro-private-cloud/v-2-x/register-data-plane.md). * A node group in that cluster, with a label you can match on. Taints are optional. * A role with node pool write permissions. Cluster Admin and System Admin can create, update, and delete pools; System Editor can update them. See [Permissions](#permissions). The worker queues feature flag doesn't gate node pools. The catalogue is available whether or not worker queues are turned on. The Astro CLI has no commands for managing node pools. Use the Astro UI or the APC API. ## Create a node pool > [tab] Astro UI > > 1. In the Astro Private Cloud UI, go to **Clusters** and select your cluster. > 2. Click the **Node Pools** tab. > 3. Click **Node Pool**. > 4. In **Name**, enter a name. Worker queues reference this name, which must be unique within the cluster. > 5. (Optional) In **Description**, enter a note for other administrators. It isn't used for scheduling. > 6. In **Node selector**, enter the label key and value that identify these nodes. Click **Add label** to match on more than one label. Leave it empty for a pool that can use any node. > 7. (Optional) Under **Tolerations**, click **Add toleration** for each taint on these nodes. See [Tolerations](#tolerations). > 8. (Optional) Set **Max CPU (millicores)**, **Max memory (MiB)**, and **Max ephemeral storage (MiB)**. See [Ceilings](#ceilings). > 9. (Optional) In **Nodes for worker queues**, enter how many of these nodes worker queues can use. > 10. Click **Add Node Pool**. > > ![The Add Node Pool form, showing name, description, node selector, tolerations, ceilings, and node count fields.](https://mintcdn.com/astronomer/NVMeGUwVk6PcG6HU/images/astro-private-cloud/node_pools_create_form.png?fit=max\&auto=format\&n=NVMeGUwVk6PcG6HU\&q=85\&s=6077a67975520225f89d6cdd74b0e234) > [tab] APC API > > ```graphql wrap theme={null} > mutation createNodePool($clusterId: Uuid!, $input: InputNodePool!) { > createNodePool(clusterId: $clusterId, input: $input) { > id > name > nodeSelector > maxCpu > maxMemory > } > } > ``` > > With these variables: > > ```json expandable wrap theme={null} > { > "clusterId": "", > "input": { > "name": "compute-optimized", > "description": "Nodes reserved for ETL workloads", > "nodeSelector": { > "node-role": "etl" > }, > "tolerations": [ > { > "key": "workload", > "operator": "Equal", > "value": "airflow", > "effect": "NoSchedule" > } > ], > "maxCpu": 15000, > "maxMemory": 30720, > "maxEphemeralStorage": 51200, > "maxNodeCount": 12 > } > } > ``` > > See [Manage node pools with the APC API](https://astronomer.io/docs/astro-private-cloud/v-2-x/houston-manage-node-pools.md) for the full `InputNodePool` field reference and every possible error. ### Tolerations Add a toleration only if the nodes carry taints. Each toleration takes a key, an operator, and optionally a value, an effect, and a number of seconds: * **Operator** is `Exists` or `Equal`. With `Exists`, the value field is disabled because `Exists` matches the key regardless of value. * Leave the key empty only with the `Exists` operator. An empty key matches every taint. * **Effect** is `NoSchedule`, `PreferNoSchedule`, `NoExecute`, or **Any effect** to match all three. * The seconds field applies only to the `NoExecute` effect. ### Ceilings The three maximums describe what a single node in this group can give a worker, after the kubelet and daemon sets take their share, so they're smaller than the instance type's nominal size. Each ceiling caps a single worker. Saving a worker queue that asks for more CPU, memory, or ephemeral storage than its pool allows is rejected, naming the Deployment, queue, field, requested value, and ceiling. The block is hard because a Pod larger than its node stays Pending permanently with no actionable error. Leave a ceiling empty to impose no pool-level limit on that dimension. The platform's own [resource limits](https://astronomer.io/docs/astro-private-cloud/v-2-x/configure-component-size-limits.md) still apply. The form shows the equivalent beneath each field. For example, `15000` for **Max CPU (millicores)** shows `15 cores`, and `30720` for **Max memory (MiB)** shows `30 GiB`. ### Node count and totals **Nodes for worker queues** multiplied by the three ceilings gives the pool's total budget. Two node pools that describe the same physical nodes should split the node count between them rather than each claiming all of them. Going over the total warns but doesn't block. A queue counts against the total at its maximum worker count, so a pool can be committed beyond its budget with nothing broken yet. Workers past the total wait for a node that never arrives. For a queue with a [worker range](https://astronomer.io/docs/astro-private-cloud/v-2-x/configure-worker-autoscaling.md), the maximum is its **Max workers**, even while it runs fewer. The over-commitment warning appears in: * An **Over total** badge in the **Committed** column of the node pool list. * A panel in the node pool form showing which queues are committed against the pool, largest first. * A warning in the Worker Queues section of any Deployment whose queues would push the pool over. ## Review node pools The **Node Pools** tab lists every pool on the cluster: ![The Node Pools tab on a cluster, listing node pools with their node selector, max pod size, node count, committed capacity, and worker queue count.](https://mintcdn.com/astronomer/NVMeGUwVk6PcG6HU/images/astro-private-cloud/node_pools_list.png?fit=max\&auto=format\&n=NVMeGUwVk6PcG6HU\&q=85\&s=debe2bc4ff238e5886e0893364bc909b) | Column | Description | | - | - | | Node Pool | The pool name, with its description beneath | | Node selector | The label keys and values, or `Any node without a selector` | | Max pod size | The CPU and memory ceilings, in cores and GiB | | Nodes | The declared node count | | Committed | What the worker queues on this pool add up to, against the pool's total | | Worker queues | How many worker queues reference this pool | | Created | When the pool was created | | Created by | Who created it | ## Edit a node pool > [tab] Astro UI > > Click **Edit** on the pool, change the fields you want, and click **Save Node Pool**. > > An update is a partial patch on the optional fields: a ceiling or toleration you leave alone keeps its stored value. To clear a ceiling, set it explicitly to empty. > [tab] APC API > > ```graphql wrap theme={null} > mutation updateNodePool($nodePoolId: Uuid!, $input: InputNodePool!) { > updateNodePool(nodePoolId: $nodePoolId, input: $input) { > id > name > maxCpu > pendingConvergenceDeploymentIds > } > } > ``` > > `name` and `nodeSelector` are non-null on `InputNodePool`, so every update must resend both. A call that sends only `{ "maxCpu": null }` fails schema validation. > > The optional fields behave as a partial patch, unlike `InputWorkerQueue` on `upsertDeployment`: > > * An optional field you omit keeps its stored value. > * An optional field you set to `null` is cleared. > > See [Manage node pools with the APC API](https://astronomer.io/docs/astro-private-cloud/v-2-x/houston-manage-node-pools.md) for the full field reference and every possible error. To fix a mis-set ceiling, clear it explicitly, because a pool in use can't be deleted and recreated. Renaming a pool is safe, because worker queues reference a pool by its identifier, not its name. A name that collides with another pool on the same cluster is refused. ### Lower a ceiling Lowering a ceiling below what an existing worker queue requests is refused. The error names every conflicting Deployment, queue, field, requested value, and new ceiling. Resize those queues or move them to another pool first. ### Change placement Changing a pool's node selector or tolerations doesn't affect Deployments that already use the pool until each one is rebuilt. Until then those workers keep selecting the old node group, while the **Node Pools** page shows the new label. After you save a placement change, the UI asks **Upgrade deployments?** and names how many Deployments haven't picked it up. Click **Upgrade** to rebuild exactly those Deployments. ![The Upgrade deployments? dialog shown after a node pool's placement changes, naming how many Deployments haven't picked it up yet.](https://mintcdn.com/astronomer/NVMeGUwVk6PcG6HU/images/astro-private-cloud/node_pools_upgrade_dialog.png?fit=max\&auto=format\&n=NVMeGUwVk6PcG6HU\&q=85\&s=1f92b464b94aba94fbcb9876a083dab9) > [!WARNING] > > If you're changing a pool's placement because the old node group is being decommissioned, upgrade the affected Deployments instead of waiting for them to converge. Workers still pointed at nodes that disappear can't be scheduled when they next restart. ## Delete a node pool > [tab] Astro UI > > Click **Delete** on the pool, then click **Delete Node Pool**. > > A pool that any worker queue references can't be deleted. The dialog shows how many queues use it, and the **Delete Node Pool** button is disabled. Move those queues to another pool, or remove them, first. > > ![The Cannot delete node pool dialog, shown when a node pool is still referenced by a worker queue, with the Delete Node Pool button disabled.](https://mintcdn.com/astronomer/NVMeGUwVk6PcG6HU/images/astro-private-cloud/node_pools_delete_blocked.png?fit=max\&auto=format\&n=NVMeGUwVk6PcG6HU\&q=85\&s=07eb826da24dd6bdc41235a8cae92ba1) > [tab] APC API > > ```graphql wrap theme={null} > mutation deleteNodePool($nodePoolId: Uuid!) { > deleteNodePool(nodePoolId: $nodePoolId) { > id > name > } > } > ``` > > A pool that any worker queue references can't be deleted. The error names the Deployments and queues that still use it. Move or remove those queues first. > > See [Manage node pools with the APC API](https://astronomer.io/docs/astro-private-cloud/v-2-x/houston-manage-node-pools.md) for the full error reference. Deleting an unused pool is immediate, and new worker queues can no longer target it. Node pools are hard-deleted, with no recovery. ## Permissions | Role | Read | Create | Update | Delete | | - | - | - | - | - | | System Admin | Yes | Yes | Yes | Yes | | System Editor | Yes | No | Yes | No | | System Viewer | Yes | No | No | No | | Cluster Admin | Yes | Yes | Yes | Yes | | Workspace Admin | No | No | No | No | | Deployment Admin or Editor | Yes, to populate the node pool selector | No | No | No | The underlying permissions are `system.nodePools.get`, `system.nodePools.create`, `system.nodePools.update`, and `system.nodePools.delete` at the system level, and the matching `cluster.nodePools.*` permissions scoped to a single cluster. A Cluster Admin holds the `cluster.nodePools.*` set on their own cluster and none on any other. See [Role and permission reference](https://astronomer.io/docs/astro-private-cloud/v-2-x/role-permission-reference.md). A user without node pool permissions who queries node pools directly gets a permission error, not an empty list. ## See also * [Configure worker queues](https://astronomer.io/docs/astro-private-cloud/v-2-x/configure-worker-queues.md) * [Manage node pools with the APC API](https://astronomer.io/docs/astro-private-cloud/v-2-x/houston-manage-node-pools.md) * [Configure component size limits](https://astronomer.io/docs/astro-private-cloud/v-2-x/configure-component-size-limits.md)