We’re excited to announce that the Kubernetes (K8s) Executor is now available on Astro to customers in a Private Preview. A user can select the Kubernetes Executor in the Astro UI when creating a deployment.
Figure 1: Create a new Deployment and select the Kubernetes Executor from the dropdown in configuration.
While a Deployment will only use one executor at a time, a user can edit the Deployment’s executor via the Astro UI, and a Workspace or Cluster can have Deployments that run different Executors.
Figure 2: Edit an existing Deployment and switch to the Kubernetes Executor.
Some benefits of the K8s Executor include:
- Isolation: Each task runs in its own K8s Pod on a worker node. This means no dependency conflicts and no noisy neighbor problems.
- Resource requests: You can specify CPU and memory requests per task.
- Simplicity: Task pods autoscale, no concurrency configurations.
If you’re an existing customer and would like access to the Private Preview, get in touch with our team: product-previews@astronomer.io. If you’re not a customer and interested in learning more, schedule a demo.
Get a demo that’s customized to your unique data orchestration & Airflow pain points.
