Use kubectl to administer Astronomer Software

Kubectl and Helm

Kubectl and helm are the two primary ways DevOps users will interact/administer the Astronomer Platform.

Setup

Both these tools are needed to deploy Astronomer onto a Kubernetes cluster. We also recommend using kubectx to simplify commands (the rest of this guide will use kubectx).

Base namespace and release

The initial helm install command to deploy Astronomer requires a namespace to deploy the base platform pods into.

This deploys a randomly named release of our helm charts into the datarouter namespace.

All pods can be listed in the namespace with kubectl.

Run helm ls to list all releases:

There is a release for Astronomer (single-chimp) and postgres (cautious-seal) in the datarouter namespace.

Creating new Airflow deployments

If you navigate to the Software UI and create a new deployment, it will create a new helm release in a new namespace.

The accurate-bolide-9914 helm release now lives in the a namespace generated by Astronomer named $basenamespace-release_name (datarouter-accurate-bolide-9914) and lives on the URL $release-name-airflow-BASEDOMAIN.

Switching into the datarouter-accurate-bolide-9914 namespace reveals the Airflow pods.

Since this deployment is running the Celery executor with 1 worker, there are pods for Redis and Flower.

Managing Pods

Airflow logs can be fetched directly from the underlying pods:

A full description of a pods status, resource request, and other data can be found with the describe command

Pods can also be deleted as a way to restart any Airflow component.

This will delete that copy of the pod and spin up a new one. All pods in an Airflow deployment are meant to be stateless, so deleting one and letting it recreate should not cause any

Other Resources

For additional kubectl commands, check out this Kubernetes Cheat Sheet.