You can create and manage environment variables for Deployments with the Astro Environment Manager in the Astro UI. The Environment Manager uses an Astro-managed secrets backend to store environment variable key-value pairs as Kubernetes Secrets.
Using the Environment Manager, you can quickly and securely create environment variables once and share them to multiple Deployments without having to set up your own secrets backend.
For example, you can configure an environment variable with credentials for a sandbox or development environment. Then, you can later configure your environment variable to be applied to all Deployments in the workspace by default. This means that when you create new Deployments, they automatically have access to your development environment. Later, you can edit the environment variable to point to your production resources by using value overrides.
When you create an environment variable in the Environment Manager instead of the Deployment UI, you can:
Workspace Owners and Operators can create and assign environment variables, while Workspace Authors can view configured environment variables and use them in Deployments. Learn more about user permissions.
When you create an environment variable in the Environment Manager, Astro stores environment variable details in an Astronomer-hosted secrets manager, and then applies environment variables to Deployments as Kubernetes Secrets. Specifically the following steps occur:
This process occurs every time you create or update an environment variable.
Environment variables marked as secret are stored securely by Astronomer and are not shown in the Astro UI. However, it’s possible for a user in your organization to create or configure a dag that exposes secret values in Airflow task logs. Airflow task logs are visible to all Workspace members in the Airflow UI and accessible in your Astro cluster’s storage.
To avoid exposing secret values in task logs, instruct users to not log environment variables in Dag code.
You can create environment variables both at the Deployment and Workspace level. When you create an environment variable at the Deployment level, the environment variable is available only to that specific Deployment. When you create an environment variable at the Workspace level, you can apply the environment variable to several Deployments and override its value as needed for each Deployment.
After you create an environment variable, you can reference it from Dag code using standard Python methods like os.getenv(). For more information and examples of using environment variables in your Dag code, see Using environment variable.
After you create an environment variable at the Workspace level, you can link it to multiple Deployments. Linking environment variables is useful for standardizing configuration across your entire team.
For the most flexibility, you can set default environment variables and override the environment variable values per-Deployment based on details like the Deployment’s usage and environment type (production or development).
You can configure Astro to link Workspace-level environment variables to all Deployments in the Workspace by default.
This is useful, for example, when you need to configure an environment variable for development environments that all Deployments in a Workspace should start with. Then, when you create new Deployments, they automatically have a default environment variable to your development resources.
When you’re ready to connect your Deployments to production resources, you can either replace the environment variable or override the environment variable values with your production resource information.
If you toggle the Automatically link to all deployments setting from Off to On, Astro respects any environment variable value overrides that you might have configured for existing linked Deployments.
If you create an environment variable at the Workspace level and link it to a Deployment, you can later edit the environment variable within the Deployment to specify a value override. When you override the value, you specify the value that you want to use for one Deployment, but not for others. This way, you can configure the environment variable a single time, but still have the flexibility to customize environment variables at the Deployment level.
For example, you might have created an environment variable that points to a development API endpoint, and then add a value override to specify a staging or production API endpoint for your staging and production Deployments to use.
When an environment variable is defined at both the Workspace and Deployment levels with the same key, the Deployment-level value always takes precedence. The Deployment environment variables page shows a unified view of both workspace and deployment environment variables, with clear indicators showing:
To override a workspace environment variable value for a specific Deployment:
Alternatively, you can override from the Workspace level: