Create environment variables in the Astro UI
You can create and manage environment variables for Deployments with the Astro Environment Manager in the Astro UI. The Environment Manager uses an Astro-managed secrets backend to store environment variable key-value pairs as Kubernetes Secrets.
Using the Environment Manager, you can quickly and securely create environment variables once and share them to multiple Deployments without having to set up your own secrets backend.
For example, you can configure an environment variable with credentials for a sandbox or development environment. Then, you can later configure your environment variable to be applied to all Deployments in the workspace by default. This means that when you create new Deployments, they automatically have access to your development environment. Later, you can edit the environment variable to point to your production resources by using value overrides.
When you create an environment variable in the Environment Manager instead of the Deployment UI, you can:
- Share the environment variable with multiple Deployments within the Workspace.
- Override the environment variable value for individual Deployments.
- Use environment variables in branch-based deploys and PR previews.
Workspace Owners and Operators can create and assign environment variables, while Workspace Authors can view configured environment variables and use them in Deployments. Learn more about user permissions.
How environment variables are stored
When you create an environment variable in the Environment Manager, Astro stores environment variable details in an Astronomer-hosted secrets manager, and then applies environment variables to Deployments as Kubernetes Secrets. Specifically the following steps occur:
- Astro stores the environment variable value in a secure secrets manager hosted by Astronomer.
- When an environment variable is assigned to a Deployment, Astro applies your environment variable as a Kubernetes Secret to your Deployment’s namespace.
- When your Deployment starts, the environment variable is loaded into the Airflow environment.
This process occurs every time you create or update an environment variable.
Environment variables marked as secret are stored securely by Astronomer and are not shown in the Astro UI. However, it’s possible for a user in your organization to create or configure a dag that exposes secret values in Airflow task logs. Airflow task logs are visible to all Workspace members in the Airflow UI and accessible in your Astro cluster’s storage.
To avoid exposing secret values in task logs, instruct users to not log environment variables in dag code.
Prerequisites
- Workspace Operator or Workspace Owner user permissions
- An Astro Deployment
- Astro Runtime 9.3.0 or greater
Create an environment variable
You can create environment variables both at the Deployment and Workspace level. When you create an environment variable at the Deployment level, the environment variable is available only to that specific Deployment. When you create an environment variable at the Workspace level, you can apply the environment variable to several Deployments and override its value as needed for each Deployment.
Create an environment variable at the Workspace level:
- In the Astro UI, click Environment in the left menu to open the Environment Variables page.
- Click + Environment Variable to add a new environment variable.
- Choose whether you want your environment variable to be Automatically link to all deployments by setting the toggle to On. See Link environment variables to Deployments for more information.
- Enter your information in the required fields.
- Choose whether you want your environment variable to be Not Secret or Secret. This allows you to store API keys, tokens, or other secrets in an environment variable without those credentials being available to future editors.
- Click Create environment variable.
- If you haven’t linked your environment variable to all deployments, make it accessible to individual Deployments. See Link environment variables to Deployments.
Create an environment variable at the Deployment level:
- In the Astro UI, select a Deployment, then click the Environment tab within the Deployment menu.
- Click Environment variables.
- Click Edit Variables.
- Enter an environment variable key and value. For sensitive credentials that should be treated with an additional layer of security, select the Secret checkbox. This permanently hides the variable’s value from all users in your Workspace.
- Click Update Environment Variables to save your changes. Your Airflow scheduler, webserver, and workers restart. After saving, it can take up to two minutes for new variables to be applied to your Deployment.
After you create an environment variable, you can reference it from dag code using standard Python methods like os.getenv(). For more information and examples of using environment variables in your dag code, see Using environment variable.
Link environment variables to Deployments
After you create an environment variable at the Workspace level, you can link it to multiple Deployments. Linking environment variables is useful for standardizing configuration across your entire team.
For the most flexibility, you can set default environment variables and override the environment variable values per-Deployment based on details like the Deployment’s usage and environment type (production or development).
- In the Astro UI, click Environment in the left menu to open the Environment Variables page.
- Click the environment variable you want to link to a Deployment.
- Click + Link Deployment.
- Choose a Deployment from the list that appears.
- (Optional) Click More options and then add any value overrides for this Deployment. For example, if your environment variable provides a development API key by default, you can override its value to instead provide a production API key.
- Click Link environment variable.
Configure environment variable sharing for a Workspace
You can configure Astro to link Workspace-level environment variables to all Deployments in the Workspace by default.
This is useful, for example, when you need to configure an environment variable for development environments that all Deployments in a Workspace should start with. Then, when you create new Deployments, they automatically have a default environment variable to your development resources.
When you’re ready to connect your Deployments to production resources, you can either replace the environment variable or override the environment variable values with your production resource information.
If you toggle the Automatically link to all deployments setting from Off to On, Astro respects any environment variable value overrides that you might have configured for existing linked Deployments.
- Click Environment in the main menu to open the Environment Variables page.
- Click the environment variable that you want to add per-Deployment overrides to.
- Toggle the Automatically link to all deployments setting to choose either:
- Off: Only share the environment variable individually to Deployments.
- On: Link to all current and future Deployments in this Workspace.
- (Optional) Change the default environment variable value.
- Click Update environment variable to save.
Override environment variable values
If you create an environment variable at the Workspace level and link it to a Deployment, you can later edit the environment variable within the Deployment to specify a value override. When you override the value, you specify the value that you want to use for one Deployment, but not for others. This way, you can configure the environment variable a single time, but still have the flexibility to customize environment variables at the Deployment level.
For example, you might have created an environment variable that points to a development API endpoint, and then add a value override to specify a staging or production API endpoint for your staging and production Deployments to use.
Environment variable precedence
When an environment variable is defined at both the Workspace and Deployment levels with the same key, the Deployment-level value always takes precedence. The Deployment environment variables page shows a unified view of both workspace and deployment environment variables, with clear indicators showing:
- The source of each environment variable (Workspace or Deployment)
- Override badges when a deployment environment variable overrides a workspace one
- Tooltips explaining the relationship between environment variables
To override a workspace environment variable value for a specific Deployment:
- In the Astro UI, select a Deployment, then click the Environment tab within the Deployment menu.
- Click Environment variables.
- Click Edit Variables.
- To override a workspace environment variable, add a new environment variable with the same key as the workspace environment variable but with a different value. The deployment value will take precedence.
- Click Update Environment Variables to save your changes.
Alternatively, you can override from the Workspace level:
- Click Environment in the main menu to open the Environment Variables page.
- Click the environment variable that you want to add per-Deployment overrides to.
- Click Edit to open the environment variable configurations for a specific linked Deployment.
- Switch the Override value toggle from No Override to Override.
- Add the override value.
- Click Update environment variable link.