Skip to main content

Set up AWS Systems Manager (SSM) Parameter Store

In this section, you'll learn how to use AWS Systems Manager (SSM) Parameter Store as a secrets backend on Astro.

If you use a different secrets backend tool or want to learn the general approach on how to integrate one, see Configure a Secrets Backend.


Step 1: Create Airflow secrets directories in Parameter Store

Create directories for Airflow variables and connections in Parameter Store that you want to store as secrets.

Variables and connections should be stored in /airflow/variables and /airflow/connections, respectively. For example, if you're setting a secret variable with the key my_secret, it should be stored in the /airflow/connections/ directory. If you modify the directory paths, make sure you change the values for variables_prefix and connections_prefix in Step 2.

For instructions, see the AWS Systems Manager Console, the AWS CLI, or the Tools for Windows PowerShell documentation.

Step 2: Set up Parameter Store locally

Add the following environment variables to your Astro project's .env file:
AIRFLOW__SECRETS__BACKEND_KWARGS={"connections_prefix": "airflow/connections", "variables_prefix": "airflow/variables", "role_arn": "<your-role-arn>", "region_name": "<your-region>"}

You can now run a DAG locally to check that your variables are accessible using Variable.get("<your-variable-key>").

Step 3: Deploy environment variables to Astro

  1. Run the following commands to export your secrets backend configurations as environment variables to Astro.

    $ astro deployment variable create --deployment-id <your-deployment-id>

    $ astro deployment variable create --deployment-id <your-deployment-id> AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_prefix": "airflow/connections", "variables_prefix": "airflow/variables", "role_arn": "<your-role-arn>", "region_name": "<your-region>"}' --secret
  2. (Optional) Remove the environment variables from your .env file or store your .env file in a safe location to protect your credentials in AIRFLOW__SECRETS__BACKEND_KWARGS.

Was this page helpful?

Sign up for Developer Updates

Get a summary of new Astro features once a month.

You can unsubscribe at any time.
By proceeding you agree to our Privacy Policy, our Website Terms and to receive emails from Astronomer.