> ## Documentation Index
> Fetch the complete documentation index at: https://astronomer.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Set up HashiCorp Vault as your secrets backend

This page provides steps for using [HashiCorp Vault](https://www.vaultproject.io/) as a secrets backend for both local development and on Astro.

To do this, you will:

* Create an AppRole in Vault which grants Astro minimal required permissions.
* Write a test Airflow variable or connection as a secret to your Vault server.
* Configure your Astro project to pull the secret from Vault.
* Test the backend in a local environment.
* Deploy your changes to Astro.

If you use a different secrets backend tool or want to learn the general approach on how to integrate one, see [Configure a Secrets Backend](/docs/astro/secrets-backend).

## Prerequisites

* A [Deployment](/docs/astro/create-deployment) on Astro.
* [The Astro CLI](/docs/cli/v1.43/overview).
* A local or hosted Vault server. See [Starting the Server](https://learn.hashicorp.com/tutorials/vault/getting-started-dev-server?in=vault/getting-started) or [Create a Vault Cluster on HCP](https://developer.hashicorp.com/vault/tutorials/cloud/get-started-vault).
* An [Astro project](/docs/cli/v1.43/develop-project#create-an-astro-project).
* [The Vault CLI](https://www.vaultproject.io/docs/install).
* Your Vault Server's URL. If you're using a local server, this should be `http://127.0.0.1:8200/`.
* (Remote Execution Only) [Helm installed](https://helm.sh/docs/intro/install/)
* (Remote Execution Only) The `values.yaml` file from the **Register Agents** modal in your **Deployments**>**Agents** page.

If you don't already have a Vault server deployed but would like to test this feature, Astronomer recommends that you either:

* Sign up for a Vault trial on [HashiCorp Cloud Platform (HCP)](https://cloud.hashicorp.com/products/vault) or
* Deploy a local Vault server. See [Starting the server](https://learn.hashicorp.com/tutorials/vault/getting-started-dev-server?in=vault/getting-started) in HashiCorp documentation.

## Step 1: Create a policy and AppRole in Vault

To use Vault as a secrets backend, Astronomer recommends configuring a Vault AppRole with a policy that grants only the minimum necessary permissions for Astro. For Remote Execution Deployments, you can use any Vault authentication method you prefer, for example Kubernetes auth if your agents and Vault are running on Kubernetes.

To do this:

1. Run the following command to [create a Vault policy](https://www.vaultproject.io/docs/concepts/policies) that Astro can use to access a Vault server:

   ```sh wrap theme={null}
   vault auth enable approle
   vault policy write astro_policy - <<EOF
   path "secret/*" {
     capabilities = ["create", "read", "update", "patch", "delete", "list"]
   }
   EOF
   ```

2. Run the following command to [create a Vault AppRole](https://www.vaultproject.io/docs/auth/approle):

   ```sh wrap theme={null}
   vault auth enable approle
   vault write auth/approle/role/astro_role \
       role_id=astro_role \
       secret_id_ttl=0 \
       secret_id_num_uses=0 \
       token_num_uses=0 \
       token_ttl=24h \
       token_max_ttl=24h \
       token_policies=astro_policy
   ```

3. Run the following command to retrieve the `secret-id` for your AppRole:

   ```sh wrap theme={null}
   vault write -f auth/approle/role/<your-approle>/secret-id
   ```

   Save this value. You'll use this later to complete the setup.

## Step 2: Create an Airflow variable or connection in Vault

To start, create an Airflow variable or connection in Vault that you want to store as a secret. It can be either a real or test value. You will use this secret to test your backend's functionality.

You can use an existing mount point or create a new one to store your Airflow connections and variables. For example, to create a new mount point called `airflow`, run the following Vault CLI command:

```sh wrap theme={null}
vault secrets enable -path=airflow -version=2 kv
```

To store an Airflow variable in Vault as a secret at the path `variables`, run the following Vault CLI command with your own values:

```sh wrap theme={null}
vault kv put -mount=airflow variables/<your-variable-name> value=<your-value>
```

To store an Airflow connection in Vault as a secret at the path `connections`, first format the connection as a URI. Then, run the following Vault CLI command with your own values:

```sh wrap theme={null}
vault kv put -mount=airflow connections/<your-connection-name> conn_uri=<connection-type>://<connection-login>:<connection-password>@<connection-host>:<connection-port>
```

To format existing connections in URI format, see [Import and export connections](/docs/astro/import-export-connections-variables#using-the-astro-cli-local-environments-only).

<Warning>Don't use custom key names for your secrets. Airflow requires the key name `value` for all Airflow variables and the key name `conn_uri` for all Airflow connections as shown in the previous commands.</Warning>

To confirm that your secret was written to Vault successfully, run:

```sh wrap theme={null}
# For variables
vault kv get -mount=airflow variables/<your-variable-name>

# For connections
vault kv get -mount=airflow connections/<your-connection-name>
```

## Step 3: Set up Vault locally

<Tabs>
  <Tab title="Astro" id="astro">
    In your Astro project, add the [HashiCorp Airflow provider](https://airflow.apache.org/docs/apache-airflow-providers-hashicorp/stable/index.html) to your project by adding the following to your `requirements.txt` file:

    ```text title="requirements.txt" wrap theme={null}
    apache-airflow-providers-hashicorp
    ```

    Then, add the following environment variables to your `.env` file:

    ```text wrap theme={null}
    AIRFLOW__SECRETS__BACKEND=airflow.providers.hashicorp.secrets.vault.VaultBackend
    AIRFLOW__SECRETS__BACKEND_KWARGS={"connections_path": "connections", "variables_path": "variables",  "mount_point": "airflow", "url": "http://host.docker.internal:8200", "auth_type": "approle", "role_id":"astro_role", "secret_id":"<your-approle-secret>"}
    ```
  </Tab>

  <Tab title="Remote Execution" id="remote-execution">
    In your Astro project, add the [HashiCorp Airflow provider](https://airflow.apache.org/docs/apache-airflow-providers-hashicorp/stable/index.html) to your project by adding the following to your `values.yaml` file to set the secrets backend class to use the Vault provider and configure your secrets backend kwargs:

    ```yaml title="values.yaml" wrap theme={null}
    secretBackend: "airflow.providers.hashicorp.secrets.vault.VaultBackend"

    commonEnv:
      - name: AIRFLOW__SECRETS__BACKEND_KWARGS
        value: '{"connections_path": "connections", "variables_path": "variables", "config_path": null, "url": "<vault-url>", "auth_type": "approle", "role_id":"<your-approle-id>", "secret_id":"<your-approle-secret>"}'
    ```

    You need to run the Remote Execution Agent with Vault credentials to fetch from your secrets manager.

    <Tip>
      For more security, you can store sensitive Kwargs containing secret ID and app role ID in a secret:

      ```yaml title="values.yaml" wrap theme={null}
      commonEnv:
        - name: AIRFLOW__SECRETS__BACKEND_KWARGS
          valueFrom:
            secretKeyRef:
              name: airflow-secret-backend
              key: '{"connections_path": "connections", "variables_path": "variables", "config_path": null, "url": "<vault-url>", "auth_type": "approle", "role_id":"<your-approle-id>", "secret_id":"<your-approle-secret>"}'
      ```
    </Tip>
  </Tab>
</Tabs>

<Info>
  If you run Vault on HashiCorp Cloud Platform (HCP):

  * Replace `http://host.docker.internal:8200` with `https://<your-cluster>.hashicorp.cloud:8200`.
  * Add `"namespace": "admin"` as an argument after `url`.
</Info>

This tells Airflow to look for variable and connection information at the `airflow/variables/*` and `airflow/connections/*` paths in your Vault server. You can now run a Dag locally to check that your variables are accessible using `Variable.get("<your-variable-key>")`.

### (Optional) Authenticate to Vault with AWS Assume Role (STS)

If your Astro environment runs in AWS, you can authenticate to Vault using AWS IAM with an STS assume role instead of an AppRole. This lets Vault verify the assumed AWS IAM role rather than requiring you to manage a long-lived AppRole secret. For more details, see [Vault authentication with AWS Assume Role STS](https://airflow.apache.org/docs/apache-airflow-providers-hashicorp/stable/secrets-backends/hashicorp-vault.html#vault-authentication-with-aws-assume-role-sts) in the Apache Airflow documentation.

To use this authentication method, set `auth_type` to `aws_iam` and provide `assume_role_kwargs` with the IAM role to assume. For example:

```text wrap theme={null}
AIRFLOW__SECRETS__BACKEND=airflow.providers.hashicorp.secrets.vault.VaultBackend
AIRFLOW__SECRETS__BACKEND_KWARGS={"connections_path": "connections", "variables_path": "variables", "mount_point": "airflow", "url": "<your-hashicorpvault-url>", "auth_type": "aws_iam", "assume_role_kwargs": {"RoleArn": "arn:aws:iam::<account-id>:role/<role-name>", "RoleSessionName": "Airflow"}}
```

For the full list of supported parameters in `assume_role_kwargs`, see the [AWS STS `assume_role` reference](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sts/client/assume_role.html).

## Step 4: Deploy configuration

<Tabs>
  <Tab title="Astro" id="astro">
    1. Run the following commands to export your environment variables to Astro:

       ```sh wrap theme={null}
       astro deployment variable create --deployment-id <your-deployment-id> AIRFLOW__SECRETS__BACKEND=airflow.providers.hashicorp.secrets.vault.VaultBackend

       astro deployment variable create --deployment-id <your-deployment-id> AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_path": "connections", "variables_path": "variables", "mount_point": "airflow", "url": "<your-hashicorpvault-url>", "auth_type": "approle", "role_id":"astro_role", "secret_id":"<your-approle-secret>"}' --secret
       ```

    2. Run the following command to push your updated `requirements.txt` file to Astro:

       ```sh wrap theme={null}
       astro deploy --deployment-id <your-deployment-id>
       ```

    3. (Optional) Remove the environment variables from your `.env` file or store your `.env` file in a safe location to protect your credentials in `AIRFLOW__SECRETS__BACKEND_KWARGS`.
  </Tab>

  <Tab title="Remote Execution" id="remote-execution">
    1. Run the following command to update your Remote Execution Agent with your new configurations.

    ```sh wrap theme={null}
    helm upgrade astro-agent astronomer/astro-remote-execution-agent -f values.yaml
    ```
  </Tab>
</Tabs>

Now, any Airflow variable or connection that you write to your Vault server can be successfully accessed and pulled by any Dag in your Deployment on Astro.