A Step-by-Step Guide to Automating Your Astro Infrastructure with the Astro Terraform Provider
*This tutorial demonstrates using the Astro Terraform Provider to automate the onboarding process for a new team by creating and managing an Astro workspace and deployment.* We recently <a href="https://www.astronomer.io/docs/astro/release-notes?__hstc=181257784.4cd43b24e00de7ab7bf7c7673191368a.1717451742456.1722518094200.1722538840801.155&__hssc=181257784.4.1722538840801&__hsfp=214951083#automate-airflow-resource-and-infrastucture-management-with-the-astro-terraform-provider" target="_blank" rel="noopener">released</a> the official <a href="https://registry.terraform.io/providers/astronomer/astro" target="_blank" rel="noopener">Astro Terraform Provider</a>. <a href="https://www.terraform.io/" target="_blank" rel="noopener">Terraform</a> is the industry-standard infrastructure-as-code tool to automate the creation and management of infrastructure. The Astro Terraform provider brings significant value to organizations by enabling automated, consistent, and scalable management of your Astro infrastructure, reducing manual errors and improving efficiency. The provider currently supports several resources including clusters, workspaces, deployments, and team to workspace role mappings. ## Getting Started With the Astro Terraform Provider In this tutorial, I will guide you through using the Astro Terraform provider to automate the onboarding of a new team by creating and managing an Astro workspace and deployment. By the end of this tutorial, you will have a fully automated setup that is reproducible and easily scalable to more teams. ## Create Your Terraform Working Directory Create a folder for your Terraform project e.g. `my-data-platform` and save the following code in a file named `terraform.tf`: Insert your organization's ID on line 11. The working directory will contain all your Terraform code, and all Terraform commands will be run from this directory. ## Initialize the Terraform Working Directory Run `terraform init` and you'll see Terraform downloading and installing the Astro Terraform provider to your local machine: The versions and hashes of providers are stored in a (generated) file `.terraform.lock.hcl`. Store this file in version control. ## Authenticating with Astro For any automated action on Astro, you need an <a href="https://www.astronomer.io/docs/astro/automation-authentication?__hstc=181257784.4cd43b24e00de7ab7bf7c7673191368a.1717451742456.1722518094200.1722538840801.155&__hssc=181257784.4.1722538840801&__hsfp=214951083#step-1-create-an-api-token" target="_blank" rel="noopener">API token</a> to authenticate. API tokens exist on different levels. Since we're going to create a workspace, we need an <a href="https://www.astronomer.io/docs/astro/organization-api-tokens?__hstc=181257784.4cd43b24e00de7ab7bf7c7673191368a.1717451742456.1722518094200.1722538840801.155&__hssc=181257784.4.1722538840801&__hsfp=214951083" target="_blank" rel="noopener">Organization API token</a> with <a href="https://www.astronomer.io/docs/astro/user-permissions?__hstc=181257784.4cd43b24e00de7ab7bf7c7673191368a.1717451742456.1722518094200.1722538840801.155&__hssc=181257784.4.1722538840801&__hsfp=214951083#organization-roles" target="_blank" rel="noopener">Organization Owner permissions</a>. Create a token and ensure it's configured as an environment variable `ASTRO_API_TOKEN` when running Terraform commands: ```shell export ASTRO_API_TOKEN=... ``` ## Define Resources in Terraform In a file `main.tf`, define two resources, an `astro_workspace` and an `astro_deployment`. These resources will represent an Astro workspace and Astro deployment, defined in Terraform code: One of the key characteristics (and benefits) of using Terraform is that it's *declarative*. Take for example the line `workspace_id = astro_workspace.my_first_tf_workspace.id`; this tells Terraform to configure the workspace ID in the deployment. This means the workspace must be created first, producing an ID which is a generated value and unknown at the time of writing. We don't have to instruct Terraform to create resources in a certain order, we only have to instruct *what* to create. The resources above can be defined in any order. Terraform takes the relationships between resources into account when deciding the order of creating resources. ## Define Outputs And (optionally) in a file `outputs.tf`, define values you'd like to log after creating the infrastructure. We'll output the workspace and deployment IDs: Showing output values is not necessary in this case, but provides us (humans) information about created resources. ## Preview the Terraform changes You should now have 3 files: 1. terraform.tf 2. `main.tf` 3. `outputs.tf` (optional) Run <a href="https://developer.hashicorp.com/terraform/cli/commands/plan" target="_blank" rel="noopener">terraform plan</a> to let Terraform create an execution plan and preview the infrastructure changes that Terraform will make. You'll see the following text: The key message to look for is "Plan: 2 to add, 0 to change, 0 to destroy.", which validates that we're about to create two resources, which are the workspace and deployment as defined in `main.tf`. ## Apply the Terraform Plan Next, run `terraform apply` and select `yes` to execute the plan. This creates the Astro resources and will print their ids, as we defined in `outputs.tf`: ## Cleaning Up Terraform-Created Resources Finally, to clean up the resources, run `terraform destroy` and select `yes`: The output shows two destroyed resources which are the workspace and deployment that we first created. ## Conclusion Terraform is an invaluable tool to platform teams who manage infrastructure for development teams. Just by running `terraform apply`, we created a workspace and deployment in seconds. Since those resources are defined in code, we can repeat those steps for additional teams and get new teams up and running quickly. Manually configuring such resources in a UI is error-prone and time-consuming so it always a good idea to automate infrastructure when you plan on repeating work more than once. For more information about the Astro Terraform provider, check out: * <a href="https://registry.terraform.io/providers/astronomer/astro/latest/docs" target="_blank" rel="noopener">https://registry.terraform.io/providers/astronomer/astro/latest/docs</a> * <a href="https://github.com/astronomer/terraform-provider-astro" target="_blank" rel="noopener">https://github.com/astronomer/terraform-provider-astro</a> Additionally, Astronomer offers several other tools to support infrastructure automation: * Astro CLI: <a href="/docs/astro/cli/overview?__hstc=181257784.4cd43b24e00de7ab7bf7c7673191368a.1717451742456.1722518094200.1722538840801.155&__hssc=181257784.4.1722538840801&__hsfp=214951083" target="_blank" rel="noopener">/docs/astro/cli/overview</a> * Astro REST API: <a href="/docs/astro/api/v-1/overview" target="_blank" rel="noopener">/docs/astro/api/v-1/overview</a> * Deployments as code: <a href="/docs/astro/manage-deployments-as-code?__hstc=181257784.4cd43b24e00de7ab7bf7c7673191368a.1717451742456.1722518094200.1722538840801.155&__hssc=181257784.4.1722538840801&__hsfp=214951083" target="_blank" rel="noopener">/docs/astro/manage-deployments-as-code</a> * Astro GitHub integration: <a href="/docs/astro/deploy-github-integration?__hstc=181257784.4cd43b24e00de7ab7bf7c7673191368a.1717451742456.1722518094200.1722538840801.155&__hssc=181257784.4.1722538840801&__hsfp=214951083" target="_blank" rel="noopener">/docs/astro/deploy-github-integration</a>
Get started free.
OR
API Access
Alerting
SAML-Based SSO
Airflow AI Assistant
Deployment Rollbacks
Audit Logging
By proceeding you agree to our Privacy Policy, our Website Terms and to receive emails from Astronomer.