← Blog | AUG 1 2024

A Step-by-Step Guide to Automating Your Astro Infrastructure with the Astro Terraform Provider

Bas Harenslak Staff Architect

4 min read |

This tutorial demonstrates using the Astro Terraform Provider to automate
the onboarding process for a new team by creating and managing an Astro
workspace and deployment.

We recently released the official Astro Terraform Provider. Terraform is the industry-standard
infrastructure-as-code tool to automate the creation and management of
infrastructure.

The Astro Terraform provider brings significant value to organizations by
enabling automated, consistent, and scalable management of your Astro
infrastructure, reducing manual errors and improving efficiency. The
provider currently supports several resources including clusters,
workspaces, deployments, and team to workspace role mappings.

Getting Started With the Astro Terraform Provider

In this tutorial, I will guide you through using the Astro Terraform
provider to automate the onboarding of a new team by creating and managing
an Astro workspace and deployment. By the end of this tutorial, you will
have a fully automated setup that is reproducible and easily scalable to
more teams.

Create Your Terraform Working Directory

Create a folder for your Terraform project e.g. my-data-platform and
save the following code in a file named terraform.tf:

Insert your organization’s ID on line 11. The working directory will
contain all your Terraform code, and all Terraform commands will be run
from this directory.

Initialize the Terraform Working Directory

Run terraform init and you’ll see Terraform downloading and installing
the Astro Terraform provider to your local machine:

The versions and hashes of providers are stored in a (generated) file
.terraform.lock.hcl. Store this file in version control.

Authenticating with Astro

For any automated action on Astro, you need an API token to authenticate. API tokens
exist on different levels. Since we’re going to create a workspace, we
need an Organization API token with Organization Owner permissions. Create
a token and ensure it’s configured as an environment variable
ASTRO_API_TOKEN when running Terraform commands:


export ASTRO_API_TOKEN=...

Define Resources in Terraform

In a file main.tf, define two resources, an astro_workspace and an
astro_deployment. These resources will represent an Astro workspace and
Astro deployment, defined in Terraform code:

One of the key characteristics (and benefits) of using Terraform is that
it’s declarative. Take for example the line workspace_id = astro_workspace.my_first_tf_workspace.id; this tells Terraform to
configure the workspace ID in the deployment. This means the workspace
must be created first, producing an ID which is a generated value and
unknown at the time of writing. We don’t have to instruct Terraform to
create resources in a certain order, we only have to instruct what to
create. The resources above can be defined in any order. Terraform takes
the relationships between resources into account when deciding the order
of creating resources.

Define Outputs

And (optionally) in a file outputs.tf, define values you’d like to log
after creating the infrastructure. We’ll output the workspace and
deployment IDs:

Showing output values is not necessary in this case, but provides us
(humans) information about created resources.

Preview the Terraform changes

You should now have 3 files:

Run terraform plan to let Terraform create
an execution plan and preview the infrastructure changes that Terraform
will make. You’ll see the following text:

The key message to look for is “Plan: 2 to add, 0 to change, 0 to
destroy.”, which validates that we’re about to create two resources, which
are the workspace and deployment as defined in main.tf.

Apply the Terraform Plan

Next, run terraform apply and select yes to execute the plan. This
creates the Astro resources and will print their ids, as we defined in
outputs.tf:

Cleaning Up Terraform-Created Resources

Finally, to clean up the resources, run terraform destroy and select
yes:

The output shows two destroyed resources which are the workspace and
deployment that we first created.

Conclusion

Terraform is an invaluable tool to platform teams who manage
infrastructure for development teams. Just by running terraform apply,
we created a workspace and deployment in seconds. Since those resources
are defined in code, we can repeat those steps for additional teams and
get new teams up and running quickly. Manually configuring such resources
in a UI is error-prone and time-consuming so it always a good idea to
automate infrastructure when you plan on repeating work more than once.

For more information about the Astro Terraform provider, check out:

Additionally, Astronomer offers several other tools to support
infrastructure automation:

Astro CLI: /docs/astro/cli/overview
Astro REST API: /docs/api
Deployments as code: /docs/astro/manage-deployments-as-code
Astro GitHub integration: /docs/astro/deploy-github-integration

Build, run, & observe your data workflows.
All in one place.

Get Started Free