Skip to main content

Astro CI/CD templates for GitLab

Use the following CI/CD templates to automate deploys to Astro from a GitLab repository.

Read the following sections to choose the right template for your project. The templates for GitLab include the image deploy templates and DAG deploy templates.

If you have one Deployment and one environment on Astro, use the single branch implementation. If you have multiple Deployments that support development and production environments, use the multiple branch implementation. If you want your CI/CD process to automatically decide which deploy strategy to choose, see DAG deploy templates.

To learn more about CI/CD on Astro, see Choose a CI/CD strategy.

Prerequisites

Each CI/CD template implementation might have additional requirements.

Image deploy templates

Use this template to push code to from a GitLab repository to Astro.

  1. Set the following environment variables in your GitLab project:

    • ASTRO_API_TOKEN: The value for your Workspace or Organization API token.
    • DEPLOYMENT_ID: The ID of your Astro Deployment. You can copy the ID from your Deployment's home page in the Astro UI.

    Astronomer recommends that you always mask your API token to prevent it from being accessible in plain text. You can also set the API token as an external secret for an extra layer of security.

  2. Go to Build > Pipeline Editor and commit the following:

astro_deploy:
stage: deploy
image: docker:latest
services:
- docker:dind

variables:
ASTRO_API_TOKEN: ${ASTRO_API_TOKEN}
DEPLOYMENT_ID: ${DEPLOYMENT_ID}

before_script:
- apk add --update curl && rm -rf /var/cache/apk/*
- apk add bash
script:
- (curl -sSL install.astronomer.io | bash -s)
- astro deploy -f $DEPLOYMENT_ID
only:
- main

DAG deploy templates

The DAG deploy template uses the --dags flag in the Astro CLI to push DAG changes to Astro. These CI/CD pipelines deploy your DAGs only when files in your dags folder are modified, and they deploy the rest of your Astro project as a Docker image when other files or directories are modified. For more information about the benefits of this workflow, see Deploy DAGs only.

info

If you stage multiple commits to DAG files and push them all at once to your remote branch, the template only deploys DAG code changes from the most recent commit. It will miss any code changes made in previous commits.

To avoid this, either push commits individually or configure your repository to Squash commits for pull requests that merge multiple commits simultaneously.

Single branch implementation

Use this template to push code to from a GitLab repository to Astro.

  1. Set the following environment variables in your GitLab project:

    • ASTRO_API_TOKEN: The value for your Workspace or Organization API token.
    • DEPLOYMENT_ID: The ID of your Astro Deployment. You can copy the ID from your Deployment's page in the Astro UI.

    Astronomer recommends that you always mask your API token to prevent it from being accessible in plain text. You can also set the API token as an external secret for an extra layer of security.

  2. Go to the Editor option in your project's CI/CD section and commit the following:

    astro_smart_deploy:
    stage: deploy
    image: docker:latest
    services:
    - docker:dind
    variables:
    ASTRO_API_TOKEN: ${ASTRO_API_TOKEN}
    DAG_FOLDER: "dags"
    DEPLOYMENT_ID: ${DEPLOYMENT_ID}
    before_script:
    - apk add --update curl && rm -rf /var/cache/apk/*
    - apk add git
    - apk add bash
    script:
    - (curl -sSL install.astronomer.io | bash -s)
    - files=$(git diff --name-only $(git rev-parse HEAD~1) -- .)
    - dags_only=1
    - echo "$DAG_FOLDER"
    - echo "$files"
    - for file in $files; do
    - echo "$file"
    - if [[ "$file" != "$DAG_FOLDER"* ]]; then
    - echo "$file is not a dag, triggering a full image build"
    - dags_only=0
    - break
    - else
    - echo "just a DAG"
    - fi
    - done
    - if [[ $dags_only == 1 ]]; then
    - echo "doing dag-only deploy"
    - astro deploy --dags $DEPLOYMENT_ID
    - elif [[ $dags_only == 0 ]]; then
    - echo "doing image deploy"
    - astro deploy -f $DEPLOYMENT_ID
    - fi
    only:
    - main

Was this page helpful?

Sign up for Developer Updates

Get a summary of new Astro features once a month.

You can unsubscribe at any time.
By proceeding you agree to our Privacy Policy, our Website Terms and to receive emails from Astronomer.