Astro CI/CD templates for GitLab

Use the following CI/CD templates to automate deploys to Astro from a GitLab repository.

Read the following sections to choose the right template for your project. The templates for GitLab include the image deploy templates and dag deploy templates.

If you have one Deployment and one environment on Astro, use the single branch implementation. If you have multiple Deployments that support development and production environments, use the multiple branch implementation. If you want your CI/CD process to automatically decide which deploy strategy to choose, see Dag deploy templates.

To learn more about CI/CD on Astro, see Choose a CI/CD strategy.

Prerequisites

• An Astro project hosted in a GitLab repository.
• An Astro Deployment.
• A Deployment API token, Workspace API token, or Organization API token.

Each CI/CD template implementation might have additional requirements.

Image deploy templates

1
astro_deploy:
2
  stage: deploy
3
  image: docker:latest
4
  services:
5
    - docker:dind
6
7
  variables:
8
      ASTRO_API_TOKEN: ${ASTRO_API_TOKEN}
9
      DEPLOYMENT_ID: ${DEPLOYMENT_ID}
10
11
  before_script:
12
    - apk add --update curl && rm -rf /var/cache/apk/*
13
    - apk add bash
14
  script:
15
    - (curl -sSL install.astronomer.io | bash -s)
16
    - astro deploy -f $DEPLOYMENT_ID
17
  only:
18
    - main

Dag deploy templates

The dag deploy template uses the --dags flag in the Astro CLI to push dag changes to Astro. These CI/CD pipelines deploy your dags only when files in your dags folder are modified, and they deploy the rest of your Astro project as a Docker image when other files or directories are modified. For more information about the benefits of this workflow, see Deploy dags only.

Single branch implementation

Use this template to push code to from a GitLab repository to Astro.

1. Set the following environment variables in your GitLab project:

• ASTRO_API_TOKEN: The value for your Workspace or Organization API token.
• DEPLOYMENT_ID: The ID of your Astro Deployment. You can copy the ID from your Deployment’s page in the Astro UI.

Astronomer recommends that you always mask your API token to prevent it from being accessible in plain text. You can also set the API token as an external secret for an extra layer of security.

2. Go to the Editor option in your project’s CI/CD section and commit the following:

1
astro_smart_deploy:
2
  stage: deploy
3
  image: docker:latest
4
  services:
5
    - docker:dind
6
  variables:
7
    ASTRO_API_TOKEN: ${ASTRO_API_TOKEN}
8
    DAG_FOLDER: "dags"
9
    DEPLOYMENT_ID: ${DEPLOYMENT_ID}
10
  before_script:
11
    - apk add --update curl && rm -rf /var/cache/apk/*
12
    - apk add git
13
    - apk add bash
14
  script:
15
    - (curl -sSL install.astronomer.io | bash -s)
16
    - files=$(git diff --name-only $(git rev-parse HEAD~1) -- .)
17
    - dags_only=1
18
    - echo "$DAG_FOLDER"
19
    - echo "$files"
20
    - for file in $files; do
21
    -   echo "$file"
22
    -   if [[ "$file" != "$DAG_FOLDER"* ]]; then
23
    -     echo "$file is not a dag, triggering a full image build"
24
    -     dags_only=0
25
    -     break
26
    -   else
27
    -     echo "just a dag"
28
    -   fi
29
    - done
30
    - if [[ $dags_only == 1 ]]; then
31
    -   echo "doing dag-only deploy"
32
    -   astro deploy --dags $DEPLOYMENT_ID
33
    - elif [[ $dags_only == 0 ]]; then
34
    -   echo "doing image deploy"
35
    -   astro deploy -f $DEPLOYMENT_ID
36
    - fi
37
  only:
38
    - main