> ## Documentation Index
> Fetch the complete documentation index at: https://astronomer.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Install Python packages from private sources

> Install Python packages from private PyPI indices or private Git repositories into your Astro project by customizing your Runtime image.

Python packages can be installed into your image from both public and private sources. To install packages listed on private PyPI indices or a private git-based repository, you need to complete additional configuration in your project.

Depending on where your private packages are stored, use one of the following setups to install these packages to an Astro project by customizing your Runtime image.

> [!NOTE]
>
> Deploying a custom Runtime image with a CI/CD pipeline requires additional configurations. For an example implementation, see the GitHub Actions CI/CD templates for [Astro](https://astronomer.io/docs/astro/ci-cd-templates/github-actions-template.md) and [Astro Private Cloud](https://www.astronomer.io/docs/software/ci-cd#github-actions-cicd).

## Setup

**Private GitHub Repo**

#### Install Python packages from private GitHub repositories

This topic provides instructions for building your Astro project with Python packages from a private GitHub repository. Although GitHub is used in these examples, the same approach works with any hosted Git repository.

**Using .netrc build secrets**

Use the `--build-secret` flag to pass `.netrc` credentials at build time. This approach lets you add private packages to your `requirements.txt` file without custom Dockerfile configuration.

> [!NOTE]
>
> This method requires Astro Runtime 3.1-14+, 3.0-15+, or later than 13.5.1.

#### Prerequisites

* The [Astro CLI](https://astronomer.io/docs/cli/v1.47/overview.md)
* An [Astro project](https://astronomer.io/docs/cli/v1.47/develop-project.md#create-an-astro-project)
* Custom Python packages that are [installable with pip](https://packaging.python.org/en/latest/tutorials/packaging-projects/)
* A private GitHub repository for each of your custom Python packages
* The [GitHub CLI](https://cli.github.com/) or a [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)

#### Step 1: Specify the private repository in your project

Add your private packages to your `requirements.txt` file using HTTPS URLs in the following format:

```text wrap theme={null}
git+https://github.com/<your-github-organization-name>/<your-private-repository>.git
```

For example, to install `mypackage1` and `mypackage2` from `myorganization`:

```text wrap theme={null}
git+https://github.com/myorganization/mypackage1.git
git+https://github.com/myorganization/mypackage2.git
```

#### Step 2: Configure credentials

Define the `NETRC_CONTENT` environment variable in your shell profile (`.bashrc` or `.zshrc`):

```bash wrap theme={null}
export NETRC_CONTENT="machine github.com login oauth2 password $(gh auth token)"
```

If you don't use the GitHub CLI, replace `$(gh auth token)` with a [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) that has access to your private repositories.

#### Step 3: Run with build secrets

Pass the `--build-secret` flag when running Astro CLI commands:

```bash wrap theme={null}
astro dev start --build-secret id=netrc,env=NETRC_CONTENT
```

To run tests:

```bash wrap theme={null}
astro dev pytest --build-secret id=netrc,env=NETRC_CONTENT
```

The `--build-secret` flag securely provides the `.netrc` content during the Docker build without storing credentials in the image. Alternatively, set the `dev.build_secrets` [configuration](https://astronomer.io/docs/cli/v1.47/configure-cli.md) once to apply it to every `astro dev` command without repeating the flag.

#### Deploy with GitHub Actions

To build your image in a GitHub Actions workflow, pass the `.netrc` content as a build secret:

```yaml wrap theme={null}
- name: Build image
  uses: docker/build-push-action@v4
  with:
    context: .
    secrets: |
      netrc=machine github.com login oauth2 password ${{ secrets.GITHUB_TOKEN }}
```

#### Deploy with the Astronomer deploy action

To deploy using the [Astronomer deploy action](https://github.com/astronomer/deploy-action), pass the `.netrc` content through `build-secrets` and set the `NETRC_CONTENT` environment variable:

```yaml wrap theme={null}
- name: Deploy to Astro
  uses: astronomer/deploy-action@v0.x
  with:
    build-secrets: id=netrc,env=NETRC_CONTENT
    # ... deployment-id, etc.
  env:
    NETRC_CONTENT: "machine github.com login oauth2 password ${{ secrets.GITHUB_TOKEN }}"
```

**Using SSH keys**

> [!NOTE]
>
> The following setup has been validated with only a single SSH key. You might need to modify this setup when using more than one SSH key per Docker image.

#### Prerequisites

* The [Astro CLI](https://astronomer.io/docs/cli/v1.47/overview.md)
* An [Astro project](https://astronomer.io/docs/cli/v1.47/develop-project.md#create-an-astro-project)
* Custom Python packages that are [installable with pip](https://packaging.python.org/en/latest/tutorials/packaging-projects/)
* A private GitHub repository for each of your custom Python packages
* A [GitHub SSH private key](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) authorized to access your private GitHub repositories

This setup assumes that each custom Python package is hosted within its own private GitHub repository. Installing multiple custom packages from a single private GitHub repository is not supported.

> [!WARNING]
>
> If your organization enforces SAML single sign-on (SSO), you must first authorize your key to be used with that authentication method. See [Authorizing an SSH key for use with SAML single sign-on](https://docs.github.com/en/enterprise-cloud@latest/authentication/authenticating-with-saml-single-sign-on/authorizing-an-ssh-key-for-use-with-saml-single-sign-on).

#### Step 1: Specify the private repository in your project

To add a Python package from a private repository to your Astro project, specify the Secure Shell (SSH) URL for the repository in a new `private-requirements.txt` file. Use the following format for the SSH URL:

```text wrap theme={null}
git+ssh://git@github.com/<your-github-organization-name>/<your-private-repository>.git
```

For example, to install `mypackage1` and `mypackage2` from `myorganization`, add the following to your `private-requirements.txt` file:

```text wrap theme={null}
git+ssh://git@github.com/myorganization/mypackage1.git
git+ssh://git@github.com/myorganization/mypackage2.git
```

This example assumes that the name of each of your Python packages is identical to the name of its corresponding GitHub repository. In other words,`mypackage1` is both the name of the package and the name of the repository.

#### Step 2: Update Dockerfile

1. (Optional) Copy and save any existing build steps in your `Dockerfile`.

2. Add the following to your `packages.txt` file:

   ```bash wrap theme={null}
   openssh-client
   git
   ```

3. In your Dockerfile, add the following instructions:

   ```docker wrap theme={null}
   USER root
   RUN mkdir -p -m 0700 ~/.ssh && \
       echo "github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl" >> ~/.ssh/known_hosts

   COPY private-requirements.txt .
   RUN --mount=type=ssh,id=github pip install --no-cache-dir --requirement private-requirements.txt
   USER astro

   ENV PATH="/home/astro/.local/bin:$PATH"
   ```

   In order, these instructions:

   * Switch to `root` user for SSH setup and installation from private repository
   * Add the fingerprint for GitHub to `known_hosts`
   * Copy your `private-requirements.txt` file into the image
   * Install Python-level packages from your private repository as specified in your `private-requirements.txt` file. This securely mounts your SSH key at build time, ensuring that the key itself is not stored in the resulting Docker image filesystem or metadata.
   * Switch back to `astro` user
   * Add the user bin directory to `PATH`

> [!NOTE]
>
> See GitHub's documentation for all available [SSH key fingerprints](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/githubs-ssh-key-fingerprints).
>
> If your repository isn't hosted on GitHub, replace the fingerprint with one from where the package is hosted. Use `ssh-keyscan` to generate the fingerprint.

#### Step 3: Build a custom Docker image

1. Run the following command to automatically generate a unique image name:

   ```sh wrap theme={null}
   image_name=astro-$(date +%Y%m%d%H%M%S)
   ```

2. Run the following command to create a new Docker image from your `Dockerfile`. Replace `<ssh-key>` with your SSH private key file name.

   ```sh wrap theme={null}
   DOCKER_BUILDKIT=1 docker build -f Dockerfile --progress=plain --ssh=github="$HOME/.ssh/<ssh-key>" -t $image_name .
   ```

3. (Optional) Test your Dags locally. See [Restart your local environment](https://astronomer.io/docs/cli/v1.47/run-airflow-locally.md#restart-a-local-airflow-environment).

4. Deploy the image to Astro using the Astro CLI:

   ```sh wrap theme={null}
   astro deploy --image-name $image_name
   ```

Your Astro project can now use Python packages from your private GitHub repository.

**Private PyPi Index**

#### Install Python packages from a private PyPI index

Installing Python packages on Astro from a private PyPI index is required for organizations that deploy a [private PyPI server (`private-pypi`)](https://pypi.org/project/private-pypi/) as a secure layer between pip and a Python package storage backend, such as GitHub, AWS, or a local file system or managed service.

To complete this setup, you'll specify your privately hosted Python packages in `requirements.txt`.

#### Prerequisites

* An [Astro project](https://astronomer.io/docs/cli/v1.47/develop-project.md#create-an-astro-project)
* A private PyPI index with a corresponding username and password

#### Add Python packages to your Astro project

To install a Python package from a private PyPI index, add the package name and version to the `requirements.txt` file of your Astro project.

If you don't specify a version, Astro installs the latest version. Use the same syntax for installing private packages that you use when you add public packages from [PyPI](https://pypi.org). Your `requirements.txt` file can contain both publicly accessible and private packages.

At the beginning of the `requirements.txt` file, add `--index-url https://myuser:example.com/api/pypi/pypi/simple` for Airflow 3 based projects and `--extra-index-url https://myuser:example.com/api/pypi/pypi/simple` for Airflow 2 based projects. Replace this example URL with your index's location.

#### Example `requirements.txt`

```requirements wrap theme={null}
--index-url https://myuser:example.com/api/pypi/pypi/simple
pandas
my-custom-package-company
apache-airflow-providers-snowflake
```

Your Astro project can now use Python packages from your private PyPi index.

#### Install packages in your own Dockerfile steps

When you list packages in `requirements.txt`, the Astro Runtime image installs them during the build and applies the constraints file at `/etc/pip-constraints.txt`. This file pins `apache-airflow` and its core packages to the versions in your Runtime image.

If you install packages in your own `RUN` steps instead, for example to point at an organization-wide index, you must apply the constraints file yourself. Without it, a dependency conflict can cause the installer to downgrade Airflow to an older version instead of failing the build.

The constraints file only pins the Airflow packages themselves. An install can still replace a package that Airflow depends on with an incompatible version, without changing Airflow. Run `uv pip check --system` or `pip check` after you install packages so that the build fails if any installed package has an unmet requirement.

**Airflow 3**

Astro Runtime 3.0 and later installs `requirements.txt` with [`uv pip`](https://docs.astral.sh/uv/pip/), and Astronomer recommends that you use `uv pip` in your own Dockerfile steps too. It resolves dependencies faster than pip and reports the full chain of requirements behind a conflict. `pip` is still available in Airflow 3 images. To keep using it, follow the instructions in the **Airflow 2** tab.

uv doesn't read `/etc/pip.conf`, so the `constraint` setting in that file has no effect on `uv pip`. Pass the constraints file with `-c` on every `uv pip install` command, or set the `UV_CONSTRAINT` environment variable:

```docker wrap theme={null}
USER root
COPY private-requirements.txt .
RUN --mount=type=secret,id=netrc,target=/root/.netrc \
    uv pip install --system \
      --index-url https://example.com/api/pypi/pypi/simple \
      -c /etc/pip-constraints.txt \
      -r private-requirements.txt && \
    uv pip check --system
USER astro
```

In order, these instructions:

* Switch to the `root` user to install packages
* Copy your `private-requirements.txt` file into the image
* Mount your index credentials as a `.netrc` build secret, so they aren't stored in the image. Pass the secret with `--build-secret id=netrc,env=NETRC_CONTENT`, as described in the **Private GitHub Repo** setup
* Install your packages from your private index, constrained to the Airflow version in your Runtime image
* Fail the build if any installed package, including Airflow, has an unmet requirement
* Switch back to the `astro` user

uv also reads its global configuration from `/etc/uv/uv.toml` instead of `/etc/pip.conf`. If you configure your index there, keep passing `-c /etc/pip-constraints.txt` on each install.

> [!NOTE]
>
> uv handles packages that exist on more than one index differently from pip, to protect against dependency confusion attacks. To make the Runtime image install `requirements.txt` with pip instead of uv, add `# ASTRO_RUNTIME_USE_PIP` to your `requirements.txt` file. See [Pip package resolution](https://astronomer.io/docs/astro/airflow3/upgrade-af3.md#pip-package-resolution).

**Airflow 2**

Astro Runtime 13 and earlier installs `requirements.txt` with `pip`. The Runtime image's `/etc/pip.conf` sets the constraints file for every `pip install` command.

If your organization configures pip globally with its own `/etc/pip.conf`, ensure that the following is included:

```text wrap theme={null}
[global]
constraint = /etc/pip-constraints.txt
```

> [!WARNING]
>
> If you replace `/etc/pip.conf` in your Dockerfile, for example with `COPY my-pip.conf /etc/pip.conf`, you also replace the Runtime image's `constraint` setting. Add the `constraint` line to your own file, or pass `-c /etc/pip-constraints.txt` on each `pip install` command.

```docker wrap theme={null}
USER root
COPY private-requirements.txt .
RUN --mount=type=secret,id=netrc,target=/root/.netrc \
    pip install --no-cache-dir \
      --extra-index-url https://example.com/api/pypi/pypi/simple \
      -c /etc/pip-constraints.txt \
      -r private-requirements.txt && \
    pip check
USER astro
```

These instructions also work in Airflow 3 images if you prefer to use pip.