> ## Documentation Index > Fetch the complete documentation index at: https://astronomer.io/docs/llms.txt > Use this file to discover all available pages before exploring further. # Connect your own model provider > Connect Otto to your own OpenAI-compatible or Anthropic-compatible model provider instead of Astro-hosted models. > [!NOTE] > > **Labs** > > This feature is in [Labs](https://astronomer.io/docs/astro/feature-previews.md). Contact your account team to enable this feature. Bring your own model (BYOM) lets you connect Otto to a model provider that your Organization already uses and has approved, instead of the models that Astronomer hosts. An Organization Owner registers the provider's endpoint and API key once for the whole Organization. The Astronomer Gateway then sends model requests to your endpoint with your key. Organizations typically use BYOM when: * Their security or compliance team has approved a specific model or provider, and no other model can process their data. * They have a contract or committed spend with a model provider and want model usage billed under that contract. * They serve their own models, for example a fine-tuned model, from an endpoint that uses a [supported provider type](#supported-provider-types). ## How BYOM works 1. An Organization Owner registers one or more AI providers. Each provider has a name, a provider type, a base URL, an API key, and a list of the models it serves. 2. Your models become available with the provider name as a prefix. For example, if you name a provider `acme` and declare the model `gpt-4o`, the model ID is `acme/gpt-4o`. 3. You choose the models that Otto uses in Astro by setting a **Default model** and, optionally, a **Fast model**. 4. (Optional) You turn off **Astro-hosted** models so that Otto can only use your own providers. Otto and the Astronomer Gateway still run on Astro. BYOM changes which model processes each request, not where Otto runs. ## Request behavior * Your provider bills you for requests to your models, under your agreement with that provider. These requests don't add to your [Astro AI spend](https://astronomer.io/docs/astro/astro-ai-spending-cap.md). You're responsible for usage, costs, and quotas on your provider account. * The Astronomer Gateway's request and token rate limits apply to BYOM requests, in addition to your provider's own limits. * If your endpoint is unavailable or returns an error, the request fails. The gateway can retry a request on your endpoint up to two more times after a connection error or a `408`, `429`, `500`, `502`, `503`, or `504` response, but it never sends a failed request to an Astro-hosted model. * When Otto has no usable model of your own for a feature, for example because you haven't set a **Default model** or you removed the selected model from its provider, the **Astro-hosted** setting controls what happens. If Astro-hosted models are on, Otto uses an Astro-hosted model. If they're off, the request fails. See [Turn off Astro-hosted models](#step-4-optional-turn-off-astro-hosted-models). * The gateway only routes the models that you declare on a provider. Astro refuses a request for any other model under your provider's prefix before the request reaches your endpoint. * Astro stores your API key in a secrets backend and doesn't show it after you save it. The gateway retrieves the key for each request, so a rotated or deleted key takes effect on the next request. The gateway never sends your Astro token to your provider. * Your endpoint must be publicly reachable from Astro over HTTPS. ## Supported provider types The provider type is the API format that your endpoint uses, not the vendor. You can connect any endpoint that implements one of these formats and is reachable from Astro over HTTPS. | Provider type | Header the gateway sends | Base URL format | | - | - | - | | **OpenAI-compatible** | `Authorization: Bearer ` | Include the version segment, for example `https://api.openai.com/v1`. For Azure AI Foundry, use `https://.openai.azure.com/openai/v1`. | | **Anthropic-compatible** | `x-api-key: ` | Don't include a version segment, for example `https://api.anthropic.com`. | ## Prerequisites * BYOM enabled for your Organization. Contact your Astronomer account team. * AI features enabled for your Organization. See [Toggle AI features for your Organization](https://astronomer.io/docs/astro/enable-disable-astro-ai.md). * The Organization Owner role. * An endpoint that uses a [supported provider type](#supported-provider-types) and is reachable over HTTPS. * An API key for the endpoint. * For each model you want to use, the exact model ID that your endpoint expects, the model's context window, and its maximum output tokens. ## Step 1: Add an AI provider **1. Open the provider list** In the Astro UI, click **Organization Settings**. Click **AI Providers & Models**. In the **Model Providers** section, click **Bring-your-own**. **2. Choose the provider type** Click **New AI Provider**, then select **OpenAI-compatible** or **Anthropic-compatible** for **Provider Type**. See [Supported provider types](#supported-provider-types). **3. Enter the provider details** Enter the following values: * **Name**: The prefix for this provider's models. For example, `acme` serves `acme/gpt-4o`. Use 2 to 32 lowercase letters, numbers, and dashes. Start with a letter, and end with a letter or number. The name must be unique in your Organization. An Organization can have up to 25 providers. * (Optional) **Description**: A description of the provider. * **Base URL**: Your endpoint's URL. The URL must use `https` and can't include credentials, a query string, or a fragment. * (Optional, OpenAI-compatible only) **Responses API**: Astro assumes that every OpenAI-compatible endpoint serves chat completions, and converts responses requests to the chat completions format. Select **This endpoint also serves /v1/responses** only if your endpoint implements `/v1/responses`. If you select it and your endpoint doesn't implement `/v1/responses`, responses requests fail. * **API Key**: The key for your endpoint. **4. Declare the models** In **Models this provider may serve**, click **Add model** for each model that the provider serves, and enter the following values. A provider must declare at least 1 model and can declare up to 100, and each model ID must be unique within the provider. * **Model ID**: The ID exactly as your endpoint expects it. The gateway sends it unchanged, including namespaced IDs such as `accounts/fireworks/models/llama-v3p3-70b-instruct`. Use up to 128 letters, numbers, and the characters `-`, `_`, `.`, `:`, `/`, and `@`, and start and end with a letter or number. * **Context window**: The maximum number of input tokens that the model accepts. Astro can't read this value from your endpoint, so you must enter it. Enter a whole number from 1 to 10,000,000. * **Max output**: The maximum number of tokens that the model generates in one reply. This value can't be greater than the context window. * **Image input**: Turn on if the model accepts images as well as text. * **Reasoning**: Turn on if the model reasons before it answers. If you turn this on for a model without reasoning, your endpoint refuses requests that include reasoning parameters. **5. Save the provider** Click **Create AI Provider**. > [!NOTE] > > You can't change a provider's name or type after you create it. The name is the prefix for all of its models, and the type sets the request format. To change either value, create a new provider. ## Step 2: Test your endpoint Astro doesn't test your endpoint when you save a provider, and the **Test AI Provider** button in the provider form isn't available yet. Instead, the provider form shows a test command in **Test your endpoint**, based on the provider type and base URL that you entered. 1. Copy the command. 2. Replace `` with one of your model IDs. 3. Set the `API_KEY` environment variable to your API key, then run the command in a terminal. A successful response confirms that your endpoint accepts the key and serves the model. ## Step 3: Choose the models that Otto uses The **Default model** and **Fast model** settings control which model Otto uses in Astro. Both settings only accept models that you declared on one of your providers. Until you set them, Otto uses Astro-hosted models if **Astro-hosted** is on, and fails if it's off. | Setting | Used for | | - | - | | **Default model** | [Otto investigations](https://astronomer.io/docs/astro/otto/otto-investigate.md), including follow-up questions on an investigation, and routines | | **Fast model** | Ask Otto in the Astro UI, including Otto in the [Astro IDE](https://astronomer.io/docs/astro/ide-overview.md), and short text tasks such as project titles, change summaries, and Observe note suggestions. If you don't set a Fast model, Otto uses your Default model. | 1. In the Astro UI, click **Organization Settings**, then click **AI Providers & Models**. 2. In the **Models** section, select a **Default model**. 3. (Optional) Select a **Fast model**. If you set a Default model, its context window must be at least 200,000 tokens for Otto investigations and routines. Astro doesn't check this when you select the model. Instead, if the context window is smaller, Astro refuses to start an investigation or routine run, and doesn't use an Astro-hosted model instead, even if Astro-hosted models are on. If you haven't set a Default model, investigations use an Astro-hosted model when Astro-hosted models are on. When you run Otto in the Astro CLI, you choose the model in your session instead. Your declared models appear in the model list with their prefixed IDs. Run `/model` to switch models, or pass `--model /` when you start Otto. See [Customize Otto](https://astronomer.io/docs/astro/otto/customize-otto.md#choose-a-model). ## Step 4: (Optional) Turn off Astro-hosted models If your Organization must not use Astro-hosted models, turn them off. When Astro-hosted models are off, the Astronomer Gateway refuses requests for Astro-hosted models from your Organization, and your Organization's model list only shows your own models. The change can take up to 5 minutes to reach the gateway. > [!WARNING] > > Set a **Default model** before you turn off Astro-hosted models. If Astro-hosted models are off and no Default model is set, Otto features in Astro fail. 1. In the Astro UI, click **Organization Settings**, then click **AI Providers & Models**. 2. In the **Model Providers** section, turn off **Astro-hosted**. ## Edit or delete an AI provider To view your providers, open the Astro UI and go to **Organization Settings** > **AI Providers & Models** > **Bring-your-own**. To edit or delete a provider, click the actions menu in its row, then click **Edit AI Provider** or **Delete AI Provider**. After you edit a provider, click **Update AI Provider** to save your changes. * To rotate an API key, edit the provider and enter the new key. To keep the current key while you edit other fields, leave **API Key** blank. * To change the base URL, enter the API key again in the same edit. * To add or remove models, edit the provider's model list. * To delete a provider, first make sure that none of its models is your Default model or Fast model. Astro refuses to delete a provider, or remove a model from it, while that model is selected. ## Limitations * [Otto code review](https://astronomer.io/docs/astro/otto/otto-code-review.md) doesn't use your Default model. By default, the review action uses the model set by its reviewer persona. To review with one of your own models, set the action's `model` input on GitHub, or the `OTTO_MODEL` variable on GitLab, to `/`. * [AI log summaries](https://astronomer.io/docs/astro/root-cause-analysis.md#ai-log-summaries) and task failure categorization in Astro Observe don't use your Default or Fast model. They use Astro-hosted models, including when you turn off Astro-hosted models. To stop these features, [turn off AI features](https://astronomer.io/docs/astro/enable-disable-astro-ai.md) for your Organization. ## Related * [Toggle AI features for your Organization](https://astronomer.io/docs/astro/enable-disable-astro-ai.md) * [Set Astro AI spend limits](https://astronomer.io/docs/astro/astro-ai-spending-cap.md) * [Models and regions](https://astronomer.io/docs/astro/otto/otto-models-and-regions.md) * [Customize Otto](https://astronomer.io/docs/astro/otto/customize-otto.md) * [Astronomer feature lifecycle](https://astronomer.io/docs/astro/feature-previews.md)