Skip to main content
LabsThis feature is in Labs. Contact your account team to enable this feature.
Bring your own model (BYOM) lets you connect Otto to a model provider that your Organization already uses and has approved, instead of the models that Astronomer hosts. An Organization Owner registers the provider’s endpoint and API key once for the whole Organization. The Astronomer Gateway then sends model requests to your endpoint with your key. Organizations typically use BYOM when:
  • Their security or compliance team has approved a specific model or provider, and no other model can process their data.
  • They have a contract or committed spend with a model provider and want model usage billed under that contract.
  • They serve their own models, for example a fine-tuned model, from an endpoint that uses a supported provider type.

​
How BYOM works

  1. An Organization Owner registers one or more AI providers. Each provider has a name, a provider type, a base URL, an API key, and a list of the models it serves.
  2. Your models become available with the provider name as a prefix. For example, if you name a provider acme and declare the model gpt-4o, the model ID is acme/gpt-4o.
  3. You choose the models that Otto uses in Astro by setting a Default model and, optionally, a Fast model.
  4. (Optional) You turn off Astro-hosted models so that Otto can only use your own providers.
Otto and the Astronomer Gateway still run on Astro. BYOM changes which model processes each request, not where Otto runs.

​
Request behavior

  • Your provider bills you for requests to your models, under your agreement with that provider. These requests don’t add to your Astro AI spend. You’re responsible for usage, costs, and quotas on your provider account.
  • The Astronomer Gateway’s request and token rate limits apply to BYOM requests, in addition to your provider’s own limits.
  • If your endpoint is unavailable or returns an error, the request fails. The gateway can retry a request on your endpoint up to two more times after a connection error or a 408, 429, 500, 502, 503, or 504 response, but it never sends a failed request to an Astro-hosted model.
  • When Otto has no usable model of your own for a feature, for example because you haven’t set a Default model or you removed the selected model from its provider, the Astro-hosted setting controls what happens. If Astro-hosted models are on, Otto uses an Astro-hosted model. If they’re off, the request fails. See Turn off Astro-hosted models.
  • The gateway only routes the models that you declare on a provider. Astro refuses a request for any other model under your provider’s prefix before the request reaches your endpoint.
  • Astro stores your API key in a secrets backend and doesn’t show it after you save it. The gateway retrieves the key for each request, so a rotated or deleted key takes effect on the next request. The gateway never sends your Astro token to your provider.
  • Your endpoint must be publicly reachable from Astro over HTTPS.

​
Supported provider types

The provider type is the API format that your endpoint uses, not the vendor. You can connect any endpoint that implements one of these formats and is reachable from Astro over HTTPS.

​
Prerequisites

  • BYOM enabled for your Organization. Contact your Astronomer account team.
  • AI features enabled for your Organization. See Toggle AI features for your Organization.
  • The Organization Owner role.
  • An endpoint that uses a supported provider type and is reachable over HTTPS.
  • An API key for the endpoint.
  • For each model you want to use, the exact model ID that your endpoint expects, the model’s context window, and its maximum output tokens.

​
Step 1: Add an AI provider

1

Open the provider list

In the Astro UI, click Organization Settings.Click AI Providers & Models. In the Model Providers section, click Bring-your-own.
2

Choose the provider type

Click New AI Provider, then select OpenAI-compatible or Anthropic-compatible for Provider Type. See Supported provider types.
3

Enter the provider details

Enter the following values:
  • Name: The prefix for this provider’s models. For example, acme serves acme/gpt-4o. Use 2 to 32 lowercase letters, numbers, and dashes. Start with a letter, and end with a letter or number. The name must be unique in your Organization. An Organization can have up to 25 providers.
  • (Optional) Description: A description of the provider.
  • Base URL: Your endpoint’s URL. The URL must use https and can’t include credentials, a query string, or a fragment.
  • (Optional, OpenAI-compatible only) Responses API: Astro assumes that every OpenAI-compatible endpoint serves chat completions, and converts responses requests to the chat completions format. Select This endpoint also serves /v1/responses only if your endpoint implements /v1/responses. If you select it and your endpoint doesn’t implement /v1/responses, responses requests fail.
  • API Key: The key for your endpoint.
4

Declare the models

In Models this provider may serve, click Add model for each model that the provider serves, and enter the following values. A provider must declare at least 1 model and can declare up to 100, and each model ID must be unique within the provider.
  • Model ID: The ID exactly as your endpoint expects it. The gateway sends it unchanged, including namespaced IDs such as accounts/fireworks/models/llama-v3p3-70b-instruct. Use up to 128 letters, numbers, and the characters -, _, ., :, /, and @, and start and end with a letter or number.
  • Context window: The maximum number of input tokens that the model accepts. Astro can’t read this value from your endpoint, so you must enter it. Enter a whole number from 1 to 10,000,000.
  • Max output: The maximum number of tokens that the model generates in one reply. This value can’t be greater than the context window.
  • Image input: Turn on if the model accepts images as well as text.
  • Reasoning: Turn on if the model reasons before it answers. If you turn this on for a model without reasoning, your endpoint refuses requests that include reasoning parameters.
5

Save the provider

Click Create AI Provider.
You can’t change a provider’s name or type after you create it. The name is the prefix for all of its models, and the type sets the request format. To change either value, create a new provider.

​
Step 2: Test your endpoint

Astro doesn’t test your endpoint when you save a provider, and the Test AI Provider button in the provider form isn’t available yet. Instead, the provider form shows a test command in Test your endpoint, based on the provider type and base URL that you entered.
  1. Copy the command.
  2. Replace <model> with one of your model IDs.
  3. Set the API_KEY environment variable to your API key, then run the command in a terminal.
A successful response confirms that your endpoint accepts the key and serves the model.

​
Step 3: Choose the models that Otto uses

The Default model and Fast model settings control which model Otto uses in Astro. Both settings only accept models that you declared on one of your providers. Until you set them, Otto uses Astro-hosted models if Astro-hosted is on, and fails if it’s off.
  1. In the Astro UI, click Organization Settings, then click AI Providers & Models.
  2. In the Models section, select a Default model.
  3. (Optional) Select a Fast model.
If you set a Default model, its context window must be at least 200,000 tokens for Otto investigations and routines. Astro doesn’t check this when you select the model. Instead, if the context window is smaller, Astro refuses to start an investigation or routine run, and doesn’t use an Astro-hosted model instead, even if Astro-hosted models are on. If you haven’t set a Default model, investigations use an Astro-hosted model when Astro-hosted models are on. When you run Otto in the Astro CLI, you choose the model in your session instead. Your declared models appear in the model list with their prefixed IDs. Run /model to switch models, or pass --model <provider-name>/<model-id> when you start Otto. See Customize Otto.

​
Step 4: (Optional) Turn off Astro-hosted models

If your Organization must not use Astro-hosted models, turn them off. When Astro-hosted models are off, the Astronomer Gateway refuses requests for Astro-hosted models from your Organization, and your Organization’s model list only shows your own models. The change can take up to 5 minutes to reach the gateway.
Set a Default model before you turn off Astro-hosted models. If Astro-hosted models are off and no Default model is set, Otto features in Astro fail.
  1. In the Astro UI, click Organization Settings, then click AI Providers & Models.
  2. In the Model Providers section, turn off Astro-hosted.

​
Edit or delete an AI provider

To view your providers, open the Astro UI and go to Organization Settings > AI Providers & Models > Bring-your-own. To edit or delete a provider, click the actions menu in its row, then click Edit AI Provider or Delete AI Provider. After you edit a provider, click Update AI Provider to save your changes.
  • To rotate an API key, edit the provider and enter the new key. To keep the current key while you edit other fields, leave API Key blank.
  • To change the base URL, enter the API key again in the same edit.
  • To add or remove models, edit the provider’s model list.
  • To delete a provider, first make sure that none of its models is your Default model or Fast model. Astro refuses to delete a provider, or remove a model from it, while that model is selected.

​
Limitations

  • Otto code review doesn’t use your Default model. By default, the review action uses the model set by its reviewer persona. To review with one of your own models, set the action’s model input on GitHub, or the OTTO_MODEL variable on GitLab, to <provider-name>/<model-id>.
  • AI log summaries and task failure categorization in Astro Observe don’t use your Default or Fast model. They use Astro-hosted models, including when you turn off Astro-hosted models. To stop these features, turn off AI features for your Organization.