> ## Documentation Index > Fetch the complete documentation index at: https://astronomer.io/docs/llms.txt > Use this file to discover all available pages before exploring further. # Astro API best practices The [Astro API](/docs/astro/api/v-1/overview) is Astronomer's REST API for managing resources on Astro, for example to create a Deployment. This guide provides the following best practices for general development, and specifically development with the Astro API, to ensure a safe and optimal experience: * Considerations for using the [Astro API](/docs/astro/api/v-1/overview), [Astro CLI](/docs/cli/v1.43/overview), and [Astro Terraform provider](/docs/astro/terraform-provider) * Error handling * Authenticating scripts using API tokens * Using a graphical REST API client for development * Handling rate limiting with exponential backoff retries * Reusing HTTP connections ## Feature overview This guide highlights the following Astro features: * [The Astro API](/docs/astro/api/v-1/overview) * [The Astro CLI](/docs/cli/v1.43/overview) * [The Astro Terraform provider](/docs/astro/terraform-provider) * Astro API Tokens * [Deployment API tokens](/docs/astro/deployment-api-tokens) * [Workspace API tokens](/docs/astro/workspace-api-tokens) * [Organization API tokens](/docs/astro/organization-api-tokens) ## Considerations for using the Astro API, Astro CLI, and Astro Terraform provider Astronomer provides several tools to manage Astro resources. Each tool offers benefits for specific use cases: ### Astro API * Has structured input and output, which allows for convenient automation * Allows you to use any programming language or framework that can make HTTP requests * Requires you to implement status checking mechanisms yourself, such as polling the API to check for Deployment creation completion ### Astro CLI * Provides a local development environment * Produces human-readable output such as text or table format, therefore less suitable for automation * Makes deploying Airflow code to Astro convenient ### Astro Terraform provider * Allows you to use Terraform, the industry standard for managing infrastructure as code * Requires Terraform development familiarity * Uses declarative language, which means you define your desired end state and, don't have to think about details, such as polling for infrastructure creation status The tool that suits you best depends on factors such as your technical experience, existing knowledge in your organization, and use of Astronomer. It's common to use all available tools. ## Error handling with the Astro API It's a best practice to ensure your code handles unexpected situations properly, such as errors. Because Astro's tooling generally distinguishes between client-side and server-side errors, you can automate a process for determining the cause of an error. All HTTP 4XX code indicate a client-side error and all HTTP 5XX status codes indicate a server-side error. For example, Deployment names are unique within an Astronomer Workspace. If you create a second Deployment with the same name as an already existing Deployment, the Astro API returns an HTTP 400 error. To ensure your code handles errors correctly, wrap requests in a `try`/`except` code block: ```python expandable wrap theme={null} import requests organization_id = "" workspace_id = "" astro_api_token = "" try: # Create a Deployment # https://www.astronomer.io/docs/astro-api/platform-api-reference/deployment/create-deployment response = requests.post( f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments", headers={"Authorization": f"Bearer {astro_api_token}"}, json={ "astroRuntimeVersion": "3.1-5", "defaultTaskPodCpu": "0.25", "defaultTaskPodMemory": "0.5Gi", "executor": "CELERY", "isCicdEnforced": False, "isDagDeployEnabled": False, "isHighAvailability": False, "name": "my_deployment", # <== this will fail if "my_deployment" already exists "resourceQuotaCpu": "10", "resourceQuotaMemory": "20Gi", "schedulerSize": "SMALL", "type": "STANDARD", "workspaceId": workspace_id, }, ) response.raise_for_status() except requests.exceptions.HTTPError as e: print("Failed creating deployment. Reason: " + e.response.json()["message"]) raise e ``` The statement, `response.raise_for_status()`, raises an exception on any HTTP response code that's not `2XX`, which are all non-successful HTTP response codes. In the `except` clause, you can handle this exception however you want. In case of an error, the Astro API returns a reason in the response body, with the key `message`. For this specific example, it returns HTTP code 400. This code example prints the error reason. Without the reason, you can't know why a request failed. In the example of a duplicated Deployment name, the logs show the following: ```text wrap theme={null} Failed creating Deployment. Reason: Invalid request: Deployment name 'my_deployment' already exists in this workspace ``` The complete error response structure is: ```json wrap theme={null} { "message": "Invalid request: Deployment name 'my_deployment' already exists in this workspace", "requestId": "f004d12a-29c8-40d8-b239-2b1b615ea45b", "statusCode": 400 } ``` For traceability purposes, you can also include the `requestId` in your logs, which is an internal Astronomer identifier that Astronomer support can use to track down your request. ## Authenticating scripts using API tokens In automated scripts, such as CI/CD pipelines, you can query the Astro API with an API token. API tokens grant access to certain Astronomer resources, so it's important to keep the token safe. **Do not hardcode the token in code.** Instead, store the token in a secret and expose it as an environment variable, `ASTRO_API_TOKEN`. Storing API tokens in a system dedicated for storing secret values, like [GitHub Actions Secrets](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository), ensures secret values are not visible to humans and only referenced by code when needed. Additionally, a best security practice is the [*Principle of Least Privilege*](https://en.wikipedia.org/wiki/Principle_of_least_privilege), where you grant only the permissions necessary to perform an action. This reduces the attack surface, or the number of ways a bad actor could cause damage, in case of a leaked API token. Astronomer provides three levels of API tokens, from least to most privilege: * [Deployment API tokens](/docs/astro/deployment-api-tokens) * [Workspace API tokens](/docs/astro/workspace-api-tokens) * [Organization API tokens](/docs/astro/organization-api-tokens) This is feature is only available if you are on the **Enterprise** tier or above. See [Astro Plans and Pricing](https://www.astronomer.io/pricing/). Consider [custom Deployment roles](/docs/astro/customize-deployment-roles) for configuring Deployment-level roles with only the necessary permissions. ## Use a graphical REST API client for development The [Astro API documentation](https://www.astronomer.io/docs/astro/api) provides a convenient web interface to try out the API: Screenshot of the Astro API documentation A graphical REST API client can be a helpful addition when developing with any REST API. Popular tools include [Postman](https://www.postman.com) and [Insomnia](https://insomnia.rest/products/insomnia). The Astro API documentation provides [downloads to the API specifications](/docs/astro/api/v-1/overview#download-openapi-specification), which are YAML files that define the API structure, that you can load into your tool of choice. Graphical REST API clients often provide convenience features over web-based documentation including query history, value sharing with variables, the ability to define environments with different values, and chained requests, which lets you use results from one query in a follow-up query. ## Handle rate limiting with exponential backoff retries The [Astro API limits requests](/docs/astro/api/v-1/overview#rate-limiting) in case the request rate passes certain thresholds, depending on the type of the request. When a request is rate limited, the API returns an `HTTP 429` status code. It's a best practice to apply an exponential backoff strategy to not overload the server side for rate-limited requests. An exponential backoff strategy gradually increases the time between requests to allow for the server to recover and respond correctly, for example, by waiting `1`, `2`, `4`, or `8` seconds between consecutive requests. Consider the scenario of waiting for a Deployment to receive status `HEALTHY` after creation. Creating a Deployment can take a moment, so repeatedly requesting the status from the Astro API without any pause between requests can cause rate limiting. The following example checks the HTTP response code and handle HTTP 429 separately from other response codes. While this script won't hit any rate limit threshold on the Astro API since the code waits 5 seconds between requests, running multiple scripts simultaneously can quickly increase requests. When receiving an HTTP 429 response, it calculates a waiting period of `2 ** (timeout_attempts - 1)` seconds and then increases the period depending on the attempt number. ```python {14-19,32-36} expandable wrap theme={null} from datetime import datetime import requests import time organization_id = "" deployment_id = "" astro_api_token = "" timeout_secs = 600 timeout_attempts = 0 start = datetime.now() while True: try: response = requests.get( f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments/{deployment_id}", headers={"Authorization": f"Bearer {astro_api_token}"}, ) response.raise_for_status() timeout_attempts = 0 deployment_status = response.json()["status"] if deployment_status == "HEALTHY": print("Deployment is healthy") break if (datetime.now() - start).total_seconds() > timeout_secs: raise Exception("Timeout") else: print(f"Deployment status is currently {deployment_status}. Waiting...") time.sleep(5) except requests.exceptions.HTTPError as e: if e.response.status_code == 429: timeout_attempts += 1 sleep_duration = 2 ** (timeout_attempts - 1) print(f"Request was rate limited. Sleeping {sleep_duration} seconds and trying again.") time.sleep(sleep_duration) else: print("Failed fetching deployment status. Reason: " + e.response.json()["message"]) raise e ``` In the previous example, the code defines custom logic for handling rate limits and exponential backoffs. While this works, [Python's requests library](https://requests.readthedocs.io) comes with several built-in utilities you can use to simplify the code. For example, you can avoid defining your own exponential backoff logic by using the Python `requests` library. `requests.session()` creates a persistent session between requests and replaces `requests.get(...)` with `session.get(...)` to use the settings configured in the session. This way you don't have to duplicate the same `if e.response.status_code == 429` business logic for every request. The code example below configures the `urllib3.Retry` exponential backoff logic for HTTP status code 429 and up to 10 attempts. This means it waits `1`, `2`, `4`, ..., `128`, `256`, and `512` seconds in between the ten attempts. ```python wrap theme={null} import requests from requests.adapters import HTTPAdapter from urllib3 import Retry session = requests.session() ratelimit_retry = Retry(status_forcelist=[429], backoff_factor=1, total=10) session.mount(prefix="https://api.astronomer.io", adapter=HTTPAdapter(max_retries=ratelimit_retry)) response = session.get(...) ``` You can set a default value for `headers` using Python's request library as a way to simplify your code. While you can write `headers={"Authorization": f"Bearer {astro_api_token}"}` with every request, it's cleaner to define this value once and then automatically apply it to every request using the `session` object: ```python wrap theme={null} import requests session = requests.session() session.headers = {"Authorization": f"Bearer {astro_api_token}"} # Before requests.get( f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments/{deployment_id}", headers={"Authorization": f"Bearer {astro_api_token}"}, ) # After session.get(f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments/{deployment_id}") ``` ## Reuse HTTP connections For repeated requests to the Astro API, or any REST API, it's a best practice to reuse connections. That means you (client) keep a connection open to the Astro API (server) for multiple requests, instead of opening and closing a connection for every request. This reduces latency and CPU usage. Reusing HTTP connections is also referred to as [HTTP persistent connections or HTTP keep-alive](https://en.wikipedia.org/wiki/HTTP_persistent_connection), where keep-alive refers to the [Keep-Alive](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Keep-Alive) header that [used to be](https://datatracker.ietf.org/doc/html/rfc9113#section-8.2.2) transmitted with the message. Using Python's requests library again, `requests.session` object reuses connections: ```python wrap theme={null} import requests session = requests.session() session.headers = {"Authorization": f"Bearer {astro_api_token}"} session.get(url=f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments/{deployment_id}") session.get(url=f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments/{deployment_id}") ``` A single connection then handles the two `GET` requests, instead of recreating a connection. This becomes visible when configuring `DEBUG` logging: ```python {10,12,21} wrap theme={null} import logging import requests logging.basicConfig(level=logging.DEBUG) # Before requests.get(url=f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments/{deployment_id}", headers={"Authorization": f"Bearer {astro_api_token}"}) requests.get(url=f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments/{deployment_id}", headers={"Authorization": f"Bearer {astro_api_token}"}) # DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.astronomer.io:443 # DEBUG:urllib3.connectionpool:https://api.astronomer.io:443 "GET /platform/v1beta1/organizations/clkvh3b46003m01kbalgwwdcy/deployments/clw9bhr2n0d9801gp7zuigsqn HTTP/11" 200 None # DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.astronomer.io:443 # DEBUG:urllib3.connectionpool:https://api.astronomer.io:443 "GET /platform/v1beta1/organizations/clkvh3b46003m01kbalgwwdcy/deployments/clw9bhr2n0d9801gp7zuigsqn HTTP/11" 200 None # After session = requests.session() session.headers = {"Authorization": f"Bearer {astro_api_token}"} session.get(url=f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments/{deployment_id}") session.get(url=f"https://api.astronomer.io/platform/v1beta1/organizations/{organization_id}/deployments/{deployment_id}") # DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.astronomer.io:443 # DEBUG:urllib3.connectionpool:https://api.astronomer.io:443 "GET /platform/v1beta1/organizations/clkvh3b46003m01kbalgwwdcy/deployments/clw9bhr2n0d9801gp7zuigsqn HTTP/11" 200 None # DEBUG:urllib3.connectionpool:https://api.astronomer.io:443 "GET /platform/v1beta1/organizations/clkvh3b46003m01kbalgwwdcy/deployments/clw9bhr2n0d9801gp7zuigsqn HTTP/11" 200 None ``` In the logs, you can see the first example creates two connections. The second example uses `requests.session`, which reuses connections, so it only creates one connection. ## See also * [Astro API documentation](https://www.astronomer.io/docs/astro/api) * [Make requests to the Airflow REST API](/docs/astro/airflow-api) * [Python requests documentation](https://requests.readthedocs.io)