PreviewThis feature is in Preview.
How to access investigations
You can trigger an investigation from any of these surfaces:- Astro UI: From the Dags list, click a Dag to open its detail page, then select a failed run to trigger an investigation.
- Astro Observe homepage: Click Investigate next to a Dag to investigate its most recent failed run, or open a specific run in the Dag’s run history.
- Catalog: Open a Dag in the Catalog and select a failed run from its run history.
- Astro alerts: When a Dag failure alert fires, open the failed Dag run from the alert notification to start an investigation. See Set up Astro alerts.
- Astro API: Trigger an investigation programmatically, poll for status, and read the result. Combine this with the Dag Trigger notification channel on an Astro alert to investigate critical Dag failures automatically.
What Otto investigates
Otto draws on Airflow context (including Dag code and task logs), Astro context (including Deployment configuration, component logs, and recent deploys), and Observe context (including lineage, run history, and operational metrics). Organizations that use Astro Observe get a richer investigation, since Otto has access to lineage, the Asset Catalog, and operational metrics in addition to Airflow and Astro context. Otto can also query external systems, such as your warehouse or dbt project, through MCP servers that an Organization Owner registers and enables for investigations. This allows Otto to investigate issues with root cause outside of Airflow. See Connect Otto to your MCP servers.Apply a suggested fix
When an investigation completes, the What to change section of the investigation lists up to three suggested fixes in order. Each fix has a label that shows its type:- Code patch: A
git applycommand that applies a fix to one file in your Dag code. Otto checks that the patch applies to the code from the deploy that failed, and doesn’t show a patch that doesn’t apply. - Airflow: An Airflow operation to clear the failed task, clear the failed run, or trigger a new run. The fix links to the run or task in Airflow.
- Manual: Written instructions, for example a fix to a connection or Deployment configuration. If the instructions include an
astrocommand, the command contains the IDs from the investigation and<SET_ME>placeholders for values that Otto doesn’t know. Replace the placeholders before you run the command.
Continue an investigation in your local project
You can send a completed investigation to Otto in the Astro CLI, or to another coding agent, to fix the code in your local Astro project.-
To continue with Otto, click Copy Otto command on the investigation. Then run the command in your terminal from the Astro project that contains the Dag:
Otto starts a new interactive session seeded with the investigation’s diagnosis, and checks the root cause against your code before it makes the fix. If the investigation is still running, the command reports that and exits without starting a session. See
astro otto. - To continue with another agent, such as Claude Code or Cursor, click the arrow next to Copy Otto command and select Copy prompt for Claude Code, Cursor and other agents. Paste the prompt into your agent. The prompt contains the full diagnosis and instructions to fix it.
Run an automatic investigation
To automatically investigate your critical Dags as soon as they fail, configure Astro alerts to call the investigation API on Dag failure. Otto runs the investigation, and your investigation Dag handles the response.- Create a Dag that calls the investigation API and routes the response to the right downstream action.
- Deploy the Dag to Astro.
- Set up a Dag failure Astro alert that monitors your critical Dags.
- Set the notification channel to Dag Trigger and select the Dag you deployed in step 2.
- Optionally, tailor investigations by adding Otto investigation guidance at the Workspace or Deployment level. See Customize Otto investigation guidance.
- Notify: Post the investigation results to Slack, email, or another notification channel.
- Open a PR: Apply the suggested fix as a pull request.
- Open an incident: Create an incident in ServiceNow, PagerDuty, or another incident management system.
Example: Post the diagnosis to Slack
The following Dag accepts the Dag Trigger notification payload, calls the investigation API, reads the streamed diagnosis, and posts a Slack message with the root cause type, severity, summary, and suggested fix. Before deploying, configure these as Airflow variables or environment variables on the Deployment:astro_organization_idorASTRO_ORGANIZATION_ID: The ID of the Organization that owns the Deployment you investigate.astro_deployment_idorASTRO_DEPLOYMENT_ID: The ID of the Deployment you investigate.astro_api_tokenorASTRO_API_TOKEN: A Deployment API token with permission to read the Deployment.slack_webhook_urlorSLACK_WEBHOOK_URL: A Slack incoming webhook URL.slack_channelorSLACK_CHANNEL: Optional. The Slack channel to post to, if not the webhook’s default.
Add context to a Dag run
Otto uses Airflow Dag run notes as Dag-level memory. When you add a note to a Dag run, future investigations of that Dag read it. Use notes to record context that the logs and code don’t show, such as a known upstream outage, a business reason for a failure, or the fix that worked last time. Astro saves the note as the Dag run note in Airflow, so the note also appears in the Airflow UI. Otto reads every Dag run note, including notes that users wrote directly in Airflow.Prerequisites
- A Deployment on Astro Runtime 7.0.0 or later (Airflow 2.5 or later).
- A role with the
deployment.airflow.dagRun.updatepermission, such as Workspace Author or higher. See Workspace role permissions.
Add a note
- Open a completed investigation for a Dag run.
- Click Add Context to This Dag Run. Otto drafts a note from the investigation, with the summary, root cause, and suggested fix.
- Edit the draft. Add the context that only your team knows.
- Click Save Note.
How Otto uses notes
When Otto investigates a failed Dag run, it reads the notes on recent runs of the same Dag, in any state. In Organizations that use Astro Observe, Otto can also search notes on older runs and on related Dags that share a data source. Otto treats a note as a lead to check against the evidence, not as proof. When a diagnosis relies on a note, the note appears in the Evidence section of the investigation.Customize Otto investigation guidance
You can add Otto investigation guidance at the Workspace or Deployment level to tailor investigations to your environment. For example, you can instruct Otto to treat tasks that begin withvalidate as non-blocking, or to interpret specific log patterns in a particular way. Deployments inherit Workspace guidance by default and can override it.
See Configure Otto investigation guidance for a Workspace and Configure Otto investigation guidance for a Deployment.