An introduction to the Airflow UI

One of the main features of Apache Airflow is its user interface (UI), which provides insights into your DAGs and DAG runs. The UI is essential for understanding, monitoring, and troubleshooting your pipelines.

This guide is an overview of some of the most useful features and visualizations in the Airflow UI. If you're not already using Airflow and want to get it up and running to follow along, see Install the Astro CLI to quickly run Airflow locally.

All images in this guide were taken from an Astronomer Runtime Airflow image. Other than some modified colors and an additional Astronomer tab, the UI is the same as when using OSS Airflow. The images in this guide are from Airflow version 2.10. If you are using an older version of Airflow, some UI elements might be slightly different or missing.

Assumed knowledge​

To get the most out of this guide, you should have an understanding of:

• Basic Airflow concepts. See Introduction to Apache Airflow.
• Airflow DAGs. See Introduction to Airflow DAGs.

DAGs​

The DAGs view is the landing page when you sign in to Airflow. It shows a list of all your DAGs, the status of recent DAG runs and tasks, the time of the last DAG run, and basic metadata about the DAG like the owner and the schedule. To see the status of the DAGs update in real time, toggle Auto-refresh.

In the DAGs view you can:

• Pause/unpause a DAG with the toggle to the left of the DAG name.
• Reparse a DAG without waiting for Airflow to refresh automatically.
• Filter the list of DAGs to show active, paused, or all DAGs.
• Filter the list of DAGs to show currently running DAGs or DAGs that failed their last DAG run.
• Trigger or delete a DAG with the buttons in the Actions section.
• Navigate quickly to other DAG-specific views from the Links section.

To see more information about a specific DAG, click its name or use one of the links.

Individual DAG view​

The DAG view gives you detailed insights into a specific DAG, including its DAG runs and task instances.

On the left side you can see a grid representation of the DAG's previous runs, including their duration and the outcome of all individual task instances. Each column represents a DAG run, and each square represents a task instance in that DAG run. Task instances are color-coded according to their status. A small play icon on a DAG run indicates that a run was triggered manually, and a small dataset icon shows that a run was triggered by a dataset update. If no icon is shown, the DAG ran according to its schedule.

On the right side you can see further details about the item (DAG, DAG run or task instance) that is currently selected.

When a DAG run, task instance, or task group instance is selected in the DAG grid, several action buttons appear:

• Clear / Clear task : This button will clear the selected DAG run, task group instance, or task instance and run it again. This is useful if you want to re-run a task or DAG run that has failed or during local development. After clicking Clear task you will be offered a detailed interface controlling which task instances should be cleared and rerun. See Manually rerun tasks or DAGs.
• Mark state as...: This button allows you to mark the selected DAG run, task group instance or task instance as successful or failed without running it. This option is often useful when the root cause of a task failure was fixed manually in the external data tool and there's no need to rerun the task. Many data teams leverage Task Instance Notes and DAG Run Notes in order to document the reason for marking a task instance as failed or successful.
• Filter DAG by task: This button allows you to filter the tasks shown in the DAG grid and DAG graph based on task dependencies. For example, when you select Filter downstream, the UI shows only the tasks downstream of your selected task.

There are several tabs available within the DAG view:

• Details: Shows more details about the DAG, DAG run or task instance.
• Graph: Shows a graph representation of the DAG or DAG run.
• Gantt: Shows the duration of each task instance in a DAG run as a Gantt chart.
• Code: Shows the DAG code.
• Audit Logs: Shows the audit logs of the DAG, DAG run or task instance.
• Run Duration: (only available when the DAG itself is selected) Shows the duration of each DAG run over time.
• Calendar: (only available when the DAG itself is selected) Shows the state of DAG runs overlaid on a calendar.
• Logs: (only available when a task instance is selected) Shows the logs of a task instance.
• XCom: (only available when a task instance is selected) Shows the XComs pushed by a task instance.
• Task Duration: (only available when a task instance is selected) Shows the duration of task instances of a task over time.

Details​

The Details tab displays information about the DAG, individual DAG runs, and in-depth information about each task instance. Here you can find information like total historic runs of a DAG, the data interval start of a DAG run, and the duration of a task instance.

To access the details of a specific DAG run or task instance, you first need to select it in the DAG grid as shown in the following gif:

If a task has been retried, you can also click on a task try button to get details about a specific attempt:

When you select a task instance in the DAG grid, two additional options appear underneath the tabs:

• More Details: Shows all attributes of a task, including variables and templates.
• List All Instances: Shows a historical view of task instances and statuses for that particular task.

Graph​

The Graph tab shows a graph visualization of the tasks and dependencies in your DAG, including Airflow datasets a DAG is scheduled on or updates. If you select a task or task group instance in a DAG grid column, the graph highlights and zooms to the selected task. You can also navigate complex DAGs using the Filter Tasks option and the minimap. This view is useful for exploring the DAG structure and task dependencies.

Gantt Chart​

The Gantt chart tab visualizes task duration and overlap, so you can quickly identify bottlenecks and costly tasks.

You can also find information about previous retries if a task has been retried:

Code​

Under the Code tab you can access the code that generates the DAG you are viewing. While your code should live in source control, the Code tab provides a quick insight into what is going on in the DAG. DAG code can't be edited in the UI.

This tab shows code only from the file that generated the DAG. It does not show any code that may be imported in the DAG, such as custom hooks or operators or code in your /include directory.

Audit Logs​

The Audit Logs tab shows a list of events that have occurred in your Airflow environment relating to the DAG, DAG run or task instance you have selected.

Run Duration​

The Run Duration tab shows a bar chart of the duration of each DAG run over time.

Kubernetes Pod Spec​

The K8s Pod Spec tab displays information about a task's K8s pod if a task has been run on one.

Task Duration​

The Task Duration tab shows a line chart of the duration of each task across all DAG runs.

Logs​

To access the logs of a specific task instance, click on the Logs tab which appears as soon as you select a task instance. By default the logs of the task execution are shown, while the Pre task execution logs and the Post task execution logs are collapsed and can be expanded by clicking on the respective log item.

Click on the Logs tab to see the logs for a selected task. If a task has been retried, you can click on a task try button to see the logs for that attempt. Hover over a button for status and duration information at a glance.

Cluster activity tab​

The cluster activity tab shows aggregated metrics for the entire Airflow cluster. It includes live metrics, such as currently occupied slots in different pools, unpaused DAGs, and scheduler health. It also includes historical metrics like the states of past DAG runs and task instances, as well as how each DAG run was triggered.

Datasets tab​

The Dataset tab links to a page showing all datasets that have been produced in the Airflow environment, with additional tabs offering:

• a dependency graph of all dependencies between datasets and DAGs.
• a table of datasets containing a URI, the total number of updates, and a timestamp of the last update for each dataset.

Click on a dataset to open the history of all updates to the dataset that were recorded in the Airflow environment. You can use the Play button to manually trigger an update to a Dataset.

Click on the Dependency graph tab to view a holistic graph of datasets and DAGs.

Security tab​

The Security tab links to multiple pages, including List Users and List Roles, that you can use to review and manage Airflow role-based access control (RBAC). For more information on working with RBAC, see Security.

If you are running Airflow on Astronomer, the Astronomer RBAC will extend into Airflow and take precedence. There is no need for you to use Airflow RBAC in addition to Astronomer RBAC. Astronomer RBAC can be managed from the Astronomer UI, so the Security tab might be less relevant for Astronomer users.

Browse tab​

The Browse tab links to multiple pages that provide additional insight into and control over your DAG runs and task instances for all DAGs in one place.

The DAG runs and task instances pages are the easiest way to view and manipulate these objects in aggregate. If you need to re-run tasks in multiple DAG runs, you can do so from this page by selecting all relevant tasks and clearing their status.

The DAG Dependencies view shows a graphical representation of any cross-DAG and dataset dependencies in your Airflow environment.

Other views on the Browse tab include:

• Jobs: Shows a list of all jobs that have been completed. This includes executed tasks as well as scheduler jobs.
• Audit Logs: Shows a list of events that have occurred in your Airflow environment that can be used for auditing purposes.
• Task Reschedules: Shows a list of all tasks that have been rescheduled.
• Triggers: Shows any triggers that occurred in this Airflow environment. To learn more about triggers and related concepts, you can check out the guide on Deferrable Operators.
• SLA Misses: Shows any task instances that have missed their SLAs.

Admin tab​

The Admin tab links to pages for content related to Airflow administration that are not specific to any particular DAG. Many of these pages can be used to both view and modify your Airflow environment.

For example, the Connections page shows all Airflow connections stored in your environment. Click + to add a new connection. For more information, see Managing your Connections in Apache Airflow.

Similarly, the XComs page shows a list of all XComs stored in the metadata database and allows you to easily delete them.

Other pages on the Admin tab include:

• Variables: View and manage Airflow variables.
• Configurations: View the contents of your airflow.cfg file. Note that this can be disabled by your Airflow admin for security reasons.
• Plugins: View any Airflow plugins defined in your environment.
• Providers: View all Airflow providers included in your Airflow environment with their version number.
• Pools: View and manage Airflow pools.

Docs​

The Docs tab provides links to external Airflow resources including:

• Airflow documentation
• The Airflow website
• The Airflow GitHub repo
• The REST API Swagger and the Redoc documentation

Customization options​

In the upper right-hand corner of the UI, you will find a light/dark theme toggle button, time zone selector with search functionality, and a login dropdown with a link to an editable user profile view.

Conclusion​

This guide provided a basic overview of some of the most commonly used features of the Airflow UI.

The Airflow community is consistently working on improvements to the UI to provide a better user experience and additional functionality. Make sure you upgrade your Airflow environment frequently to ensure you are taking advantage of Airflow UI updates as the community releases them.