> ## Documentation Index
> Fetch the complete documentation index at: https://astronomer.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Apache Airflow® Executors

Executors are a configuration property of the [scheduler process](/docs/learn/2.x/airflow-components) of every [Apache Airflow®](https://airflow.apache.org) environment. The executor you choose for a task determines where and how a task is run. You can choose from several pre-configured executors that are designed for different use cases, or you can define a [custom executor](https://airflow.apache.org/docs/apache-airflow/stable/executor/index.html). Airflow 2.10 introduced the [experimental multiple executor configuration](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/index.html#using-multiple-executors-concurrently) feature, which allows you to specify executors for individual tasks without needing to use pre-configured combined executor options.

In this guide, you'll learn how to choose and configure executors in Airflow.

<Info>
  To learn more about how to adjust scaling parameters for task execution in Airflow, see [Scaling Airflow to optimize performance](/docs/learn/airflow-scaling-workers).
</Info>

## Assumed knowledge

To get the most out of this guide, you should have an understanding of:

* Basic Airflow concepts. See [Introduction to Apache Airflow](/docs/learn/2.x/intro-to-airflow).
* Airflow components. See [Airflow components](/docs/learn/2.x/airflow-components).

## Choose an executor

There are several pre-configured executors in Airflow for local and production use cases. In production, Astronomer recommends using either of the following executors:

* **[CeleryExecutor](https://airflow.apache.org/docs/apache-airflow/stable/executor/celery.html):** Uses a Celery backend (such as Redis, RabbitMq, Redis Sentinel or another message queue system) to coordinate tasks between pre-configured workers. This executor is ideal for high volumes of shorter running tasks or in environments with consistent task loads. The CeleryExecutor is available as part of the [Celery provider](https://airflow.apache.org/registry/providers/celery/).
* **[KubernetesExecutor](https://airflow.apache.org/docs/apache-airflow/stable/executor/kubernetes.html):** Calls the Kubernetes API to create a separate Kubernetes pod for each task to run, enabling users to pass in custom configurations for each of their tasks and use resources efficiently. The KubernetesExecutor is available as part of the [CNCF Kubernetes provider](https://airflow.apache.org/registry/providers/cncf-kubernetes/). This executor is ideal in the following scenarios:

  * You have long running tasks that you don't want to be interrupted by code deploys or Airflow updates.
  * Your tasks require very specific resource configurations.
  * Your tasks run infrequently, and you don't want to incur worker resource costs when they aren't running.

For local development, Astronomer recommends using the **[LocalExecutor](https://airflow.apache.org/docs/apache-airflow/stable/executor/local.html)**. It executes tasks locally inside the scheduler process and doesn't require workers. It supports parallelism and hyperthreading.

Other available executors are the [SequentialExecutor](https://airflow.apache.org/docs/apache-airflow/stable/executor/sequential.html), the [experimental AWS ECS Executor](https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/executors/ecs-executor.html) and the [experimental AWS Batch Executor](https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/executors/batch-executor.html).

Additionally, you can write and use your own [custom executor](https://airflow.apache.org/docs/apache-airflow/stable/executor/index.html).

<Info>
  Two statically coded hybrid executors exist, the [CeleryKubernetes Executor](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/celery_kubernetes.html) and the [LocalKubernetes Executor](https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/local_kubernetes_executor.html), allowing you to use two different executors in the same Airflow environment in versions 2.9 and earlier. These executors are rarely used and as of Airflow 2.10 [no longer recommended](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/index.html#statically-coded-hybrid-executors).
</Info>

## Configure your executor on Astro

Astro users can choose and configure their executors when creating a deployment. See [Manage Airflow executors on Astro](/docs/astro/executors-overview) for more information. Astro supports the CeleryExecutor and the KubernetesExecutor.

Astronomer's open-source local development environment, the [Astro CLI](/docs/cli/v1.43/overview), uses the LocalExecutor.

## Configure your executor for self-hosted Airflow

When working with self-hosted Airflow solutions, you can set your executor using the [`core.executor` Airflow config variable](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#executor).

Note that when using self-hosted Airflow with executors appropriate for production, you will need to configure your own Celery or Kubernetes setup. For more information on available configuration parameters, see the configuration references for the [CeleryExecutor](https://airflow.apache.org/docs/apache-airflow-providers-celery/stable/configurations-ref.html) and [KubernetesExecutor](https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/configurations-ref.html).

### Run multiple executors concurrently

In Airflow 2.10+, you can run a multi-executor configuration. Note that this feature is experimental. When using multiple executors, you need to provide the relevant classes to the [`core.executor` Airflow config variable](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#executor) as a comma separated string. You can provide short names for your executor classes after a `:` character:

```text wrap theme={null}
[core]
executor = 'CeleryExecutor,KubernetesExecutor,my_custom_package.MyCustomExecutor:MyExecutor'
```

The first executor in the list is the default executor. To assign a specific task to another executor from the list, set its `executor` parameter to the class name or short name of the executor.

```python wrap theme={null}
# from airflow.decorators import task

@task(executor="MyExecutor")
def my_task_with_custom_execution():
    print("Hi! :)")

my_task_with_custom_execution()
```

<details>
  <summary>Traditional</summary>

  ```python wrap theme={null}
  # from airflow.operators.bash import BashOperator

  BashOperator(
      task_id="my_task_in_its_own_pod",
      executor="KubernetesExecutor",
      bash_command="echo 'hi :)'",
  )
  ```
</details>

You can also override the default executor for all tasks in one DAG using the `default_args` DAG parameter. Note that this feature isn't yet supported by Astro Runtime.