Documentation

Astronomer Roadmap


Below is a summary of our current release plan. If you'd like to request a feature or upvote something you really want, visit our Github.

ReleaseFeatures
v0.7Override environment variables on airflow deployments As a user, I need to configure my SMTP settings (email alerts), settings for parallelism, and other ENV level settings without exposing credentials in my dockerfile.

Worker scaling: Control worker count, Control worker size

Control worker termination grace period As a deployment.owner I should be able to set the grace period for my workers to restart during a code push.

Logging: Add elasticsearch and fluentd components w/ helm as part of the Astronomer Platform initial deployment
v0.8Logging: pipe Airflow task logs from Elasticsearch to Airflow UI, and scheduler and web-server logs from Elasticsearch to Astronomer UI

Improve StatsD metrics Add counters for success and failed dag_runs, tracking dag_id and execution_date

Container Status As a deployment.user, I want to see the status of my airflow cluster components in real time: Webserver, Scheduler, Workers.
v0.9Support for multiple Airflow versions: Airflow 1.8, 1.9, 1.10

Kubernetes Executor: support for Kubernetes Executor, platform owner can set concurrency per Airflow Cluster, Cluster owner CRUD custom task pod sizes, user can change concurrency and worker size for an Airflow cluster
LaterOther OAuth providers, as requested

Audit Logging. Collect audit logs, present in UI.

Airflow package manager. Provide command line interface to bring in plugins.

Improvements to Airflow UI (it should update in real time)

Platform usage metrics. Trends for daily DAGs, tasks, user engagement in UI.