Secrets Management in Airflow 2.0

WATCH ON DEMAND

Summary:

Best practices for managing Secrets with various backends in Apache Airflow 2.0.

In this webinar we covered:

  • What are Airflow Secrets?
  • Types of Secrets
  • Airflow Metastore DB
  • Alternative Secrets Backend
  • Best Practices
  • And More...

Missed the Webinar? Sign up for the Recap

Recap Preview

What are Airflow Secrets?

Secrets are sensitive information that are used as part of your DAG.

Types of Secrets:

  • Variables are a generic way to store and retrieve arbitrary content or settings as a simple key value store within Airflow.
  • Connections are information needed to connect to external systems. Must be stored in the format of a URL-encoded URI.
  • Configurations are Airflow Configurations that can be stored in your secrets backends.

Where are all the Secrets?

Environment Variables

  • Simple and easy to use, difficult to maintain.
  • No encryption built in.
  • Cannot access them from UI or the Airflow CLI.
  • Using them from Kubernetes secrets is safer.
  • Lives and dies with your Airflow deployment or its configuration.

Hosted By

Viraj Parekh

Viraj Parekh

Field CTO