Security is one of the least visible parts of Airflow for most users, but it shapes what features teams can turn on in production. In this episode, Kenten Danas talks with Anish Giri, Senior Software Engineer and active Apache Airflow contributor focused on security, about three recent contributions that touch nearly every Airflow 3 deployment: hardening connection testing, adding JWT revocation, and splitting execution tokens by scope.
Key Takeaways:
- 00:00 Introduction.
- 01:16 Anish's inside-out path into Airflow, starting from source code and docs rather than running pipelines in production.
- 03:13 Why security engineering flips the usual software question from "what can this do?" to "what can anyone make this to?"
- 04:49 How connection testing worked before, why it was disabled in Airflow 2.7 after a CVE, and how the redesign moves execution off the API server.
- 06:26 Running connection tests on the worker in Airflow 3 so credentials never leave the API server boundary.
- 07:50 What the mailing list and architecture review looked like, including a first-round rejection and eventual consensus.
- 10:00 The JWT logout problem in Airflow 3 and the revocation table that makes logout an actual logout.
- 13:12 A production incident with a GPU pipeline task hitting 403 after 15 minutes, and why it required splitting the token in two.
- 15:41 How the workload token and the auth token now have different lifetimes and scopes, tied to the task queue timeout.
- 19:53 Advice for new Airflow contributors: start small, don't fear the PR review, and treat feedback as mentorship.
Resources Mentioned:
Thanks for listening to "The Data Flowcast: Mastering Apache Airflow® for Data Engineering and AI." If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.
