Beyond the Dag hackathon: meet the winners
8 min read |
Beyond the Dag: meet the winners
Modern Airflow has evolved beyond just moving data. That was the idea behind Beyond the Dag, the data engineering hackathon we ran this summer: build something on Apache Airflow 3.1 or later that shows what the platform can do today.
From July 30 to September 24, builders worked on projects across four categories:
| Category | The challenge |
|---|---|
| Plugin Powerhouse | Extend Airflow itself with an Airflow 3.1 plugin for the UI or API. |
| The Agentic Pipeline | Orchestrate intelligence with the new Common AI provider. |
| Keep a Human in the Loop | Make the Dag ask for help with the new human in the loop operators. |
| Airflow Can Do That?! | The wildcard. Use Airflow as the engine for something unexpected. |
25 projects came in. A panel of nine judges, made up of Apache Airflow PMC members and committers, Astronomer engineers, and Astronomer Champions, scored every one of them on creativity and originality, Airflow feature usage, impact and usefulness, demo and story, and technical implementation.
In the submissions, Airflow runs API migrations, red teams pipelines, drives a robot, and heals failed tasks, and a human approves every step that matters.
The winners
And these are the winners who can look forward to receiving some fun and well deserved prizes.
| Award | Project | Built by |
|---|---|---|
| Overall winner | Cascade | Joseph |
| Plugin Powerhouse | Cascade | Joseph |
| The Agentic Pipeline | Premortem | GitHub: catmei |
| Airflow Can Do That?! | DAGstronaut | Rahul Rajasekharan |
| Keep a Human in the Loop | Aqueduct Airflow Provider | Sadig Akhund |
Let's have a look in more detail.
Overall winner and Plugin Powerhouse: Cascade
Cascade by Joseph turns one of the challenging and most error prone jobs in software, shutting down an old API without breaking your customer's workflows, into an Airflow application.
When a vendor turns off API v1, Cascade finds every customer still calling it and tracks each one as its own mapped task. When a customer is blocked, the Dag stops and asks a person what to do. A customer only counts as migrated when their telemetry shows it. All of it lives in a dashboard built right into the Airflow UI with a plugin.
"A very slick and thorough solution to a very common problem." A Beyond the Dag judge
Why it won: Cascade is the clearest answer to "Airflow is more than moving data" in the whole hackathon. It uses mapped tasks, human in the loop, Common AI, and a plugin, and every one of them is there for a reason. Judges kept coming back to how real the use case felt and to the polish of the demo. One judge also liked that Cascade counts thousands of mapped tasks by state instead of paging through them one by one.
Cascade takes home both Best in Plugin Powerhouse and the title of overall winner of Beyond the Dag.
The Agentic Pipeline: Premortem
Every task is green, and the numbers are still wrong. Most data engineers have been there.
Premortem red teams a pipeline before it ships. You describe the pipeline and write an output contract that says what correct output means. Then an agent built with the Common AI provider's @task.agent inspects the pipeline and proposes realistic faults: schema drift, unit shifts, duplicate batches, a flaky weather API. An engineer reviews the plan in a chat inside the Airflow UI, approves it, and the scenarios run in isolated sandboxes.
Why it won: Premortem gets the split between AI and code exactly right. The agents propose and explain. Deterministic Python checks decide every pass or fail. Judges loved the four validation layers and how deeply the plugin is woven into the Airflow UI. In the demo on an NYC taxi and weather pipeline, the agent ranked the replay scenario last, and the deterministic checks still caught the doubled totals.
Airflow Can Do That?!: DAGstronaut
DAGstronaut by Rahul Rajasekharan puts Airflow in charge of a real robot.
It has two personalities. Its emote Dags turn task outcomes into happy and sad rover reactions through success and failure callbacks, so your pipeline literally has feelings now. Its exploration Dag drives a micro:bit rover that detects obstacles, photographs them, and classifies what it sees with local Gemma 3 Vision. Before the rover moves, Airflow pauses and a human makes the call on a Flight Director Console.
"One of the most interesting and actually 'Airflow can do that?' submissions." A Beyond the Dag judge
Why it won: this category asked for the project that makes people say "wait, Airflow can do that?", and DAGstronaut earned full marks for creativity from every single judge. It also brings together a plugin, a Common AI step, and a human in the loop decision, and one judge praised how carefully it draws the line between what the robot does on its own and what Airflow should control.
Keep a Human in the Loop: Aqueduct Airflow Provider
A task fails at night. Normally that means someone reads logs, guesses at the cause, writes a fix, and hopes.
With the Aqueduct Airflow Provider by Sadig Akhund, a heal loop kicks in instead. It diagnoses the failure, writes a structured patch, and checks it five ways: guardrails, engine config, lineage, a sandbox replay, and dependency resolution. Then it stops and hands the decision to a person, with the diff and an Approve button right on the failing task in the Airflow UI.
"Self healing implementation with HITL is the best part of this." A Beyond the Dag judge
Why it won: the AI diagnoses the failure and writes the patch, and a person decides whether it ships. Judges praised the clear demo and described the implementation as solid. That approval step is exactly what the human in the loop operators in Airflow 3.1 were built for.
Honorable mentions
We were highly impressed by the overall quality of submissions, and picking four winners out of 25 was not easy. These three projects were close runners up, and also highlighted Airflow's strength and flexibility in unique ways.
AirflowOS
AirflowOS by Andrii Roiko is a Windows 95 desktop for Airflow, shipped as a native plugin. Task instances are processes, Dag runs are folders, and pools are memory. There is a Task Manager, an Explorer, a Human Input Required inbox, a Deadlines mailbox, a blue screen for failures, and of course Clippy, who triages a failed run with an LLM as a real Dag run.
It finished second in Plugin Powerhouse. It is pure fun, and it also shows how far the Airflow 3.1 plugin system goes. One judge summed it up: "I hate it. It's a brilliant idea."
Throughline
Throughline by Sweta M traces a single record through a Dag and shows exactly what each task did to it: values changed, fields added or dropped, rows multiplied. When a number looks wrong, the shape strip shows where the record fanned out, and you can replay it against the exact code that ran, with production attached read only.
Adoption is one decorator per task, and the result lives as a tab on the Dag page. Judges called it genuinely useful for debugging, and one of them named it one of their favorite submissions. If you have ever hunted for the task where one row quietly became two, you know why.
ACH Payment Fraud Detector
The ACH Payment Fraud Detector combines a machine learning model with human review to catch fraudulent payments. Every flagged payment becomes its own review task with a 24 hour timeout and a safe default, and the dashboard syncs decisions back through Airflow's own API.
Judges praised the clear demo and called it one of the best versions of this pattern in the whole hackathon.
The rest of the field
The other projects brought plenty of surprises too: a plugin that turns the scheduler into a metronome so your Dags make music, a backfill that fixes continuity errors in a story, a Blockly and Raspberry Pi setup that teaches kids to build Dags, and a tool that replays past policy decisions under a new rulebook. None of them is a classic data pipeline, and all of them run on Airflow.
As a thank you, everyone who submitted a project will receive a certificate of participation from us in the coming weeks, together with individual feedback from the judges.
Thank you
Thank you to every participant, to our judges, and to our mentors, who held office hours throughout the build window. Congratulations to all winners.
Every project is open source, so the repositories above are a good place to start if one of them sparked an idea. What will you build beyond the Dag?
