Airflow in Action: How Ramp Scaled from One Notebook to 100+ ML Models and On-Call Agents

At Airflow Summit 2026, Ryan Stevens, Director of Applied Science at Ramp, showed how his team grew from one scheduled notebook to more than 100 production ML models, with Airflow orchestrating every step. The session shows how Ramp pulled daily credit risk and spend forecasts forward by hours, standardized model delivery with the goal of adding models faster than engineers, and put AI agents to work diagnosing failed pipelines.

First principles: every ML system is three jobs

Ramp is a financial technology platform serving more than 70,000 businesses with charge cards, expense management, bill pay, treasury and travel. Ryan leads its applied science team and previously worked on ads ranking at Meta. He reduces every ML system to three jobs (build features, train a model, run predictions) and frames Ramp's path to scale in three phases: crawl, walk and run.

Figure 1: Every ML system reduces to three jobs, from raw data to training to predictions and decisions. Image source.

Crawl: notebooks do not survive success

In the crawl phase, Ramp's first model, a daily business risk assessment, ran in a scheduled Databricks notebook. Ryan argues a notebook plus cron is enough while a team proves ML delivers value in the product. Once models became critical to business decisions, the setup started to break. A failure midway through the notebook forced a full rerun from the first cell.

Ramp needed task-level retries, durability and coordination with dbt. Its dbt ETL jobs already ran on Airflow, so extending the same orchestrator to ML let training and prediction follow the data that fed them.

Walk: couple prediction to data, not the clock

In the walk phase, Airflow became the orchestrator for prediction as well as ETL, coupling the two so prediction ran after ETL rather than on a fixed schedule. Sensors in reschedule mode waited for the dbt build to finish, freeing workers between checks. Then Ramp's dbt project grew past 10,000 models, and full builds stretched to four to six hours, pushing risk and spend forecasts to midday instead of 9 a.m. Eastern. The team moved to table-level coupling: a custom sensor queries Snowflake's information schema and fires prediction as soon as the specific feature table updates, cutting latency by hours.

Once the business saw what its first models could do, it asked for more, and Ramp's model count climbed on a hockey-stick curve. Hand-building a pipeline for each model would not scale, so the team needed a standard way to create and run them.

Run: Turbo defines what, Airflow decides when

The run phase called for Turbo, Ramp's internal ML platform.

Figure 2: Ramp's Turbo ML platform, with Airflow on Astro scheduling every train and predict job. Image source.

Model authors write a YAML JobSpec declaring data, features, label, model type and schedule. Turbo defines what runs, Airflow on Astro decides when, Metaflow executes and MLflow stores the model. Airflow dynamically generates Dags from a database of registered models, including a table freshness sensor and a predict task in each.

Ryan credits this framework with scaling to handle Ramp's model growth. His argument is that AI agents perform best on compressed information: searching an entire codebase for how to build a model is expensive and error-prone, but a short YAML JobSpec captures everything a model needs in one readable file. With validation on each JobSpec and a single agent skill that teaches the format, agents now author new models correctly inside the framework, with only the most complex models still built by hand.

Building is free. Maintenance is not.

Agents have made writing code nearly free, which moves the real cost of ML into what happens after launch. "Building is cheap. It's zero cost these days. Maintenance still exists," Ryan said. He pointed to mature companies with large ML organizations and argued that much of their headcount exists to keep deployed models alive: retraining them, chasing errors and studying their behavior to improve them. Ramp's goal runs the other way. Model count should grow linearly or faster while headcount grows sublinearly, so the number of models each person supports keeps rising.

A single framework is how Ramp bends that curve to meet its goal. Because every ML job runs through Turbo and Airflow, it inherits the same Slack failure callback without extra work from the model author. Each alert opens an incident for an on-call agent that keeps its own instructions and memories for specific teams, models and past incidents. The agent summarizes what failed, recommends a fix and proposes a pull request, often before the on-call engineer has looked at the alert.

That loop only works because every job follows one path. Organizations with legacy stacks of isolated systems miss the benefit of agentic operations because agents perform best when every job is built from the same framework, writes to the same schemas and follows the same conventions. Ryan acknowledged the tradeoffs of forcing everything through one framework, but argued that the maintenance savings outweigh them, and that centralization matters more now that creating code costs almost nothing.

Build on Airflow, not around it

Ryan closed with three lessons for any team scaling ML on Airflow:

• Model the real dependency. Airflow should wait on the condition that makes downstream work safe to run, which for Ramp meant a specific feature table updating rather than an entire dbt build finishing.
• Centralize repeated Airflow behavior. Sensors and failure alerting belong in one Dag factory instead of in every model, so each new model inherits them and agents have a single pattern to follow.
• Keep Airflow as the control plane. Higher-level ML abstractions such as Turbo sit on top of Airflow while compute runs elsewhere. That separation has let Ramp swap in at least three drift and anomaly detection systems and scale Metaflow independently, while Airflow continued to coordinate the work.

What’s next?

Watch the session replay to see Ryan's code for table-level sensors and dynamic Dag generation, a walkthrough of the Turbo JobSpec that agents build models against, and his case for why centralized frameworks matter more as the cost of writing code falls toward zero.

Ramp runs its ML control plane on Airflow with Astro. Start building on Astro, the unified orchestration platform for Apache Airflow.