Your First Airflow Job Starts Here: How BU Questrom Trains Data Engineers on Astro

The Customer

Boston University's Questrom School of Business launched its Master of Science in Business Analytics (MSBA) in 2019 on a conviction that has only grown more urgent since: the business leaders of tomorrow need more than exposure to data. They need to understand how it works, how it fuels AI, and how to apply both in ways that drive real outcomes for real organizations. The program embedded AI and machine learning into its core from day one, treating them as foundational competencies rather than electives, and always through the lens that defines a business school: what does this actually do for the business? Not AI for its own sake, but data and AI in service of decisions, products, and growth.

Brock Tibert has been central to that vision for nearly eight years, first as a Lecturer and now as Senior Lecturer and Faculty Director of the MSBA. With more than two decades of industry experience building data products, leading data teams, and advising organizations at the intersection of product and data, he brings an applied lens to his courses: not just what data teams do today, but where they need to be.

One of the program's three concentration tracks, the Data and Methods Concentration, is designed for students targeting roles in data engineering, machine learning, and applied data science. BA882: Deploying Analytics Pipelines sits squarely in that track. Built by Tibert and now one of the program's most technically demanding electives, it asks students to take everything they have learned and answer a practical question: what does it actually mean to ship analytics, ML models, and AI-infused pipelines to production?

The Challenge

BA882 students spend the full semester running live infrastructure. They are expected to:

• Ingest live data from public APIs and sources updating weekly or even hourly, flowing continuously into a cloud data warehouse
• Retrain and evaluate machine learning models in the cloud, with branching logic to decide automatically whether to promote a model to production based on performance metrics
• Orchestrate AI workflows that coordinate tasks like chunking, embedding text data, and running LLM inference, making datasets accessible to downstream agents

All of it needed to run in the cloud, all semester, on real infrastructure. Tibert wanted students to wrestle with real trade-offs: what happens when a deployment runs constantly and burns through credits, how to debug a job that fails in the cloud, why it makes sense to separate orchestration from compute. But he could not afford for those trade-offs to swamp a team's final project. As he put it: "When students are doing their final project and they run out of free resources, their infrastructure is locked. What do I do now? That kind of panic, I needed to eliminate it entirely."

Managing Kubernetes infrastructure was equally off the table. Cluster configuration and maintenance weren't part of the learning goals, and he didn't want them becoming a distraction from what the course was actually about. He knew he needed a managed service.

The Solution

The tool choice was Airflow: widely adopted in production data teams, a platform Tibert knew from years of industry work, and a credential with real weight in the job market.

Tibert reached out to Astronomer to see whether they could support the course. He had seen firsthand what happens when teams self-manage Airflow on underpowered infrastructure: pipelines fail, engineers blame Airflow, and the real culprit (an undersized VM or a tangled configuration) goes undiagnosed. Astro removed that entire failure mode from the equation. Students would work with Airflow as a production-grade managed service, not a configuration project.

Astronomer provided Astro credits and helped spin up a separate Astro workspace for each project team, with technical guidance on workspace configuration and how to use dev Deployments and hibernation to keep costs under control. The setup mirrored, as closely as possible, how a real data team would operate on Astro.

The course ran in three phases, each building on the last:

• Phase 1: Analytics Engineering. Teams of students defined a problem, identified live data sources, and built pipelines that ingested and transformed data throughout the semester into BigQuery (via GCP education credits) or Mother Duck as a cloud data warehouse, and leveraged Tableau or Apache Superset as their BI tool.
• Phase 2: MLOps. Teams deployed machine learning models to the cloud, scheduled retraining runs, and used Airflow branching logic to evaluate model performance and decide whether to promote to production.
• Phase 3: AI Orchestration. The final phase used Airflow to orchestrate AI tasks end to end. Teams working with text-heavy data sources built pipelines that handled chunking, embedding, storage, and LLM inference. The framing: how do data teams make their data accessible to agents?

The Astro CLI anchored the course's most important teaching moment. Students would build and test locally in a familiar environment, then run astro deploy and watch their work appear in the cloud.

The Results

Across a single semester, almost 50 MSBA students across 10 project teams moved from Jupyter Notebooks to running live, multi-phase data pipelines in the cloud, spanning analytics engineering, MLOps, and AI-orchestrated workflows, all on Airflow via Astro. Across those teams, students deployed 88 DAGs for their final projects, ranging from 5 to 17 per team. Some ran on schedules; others fired in response to upstream DAGs completing, reflecting the kind of dependency-aware orchestration logic that defines real production pipelines.

The range of what those teams actually built makes the point. The problems were different; the infrastructure was the same:

• Outbreak detection. A team built pipelines ingesting CDC public health data to model and monitor disease trends.
• NCAA football rankings. One team built their own ranking model, deploying it right as bowl season arrived and the official rankings dropped.
• AI news discovery. Another created a web app surfacing AI news, blog posts, and arXiv papers as personalized recommendations.
• Inventory intelligence. One team orchestrated an analytics layer for inventory management decision-making.
• Hiring trends. Another parsed job market APIs to surface real-time hiring signals for data roles.
• Boston transit. One team analyzed MBTA and Blue Bike ridership data to understand commute delays and student mobility patterns across the city.

For Tibert, the clearest signal of success was what he didn't hear. Students weren't coming to office hours stuck on the platform. No team hit a credit ceiling during finals. No one panicked about getting their project to run in the final stretch. He calls the overall response "frictionless," and after years of teaching, he reads the absence of complaints as its own form of endorsement.

The career outcomes are already visible. Graduates are landing data science and ML engineering roles and pointing directly to hands-on Airflow experience as a differentiator in their job search. For Tibert, that is the proof of concept: students leave BA882 with real, cloud-based Airflow on their resume, a credential that is increasingly recognized by the teams they are joining.

What's Next

Tibert is running BA882 again in the fall, with enrollment already up to 55 students. The next iteration will formalize dbt as a teaching component, and he is exploring Astronomer Cosmos as a way to orchestrate dbt runs through Airflow, giving students exposure to another pairing they will encounter in the field.

Tibert sees the course as a living curriculum because the underlying landscape keeps shifting: data that was never part of any pipeline, documents, internal knowledge bases, unstructured content, now needs to be chunked, embedded, and made accessible to agents. The scope of what a data team owns is expanding, and the orchestration skills students practice in BA882 sit right at the center of it.