Real-time data expectations are reshaping how modern data teams think about orchestration and dependencies. As event-driven architectures become more common, teams need to rethink how pipelines react to data changes, rather than schedules.

In this episode, Andrea Bombino, Co-Founder and Head of Analytics Engineering at Astrafy, joins us to discuss how event-driven scheduling in Airflow is evolving and how Astrafy applies it to deliver faster, more responsive data pipelines.

Key Takeaways:

• 00:00 Introduction.
• 02:02 Astrafy’s role in guiding clients across the modern data stack.
• 03:15 Strong DAG dependencies create challenges for time-based scheduling.
• 04:48 Event-driven pipelines respond to increasing real-time data demands.
• 05:30 Airflow 3 introduces native support for event-driven orchestration.
• 06:27 Sensor-based workflows reveal scalability and efficiency limitations.
• 11:32 Event-driven assets improve efficiency and pipeline elegance.
• 14:45 Governance and cross-instance coordination emerge as ongoing challenges.

Resources Mentioned:

• Astrafy website
• Apache Airflow
• Google Cloud
• Google Pub/Sub
• Google BigQuery

Thanks for listening to “The Data Flowcast: Mastering Apache Airflow^® for Data Engineering and AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.