Be Our Guest
Interested in being a guest on The Data Flowcast? Fill out the form and we will be in touch.
Using Airflow To Power Machine Learning Pipelines at Optimove with Vasyl Vasyuta
Data orchestration and machine learning are shaping how organizations handle massive datasets and drive customer-focused strategies. Tools like Apache Airflow are central to this transformation. In this episode, Vasyl Vasyuta, R&D Team Leader at Optimove, joins us to discuss how his team leverages Airflow to optimize data processing, orchestrate machine learning models and create personalized customer experiences.
Key Takeaways:
- (01:59) Optimove tailors marketing notifications with personalized customer journeys.
- (04:25) Airflow orchestrates Snowflake procedures for massive datasets.
- (05:11) DAGs manage workflows with branching and replay plugins.
- (05:41) The “Joystick” plugin enables seamless data replays.
- (09:33) Airflow supports MLOps for customer data grouping.
- (11:15) Machine learning predicts customer behavior for better campaigns.
- (13:20) Thousands of DAGs run every five minutes for data processing.
- (15:36) Custom versioning allows rollbacks and gradual rollouts.
- (18:00) Airflow logs enhance operational observability.
- (23:00) DAG versioning in Airflow 3.0 could boost efficiency.
Resources Mentioned:
Thanks for listening to “The Data Flowcast: Mastering Airflow for Data Engineering & AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.
