Airflow at Faire: Democratizing Machine Learning at Scale

Data science and machine learning form the core of Faire’s industry-celebrated marketplace (a16z top-ranked marketplace), driving powerful search, navigation, and risk functions. These functions are powered by ML models trained on over 3000 features defined by their data scientists.

Previously, the process of defining, backfilling and maintaining feature lifecycle was error-prone. To address this challenge, Faire’s data team partnered with Astronomer and chose Apache Airflow on Astro as their tool to democratize their ML feature store framework.

In this webinar, we’ll dive into this story with Faire and discuss:

• How Faire leverages Airflow to power ML training and extends it with a framework that powers feature stores.
• How this framework enables data scientists to define new features and backfill them using dynamic DAGs.
• How they constructed a framework that builds datasets to train models and how empowering end-users with tools frees up engineering teams to focus on strategic initiatives.
• How the Faire team capitalized on Astronomer’s unique capability to provide multi-tenant Airflow, enabling efficient scaling and resource management.