Run LLMOps in production with Apache Airflow

Watch Video On Demand

Hosted By

  • Michael Gregory
  • Tamara Fingerlin
  • Kenten Danas

In the past months, Large Language Models (LLMs) have changed the data science landscape and revolutionized the application of machine learning technologies. There is hardly a data-driven organization which is not looking into ways to leverage LLMs for augmenting user-facing applications, creating new products, and supercharging their internal analytics. 

Apache Airflow has the world’s most active open-source community and is the leading workflow and data orchestration framework. As such, Airflow plays an important role in machine learning operations. When it comes to LLMs, many organizations are also looking to Airflow to operationalize their Retrieval Augmented Generation pipelines.

At Astronomer, we have built templates and reference architectures for operationalizing LLM workflows.In this webinar, we showed how to use the Ask Astro LLMOps reference architecture to create your own retrieval augmented LLM application, feeding state of the art models with domain specific knowledge. You can find the demo repository here, and a write-up of the code and use case here.

Ready to Get Started?

Get Started Free

Try Astro free for 14 days and power your next big data project.