Orchestrate next-generation AI tools: Explore six new Airflow providers

2023 was a year of monumental leaps in Artificial Intelligence (AI) innovation with cloud-based large language models (LLMs), which opened up a multitude of new use cases for data professionals. A paradigm shift led to the formation of large language model operations (LLMOps) as part of the modern data stack, with Apache Airflow at the center seamlessly integrating data engineering with machine learning operations (MLOps), including LLMOps.

This integration just got a whole lot easier with the release of six new Airflow providers for popular LLMOps tools. You can now use operators and hooks to call OpenAI and Cohere endpoints, interact with vector databases like Weaviate, Pinecone and pgvector, as well as orchestrate OpenSearch operations—all within your existing Airflow environments.

In this webinar we explore these providers, and show how you too can leverage them to take your LLMOps to the next level. You can find the code covered in this webinar here.