Skip to main content

​
Overview

Extracting structured fields from free-form text is a common LLM use case. Downstream tasks can use fields extracted by AI in many different situations, for example loading them into a database or turning them into features for a traditional machine learning model, so in this pattern, the AI step is a helper for a conventional ETL or MLOps pipeline. This architecture is an MLOps pipeline for a sales team predicting expected revenue on open deals with a regression model. Predicting a continuous value is a regression problem, which a traditional machine learning model is best suited for. The LLM’s task is part of the feature engineering: turning call transcripts, sales emails, and account research into the numeric and categorical features the model needs.
For general information on how to orchestrate LLM calls with Airflow, see LLM orchestration with Apache Airflow®.

​
Architecture

Structured extraction reference architecture diagram. A monthly Dag extracts call transcripts, sales emails, account research, and known deal outcomes, pulls features with a mapped @task.llm call, and retrains a regression model. A daily Dag runs the same extraction on in-flight deals, runs model inference, and sends an expected revenue report.
This architecture consists of two Dags that use parts of the same modularized code:
  • Monthly training Dag: Extracts the unstructured sources plus the known deal outcomes, runs the LLM extraction, processes the extracted features, loads them as training data, splits into test and train sets, and retrains the model.
  • Daily inference Dag: Extracts the same unstructured sources for deals with no known outcome yet, runs the same LLM extraction with the same output class, loads the features as inference data, runs the regression model, and sends an expected revenue report to sales leaders.
The code shared between the two Dags is stored in a module that both Dags import. If the training extraction and the inference extraction were to drift apart, the model would be served features that do not match what it learned from, and the predictions would degrade.

​
Airflow features

  • Dynamic task mapping: Extraction is mapped over the list of deals, so both Dags scale with how many deals are in scope for that run.
  • @task.llm: Runs the feature extraction as a single model call per input.
  • Structured output: Because the output_type is defined as a Pydantic model, you can rely on the AI output fitting the defined schema for downstream tasks.
  • Automatic retries: Extraction calls retry per deal in case of rate limits.

​
Considerations

  • Make sure the Dags use the same extraction step. The training and inference paths have to produce an identical feature set. Put the output class, the prompt, and the formatting in one importable module.
  • Keep the extracted features inspectable. Loading the intermediate features to a table helps with debugging when prediction accuracy drops over time.
  • Watch for changes in the unstructured sources. The model might need retraining or changes in its feature engineering if the sales team changes how it writes call notes.
  • Version the output class alongside the model. When the extraction fields change, the model needs retraining before the inference Dag uses the new fields.

​
Next steps