How to Orchestrate Databricks Jobs Using Airflow

Watch On Demand

Hosted By

  • Daniel Imberman
  • Kenten Danas

This “Live with Astronomer” session covers how to use the Astro Databricks provider to orchestrate your Databricks Jobs from Airflow. This new provider allows you to monitor your Databricks Jobs from Airflow, write tasks that run Databricks Jobs using the DAG API you’re familiar with, and even send repair requests to Databricks when tasks fail.

Questions covered in this session include:

Learn more about the Astro Databricks Provider and see example code in the official repo.