Watch the recording
By proceeding you agree to our Privacy Policy, our Website Terms and to receive emails from Astronomer.
How to use DuckDB with Airflow
Recorded on
Tamara FingerlinSenior Developer AdvocateAstronomer
Jeff FletcherDirector of Field Engineering EMEAAstronomer
This “Live with Astronomer” session covers the best ways to use DuckDB with Airflow. We walk through some basic data processing and machine learning DAGs to show how and where DuckDB can be an optimal fit for your data pipelines.
Questions covered in this session include:
- How can I run DuckDB queries using Airflow?
- How can I use the Astro Python SDK, an open source framework for east DAG authoring, to integrate DuckDB into my data pipelines?
- What are some tips and tricks for using DuckDB and Airflow with larger datasets?
All code shown in this session can be found in this repo. You can read more about using DuckDB with Airflow on the Airflow medium.
