ELT for Beginners: Extract from S3, Load to Databricks and Run Transformations
This blog post shows how you can create a best-practice daily ELT pipeline to move data from S3 into Databricks and run transformations defined in Databricks Notebooks.
Astronomer Blog
The latest insights from our team of Apache Airflow® experts.
Airflow in Action: ML and LLM Ops Insights from ASAPP — Reducing Workflow Runtimes by 85%
Discover how ASAPP turbocharges MLOps with Airflow and Spark, scaling LLM workflows to boost efficiency & achieve lightning-fast processing in hours vs days.
Airflow in Action: Infrastructure Management Insights from 1 Million Monthly Deploys at LinkedIn
Discover how LinkedIn uses Airflow to orchestrate 12k pipelines managing 1m deploys for 7k services. And Airflow 3.0 will make provisioning even easier!
Astronomer v0.10
Learn more about Astronomer v0.10 and its key updated functionalities. See how the new Astronomer Platform supports the latest version of Apache Airflow®.
7 Common Errors to Check When Debugging Airflow DAGs
Get to know best practices for debugging Apache Airflow® DAGs. Check out the list of common Airflow deployment errors, and see how to find and remove them.
Airflow Design Principles: Multi-tenant vs. Monolithic Architecture
Find out how to design an Airflow infrastructure, and whether it makes more sense to power your DAGs with one monolithic Airflow instance, or many.
Astronomer v0.8.0 Release Notes
Discover the newly launched Astronomer v0.8.0 and the features it includes. Find out what’s been fixed, improved, and added to the Astronomer Platform.
Astronomer on Astronomer: Loading Thousands of Files Into Redshift With Apache Airflow®
Get to know why Astronomer switched to Apache Airflow®. Learn how we optimized Airflow to fit our initial needs and what features we're planning to build.
Astronomer v0.7.0 Release Notes
Release notes covering the features released with v0.7.0 of the Astronomer platform.
Astronomer v0.6.0 Release
Release notes for v0.6.0 of the Astronomer platform.
Astronomer v0.5.0 Release
Release notes from our recent platform update to v0.5.0.
Astronomer v0.4.1 Release
Check out the highlights of the Astronomer v0.4.1 release. See the full summary of upgrades and learn more about the Astronomer Platform's new features
