ELT for Beginners: Extract from S3, Load to Databricks and Run Transformations
This blog post shows how you can create a best-practice daily ELT pipeline to move data from S3 into Databricks and run transformations defined in Databricks Notebooks.
Astronomer Blog
The latest insights from our team of Apache Airflow® experts.
Airflow in Action: ML and LLM Ops Insights from ASAPP — Reducing Workflow Runtimes by 85%
Discover how ASAPP turbocharges MLOps with Airflow and Spark, scaling LLM workflows to boost efficiency & achieve lightning-fast processing in hours vs days.
Airflow in Action: Infrastructure Management Insights from 1 Million Monthly Deploys at LinkedIn
Discover how LinkedIn uses Airflow to orchestrate 12k pipelines managing 1m deploys for 7k services. And Airflow 3.0 will make provisioning even easier!
When Should You Start to Warehouse Your Data?
Learn how to leverage your business data with data warehousing. Discover the best time to create a data warehouse, and see which warehousing tools to use.
Why We Drove to NY and Back Over the Past 48 Hours for a 15-Minute Meeting
Find out why it was worth driving from Cincinnati to New York for a fifteen-minute meeting with the number one ranked startup accelerator in the world.
