Watch the recording
By proceeding you agree to our Privacy Policy, our Website Terms and to receive emails from Astronomer.
Best Practices for Running Airflow at Scale
Recorded on
Naveen SukumarHead of PMM & DevRelAstronomer
Jake McGrathAssociate Sales EngineerAstronomer
Stephen FaigResearch DirectorUnisphere Research and DBTA
Note: There is a newer version of this webinar available: Best practices for scaling Apache Airflow®.
Leading companies like Uber, and LinkedIn rely on Apache Airflow to orchestrate hundreds of thousands of tasks daily. However, scaling multiple Airflow deployments effectively requires optimized resource management, streamlined workflow development, and robust observability to avoid performance bottlenecks and developer inefficiencies.
Watch this webinar to discover practical strategies for running Airflow efficiently at scale.
You'll learn:
- How to select and optimize Airflow configurations tailored to your workloads.
- Techniques for automating Airflow infrastructure management with Terraform, REST APIs, and self-service platforms.
- Ways to accelerate workflow development using templating tools.
- Approaches to standardize shared code libraries to reduce redundancy.
- How to maintain the health of your Airflow environments and enhance visibility into complex workflows.
Whether engineering resilient pipelines or managing scalable Airflow infrastructure, this session will equip you with actionable insights for successful Airflow operations.
