Develop ML Pipelines with the Astro Cloud IDE
Hosted By
In this webinar, we’ll show how the Astro Cloud IDE is the easiest way to develop and test your ML pipelines and schedule them with Airflow.
Astronomer Webinars
Join us for upcoming online events!
Past Webinars
Airflow 101: Essential Tips For Beginners
What is Airflow? Apache Airflow is a platform used to programmatically author, schedule, and monitor data pipelines.
Data-Aware Scheduling with the Astro Python SDK
Live with Astronomer dives into implementing data-aware scheduling with the Astro Python SDK. The new Airflow Datasets feature allows you to schedule DAGs based on updates to your data and easily view cross-DAG relationships. This feature is part of the Astro Python SDK, so it requires almost no effort from the DAG author to implement. We'll show you everything you need to do (and don't need to do) to take advantage of Datasets.
Running Airflow Tasks in Isolated Environments
Running tasks in a separate environment can help you avoid common data pipeline issues, like dependency conflicts or out-of-memory errors, and it can save resources. Airflow DAG authors have multiple options for running tasks in isolated environments. In this webinar, we'll cover everything you need to know.
How to Migrate from Oozie to Airflow: A Guided Walkthrough
Migrating between orchestrators can be a difficult process fraught with technical and organizational hurdles. However, the end result of applying Airflow’s orchestration capabilities is worth the effort, and working with the right partner can make this journey much easier. In this webinar, we’ll cover everything you need to know about migrating from Oozie to Airflow.
The New DAG Schedule Parameter
Live with Astronomer will discuss the new consolidated `schedule` parameter introduced in Airflow 2.4. We’ll provide a quick refresher of scheduling concepts and discuss how scheduling DAGs is easier and more powerful in newer versions of Airflow.
Batch Inference with Airflow and SageMaker
We’ll walk through using the new SageMaker Async Operators, as well as the new SageMaker OpenLineage integration, for end-to-end ML OPs for batch inference use cases.
Dynamic Task Mapping on Multiple Parameters
On October 25, Live with Astronomer will dive into updates to the dynamic task mapping feature released in Airflow 2.4. We’ll show a couple of new methods for mapping over multiple parameters, and discuss how to choose the best mapping method for your use case.
Dynamic Tasks in Airflow
With the releases of Airflow 2.3 and 2.4, users can write DAGs that dynamically generate parallel tasks at runtime. In this webinar, we’ll cover everything you need to know to implement dynamic tasks in your DAGs.
Data Driven Scheduling
In this session, Live with Astronomer explores the new datasets feature introduced in Airflow 2.4. We’ll show how DAGs that access the same data now have explicit, visible relationships, and how DAGs can be scheduled based on updates to these datasets.
