Join us for upcoming online events!
Hosted By
George YatesLead Developer Advocate
Juliana O'DonohueDirector, Product
Kenten DanasManager, Developer RelationsThis webinar will provide a comprehensive overview of the new features in the latest Astro release, complete with live demos. Whether you're already using Astro or considering it for your data orchestration needs, don't miss this opportunity to learn from the experts and have your questions answered.
A paradigm shift occurred leading to the formation of the large language model operations (LLMOps) modern data stack with Apache Airflow right at the center. Join us in this webinar to explore these providers and learn how you can leverage them to take your LLMOps to the next level.
The Airflow project is rapidly evolving, with frequent releases bringing advancements in DAG authoring, observability, and project stability. We’re super excited for the release of Airflow 2.3, which comes with big changes in the flexibility of DAG creation, improvements to the Airflow UI, and much more.
Airflow is sometimes thought of as primarily a data engineering tool, but its use cases are really much broader. A data analyst’s workflow typically involves ingesting and transforming data to extract insights, then presenting the insights in a manner that allows business stakeholders to easily interpret trends and take appropriate action. Airflow’s ease of use and extensive provider ecosystem make it an ideal tool for orchestrating such analytics workflows.
Data lineage is the complex set of relationships between your jobs and datasets. Using OpenLineage with Apache Airflow, you can observe and analyze these relationships, allowing you to find and fix issues more quickly. This webinar will provide a deeper dive on OpenLineage, extending beyond the basics into key implementation details and best practices.
Apache Airflow is flexible and powerful. It has a rich ecosystem and an incredibly active community. But are you sure you haven’t missed anything? A new feature or concept that could put your DAGs at another level? It can be challenging to keep up with the latest Airflow features, and sometimes we miss the most useful ones. For this webinar, I'd like to introduce you to a couple of lesser-known features of Apache Airflow that can dramatically improve your data pipelines.
Airflow is purpose-built for high-scale workloads and high availability on a distributed platform. Since the advent of Airflow 2.0, there are even more tools and features to ensure that Airflow can be scaled to accommodate high-throughput, data-intensive workloads. In this webinar, Alex Kennedy will discuss the process of scaling out Airflow utilizing the Celery and Kubernetes Executor, including the parameters that need to be tuned when adding nodes to Airflow and the thought process behind deciding when it’s a good idea to scale Airflow, horizontally and vertically. Consistent and aggregated logging is key when scaling Airflow, and we will also briefly discuss best practices for logging on a distributed Airflow platform, as well as the pitfalls that many Airflow users experience when designing and building their distributed Airflow platform.
At this webinar, Benji Lampel (Enterprise Platform Architect @ Astronomer) and Tal Gluck (Software Engineer @ Superconductive) will present several Airflow DAGs using Great Expectations that cover more advanced DAG patterns and data quality checking cases.
Did you know that Airflow has a fully stable REST API? In this webinar, we’ll cover how to use the API, and why it’s a great tool in your Airflow toolbox for managing and monitoring your data pipelines.
With Airflow 2.0, we introduced the concept of providers. We’re taking that to the next level with Astro, a new DAG writing experience, brought to you by Astronomer.
If one out of your hundreds of DAGs fails, how do you know which downstream datasets have become out-of-date? The answer is data lineage. Data lineage is the complex set of relationships between your jobs and datasets. In this webinar, you'll learn how to use OpenLineage to collect lineage metadata from Airflow and assemble a lineage graph - a picture of your pipeline worth way more than a thousand words.
Try Astro free for 14 days and power your next big data project.