June brought with it a flurry of activity within the Airflow Community, from developments in the Airflow Summit agenda to the announcement of a new committer. This month, we’re happy to announce:

• The release of the speakers and sessions for Airflow Summit 2024

• The impending launch of our Los Angeles Chapter

• And the announcements of a new committer- Ryan Hatter!

As always, please reach out with items for the next issue.

Thanks for reading!

NEW KEYNOTE ANNOUNCED!

In this session, dive into the winning playbook of the 2023 World Series Champions Texas Rangers, and discover how they leverage Apache Airflow® to streamline their data pipelines. 

Check out the keynote and secure your tickets to celebrate 10 years of Airflow this fall!

The next Airflow Monthly Virtual Town Hall will be held on Wednesday, July 10th. We’ll receive demos on impactful PRs made in the past month and get a deep dive on the OpenLineage project. Please register here if you haven’t already.

Airflow 2.9.2
Airflow 2.9.2 was released on June 10th, and includes several improvements and relevant bug fixes.

• PyPI | Release Notes | Docs | Sources | Docker | Constraints

Providers

New versions of 43 Airflow Providers packages were released.

For all source releases, PyPI packages and docs, see: 

• Sources | PyPI | Docs

Ecosystem

Recent releases: 

• OpenLineage 1.17.1

• Cosmos 1.5.0

   

• 7/9 Webinar: Best Practices for Effectively Running dbt in Airflow

• 7/10 Webinar: Airflow Monthly Town Hall

• 7/17 Meetup in NYC: Airflow Rooftop Happy Hour

• 7/24 Meetup in London: Data Engineer Thing SUMMER BASH!

• 8/7 Meetup in LA: Los Angeles Apache Airflow® Chapter Launch!

• 9/10-12 Conference in San Francisco: Airflow Summit 2024

• 11/19-21 Virtual Conference: The Open Source Analytics Conference

• 6/5 Webinar: Airflow Monthly Town Hall

• 6/3-5 Conference in Bratislava: Community Over Code

• 6/6 Conference in San Francisco: Snowflake Dev Day

• 6/13 Webinar: How Grindr uses Airflow to monitor and optimize Snowflake usage

• 6/10-13 Conference in San Francisco: Data and AI Summit

• 6/27 Webinar: How to write data pipelines with Airflow

Read

• Exploring Airflow 2.9 Features with Astronomer's 29 Days of Airflow 2.9 Series by Radhika Gulati & Constance Martineau on the Astronomer Blog

• Start Airflow task execution directly from the triggerer by Wei Lee on Airflow Medium

• Observations from migrating away from Control-M to Airflow by Arjun Anandkumar on Airflow Medium

• Introducing the First Generative AI Cookbook for Data Orchestration by Kenten Danas and Tamara Fingerlin on the Astronomer Blog

• How we orchestrate 2000+ DBT models in Apache Airflow® by Alexandre Magno Lima Martins on Airflow Medium

• How Vibrant Planet's Self-Healing Pipelines Revolutionize Data Processing with Cyrus Dukart & David Sacerdote on The Data Flowcast

• Learn from the experts: how to write data pipelines with Airflow on Astronomer Webinars

• How Grindr uses Airflow to monitor and optimize Snowflake usage on Astronomer Webinars

Votes: 

• [RESULT][VOTE] Airflow Providers - release of May 30, 2024

• [RESULT][VOTE] AIP-59 Performance Testing Framework

• Airflow Providers prepared on June 07, 2024

• [RESULT][VOTE] Airflow Providers - release of June 07, 2024

• Release Airflow 2.9.2 from 2.9.2rc1

• [RESULT][VOTE] Release Airflow 2.9.2 from 2.9.2rc1

• Add YDB provider

• [RESULT][VOTE] Airflow Providers - release of May 30, 2024

• [RESULT][VOTE] AIP-59 Performance Testing Framework

• Release Apache Airflow® Helm Chart 1.14.0 based on 1.14.0rc1

• [RESULT][VOTE] Release Apache Airflow® Helm Chart 1.14.0 based on 1.14.0rc1

• Introduce common.compat provider

• Remove "Experimental' banner for OpenTelemetry Metrics

• Remove "Experimental' banner for OpenTelemetry Traces

• Airflow Providers prepared on June 22, 2024

• [RESULT][VOTE] Airflow Providers - release of June 22, 2024

• Auto-labeling Airflow issues with DoSu AI

• June 2024 PR of the Month

• [RESULT][VOTE] June 2024 PR of the Month

   

Discussed:

• Proposal around the Injection of Task Execution Secrets

• Number of queries to Airflow database in "DAG File Processing Stats"

• Call with Nielsen team demoing their DAG debugging feature

• System Test Dashboards - Phase Two??

• Implement common.dataframe (or similar) provider based on Ibis

• [AIP-38 Modern Web Application]

• Using AI / Dosu to help us with triaging issues

   

Announced:

• Apache Airflow® Providers prepared on May 30, 2024 are released

• Apache Airflow® Newsletter May 2024

• Minutes and Recording from June 2024 Airflow Monthly Town Hall

• Apache Airflow® Providers prepared on June 07, 2024 are released

• Apache Airflow® 2.9.2 Released

• [Meeting Notes] Airflow 3 Dev call - 4 June 2024

• Apache Airflow® Providers prepared on May 30, 2024 are released

• [Meeting Notes] Airflow 3 Dev call - 13 June 2024

• Google Provider Package System Tests Dashboard

• Apache Airflow® Helm Chart version 1.14.0 Released

• [Airflow 3] The Airflow actors

• Airflow 3 Dev call tomorrow

• [Meeting Notes] Airflow 3 Dev call - 20 June 2024

• Airflow Summit 2024: Sessions are ready!

• Airflow 3 Dev call today

• Apache Airflow® Providers prepared on June 22, 2024 are released

• New committer: Ryan Hatter

   

Proposed:

• Automated management of lower-bounds of airflow dependencies

• Proposal for Enhanced Data Awareness in Airflow

• Use Trusted Publishing workflow for Airflow releases to PyPI

   

• AIP-44: Airflow Internal API

• AIP-49: Open Telemetry Support

• AIP-51 Removing Executor Coupling from Core Airflow

• AIP-57: Refactor SLA feature

• AIP-58: Airflow ObjectStore (AS)

• AIP-59: Performance test framework

• AIP-61 Hybrid Execution

• AIP-62 Getting Lineage from Hook Instrumentation

• AIP-63: DAG Versioning

• AIP-64: Keep TaskInstance try history

• AIP-65: Improve DAG history in UI

• AIP-66: Execution of specific DAG versions

• AIP-67 Multi-tenant deployment of Airflow components

• AIP-68 Extended Plugin Interface for Custom Grid View Panels

• AIP-69 Remote Executor

• AIP-71 Generalizing DAG Loader and Processor for Ephemeral Storage

• See the Wiki for more of the project’s approved AIPs.

   

Did you know that bumblebees can fly higher than Mount Everest? A pair of researchers has found that alpine bumblebees are able to fly at altitudes in excess of twenty nine thousand simulated feet—higher than Mount Everest!

A friendly reminder that even the smallest among us can reach the highest heights :).

Source: BBC

Please subscribe to this newsletter to ensure you always receive the latest issue:  http://eepurl.com/hXUA3r.

Know of an upcoming event or publication the community should know about? Let us know at briana.okyere@astronomer.io.