With Airflow Summit in the rearview mirror, and Airflow 3 on the horizon, this has been a massive month for the community. Let’s reflect back on everything we've accomplished last month, including:

• The release of Airflow 2.10.1 & 2.10.2

• Hosting the single largest gathering of Airflow enthusiasts at Airflow Summit!

• And the extensive progress made on the upcoming release of Airflow 3. 

As always, please reach out with items for the next issue.

Thanks for reading!

The next Airflow Monthly Virtual Town Hall will be held on Wednesday, November 6th and will feature updates on the project, top PRs from the month, exciting new use case discussions, and more! Please register here if you haven’t already.

Airflow 2.10.1 was released on September 6th, and includes several improvements and relevant bug fixes.

• PyPI | Release Notes | Docs | Sources | Docker | Constraints

Airflow 2.10.2 was released on September 20th, and includes several improvements and relevant bug fixes.

• PyPI | Release Notes | Docs | Sources | Docker | Constraints

Providers

New versions of 49 Airflow Providers packages were released.

For all source releases, PyPI packages and docs, see: 

• Sources | PyPI | Docs

Ecosystem

Recent releases: 

• OpenLineage 1.22.0

• 10/3 Meetup in London: Astronomer’s Airflow Forum

• 10/3 Webinar: How to orchestrate your dbt workflows with Airflow

• 10/24 Meetup in NYC: Astronomer's Forum for Apache Airflow^®

• 10/30 Webinar: Spooktacular Spirits: A Halloween Cocktail Affair with Astronomer

• 11/4 Webinar: Airflow Monthly Town Hall

• 11/7 Workshop: DAG Writing Best Practices

• 11/18-22 Conference in Chicago: Microsoft Ignite

• 9/10-12 Conference in San Francisco: Airflow Summit 2024

• 9/11 Event in San Francisco: Airflow Summit After Party

• 9/19 Webinar: How to build reliable data products with Astro

• 9/26 Webinar: Implementing reliable ETL & ELT pipelines with Airflow and Snowflake

• 11/19-21 Virtual Conference: The Open Source Analytics Conference

Read

• Best Practices and Solutions for Multi-Tenant Airflow by Viraj Parekh on the Astronomer Blog

• Amazon MWAA now supports Apache Airflow version 2.10 on the AWS Blog

• Example reference architectures and implementations with Apache Airflow on Astronomer Learn

• Introducing Cosmos 1.6: The Best Way to Run dbt-core in Airflow by Pankaj Singh on the Astronomer Blog

• Watsonx.data introduces support for a suite of modern dataops tools on the IBM Blog

• Astro and Terraform: Empowering Infrastructure as Code for Modern Data Orchestration by Isaac Chung on the Astronomer Blog

• Orchestrate Ray jobs with Apache Airflow on Astronomer Docs

• Navigating the 2024 Gartner^® Market Guide for DataOps Tools: What You Need to Know on the Astronomer Blog

• ETL for Beginners: Data Ingestion at Scale with S3 and Snowflake by Tamara Fingerlin on the Astronomer Blog

• From Task Failures to Operational Excellence at GumGum with Brendan Frick on The Data Flowcast

• AI-Powered Vehicle Automation at Ford Motor Company with Serjesh Sharma on The Data Flowcast

   

Votes: 

• Release Airflow 2.10.1 from 2.10.1rc1

• "Standard" provider without deprecations as a 2 -> 3 bridge

• [RESULT][VOTE] Release Airflow 2.10.1 from 2.10.1rc1

• "Edge" as the name for AIP-69

• Release Airflow 2.10.2 from 2.10.2rc1

• [RESULT][VOTE] Release Airflow 2.10.2 from 2.10.2rc1

• Airflow Providers prepared on September 21, 2024

• [RESULT][VOTE] Airflow Providers - release of September 21, 2024

• Airflow 2.11 as bridge release

• [RESULT][VOTE] Airflow 2.11 as bridge release

• Airflow Providers prepared on September 27, 2024

• September 2024 PR of the Month

• [RESULT][VOTE] September 2024 PR of the Month

Discussed:

• What to expect from FAB provider after AIP-79

• UI and API Contributions for Airflow 3

• Enable "Auto-Merge" Feature for Github PRs

• Proposal to Add Couchbase Provider to Apache Airflow

• AIP-86 Airflow Alerts (Formerly SLA)

Announced:

• Apache Airflow 2.10.1 Released

• Apache Airflow 2.10.2 Released

• Apache Airflow Providers prepared on September 21, 2024 are released

• Starting a CI/CD knowledge transfer process

• Add streaming support to PartialOperator

• AIP-44: Airflow Internal API

• AIP-49: Open Telemetry Support

• AIP-51 Removing Executor Coupling from Core Airflow

• AIP-57: Refactor SLA feature

• AIP-58: Airflow ObjectStore (AS)

• AIP-59: Performance test framework

• AIP-61 Hybrid Execution

• AIP-62 Getting Lineage from Hook Instrumentation

• AIP-63: DAG Versioning

• AIP-64: Keep TaskInstance try history

• AIP-65: Improve DAG history in UI

• AIP-66: Execution of specific DAG versions

• AIP-67 Multi-tenant deployment of Airflow components

• AIP-68 Extended Plugin Interface for Custom Grid View Panels

• AIP-69 Remote Executor

• AIP-71 Generalizing DAG Loader and Processor for Ephemeral Storage

• AIP-72 Task Execution Interface aka Task SDK

New!

• AIP-73 Expanded Data Awareness

• AIP-78 Scheduler-managed backfill

• AIP-80 Explicit Template Fields in Operator Arguments

• AIP-81 Enhanced Security in CLI via Integration of API

• AIP-82 External event driven scheduling in Airflow

• AIP-83 Remove Execution Date Unique Constraint from DAG Run

• AIP-85 Extendable DAG parsing controls

• AIP-86 Airflow Alerts (Formerly SLA)

• AIP 87: Run Sensors in Triggerer with AsyncBaseSensor

Did you know that it’s Fat Bear Week? 

That’s right, it’s time to vote for the fattest bear of the year! Some of the largest brown bears on Earth make their home at Brooks River in Katmai National Park, Alaska. Brown bears get fat to survive and Fat Bear Week is an annual tournament celebrating their success in preparation for winter hibernation. Matchups are open for voting October 2 - 8 between 12 - 9  p.m. Eastern (9 a.m. - 6 p.m. Pacific).

Source: explore.org

Please subscribe to this newsletter to ensure you always receive the latest issue:  http://eepurl.com/hXUA3r.

Know of an upcoming event or publication the community should know about? Let us know at briana.okyere@astronomer.io.