Happy New Year! This community accomplished so much in the month of December, and I’m excited to be sharing it all with you now! From a new release to record breaking survey results, we closed out 2023 with the same vivacity we entered it with. Highlights from the month of December include:

• The release of Airflow 2.8

• The launch of the 2023 Survey Results

• Announcing Airflow Summit in San Francisco Sept. 10-12th, 2024!

As always, please reach out with items for the next issue.

Thanks for reading!

Airflow 2.8

Airflow 2.8 was released on December 18th and includes a host of significant enhancements and new features that will greatly benefit our community.

• PyPI | Release Notes | Docs | Sources | Docker | Constraints

Providers

New versions of 59 Airflow Providers packages were released.

For all source releases, PyPI packages and docs, see: 

• Sources | PyPI | Docs

Ecosystem

Initial releases: 

• Astronomer Cosmos 1.3.0

• OpenLineage 1.7.0

• 1/11 Webinar: Airflow Town Hall

• 1/11 Webinar: What’s new in Airflow 2.8

• 1/18 Webinar: How to manage connections in Airflow

• 1/31/24 Webinar: Run MLOps production pipelines with Astronomer and Snowpark

• 2/3-4 Conference in Brussels: Data workflows: translating dbt to Apache Airflow® & `New` Workflow Orchestrator in town: "Apache Airflow® 2.x"

• 2/9 Meetup in São Paulo: First São Paulo Airflow Meetup at iFood!

• 2/13 Meetup in Bangalore: Bangalore Airflow Meetup @ Cloudera

• 12/7 Webinar: What’s New in the Astro Platform Release, Q4 2023

• 12/7 Meetup in Prague: First Airflow Prague Meetup with Kiwi.com & Astronomer

• 12/12 Meetup in Hyderabad: Hyderabad Airflow Meetup @ Microsoft HQ

• 12/14 Webinar: Orchestrate next-generation AI tools: Explore six new Airflow providers

• Apache Airflow® 2.8.0 is here by Ephraim Anierobi on the Airflow Blog

• Configuring the KubernetesExecutor to Hum at Etsy by Victor Chiapaikeo on Airflow Medium

• Airflow 2.8 Release: Meet the Contributors on Youtube

• Unraveling the Code: Navigating a CI/Release Security Vulnerability in Apache Airflow® by Jarek Potiuk on Airflow Medium

• Introducing Airflow 2.8 by Kenten Danas on the Astronomer Blog

   

• Apache Airflow® at Adyen: Our journey and challenges to achieve reliability at scale by Jorrick Sleijster and Natasha Shroff on Data Engineering Weekly

• How to share code across Airflow projects on Astronomer Learn

• Airflow in action: providing environmental forecast services by Eloi Codina-Torras on Airflow Medium

• Use Airflow object storage to interact with cloud storage in an ML pipeline on Astronomer Learn

• Leveraging Customization in Airflow with Databricks by Hanish Bansal on Airflow Medium

• LIVE with Astronomer: How Chartboost Uses Airflow Plugins with George Yates on Youtube

• Deploy Rollbacks: Upgrade Airflow and Deploy DAGs with Confidence by David Koenitzer on the Astronomer Blog

• Use a listener to send a Slack notification when a dataset is updated on Astronomer Learn

• LIVE with Astronomer: DAG Serialization Deep Dive with Utkarsh! by George Yates on Youtube

   

• Introducing Apache Airflow® on Astro – an Azure Native ISV Service by Conor Beverland on the Astronomer Blog

• Accelerating ML Application Development: Production-Ready Airflow Integrations with Critical AI Tools by Steven Hillion on the Astronomer Blog

• Enhanced Authentication Security to your Data Services on Azure with Astro by Piotr Chomiak on the Astronomer Blog

• Exploring Airflow’s XComs: Unveiling the Power of Shared State by Amitabh Roy Choudhary on Airflow Medium

• Introducing the Astro Platform Release, Q4 2023 by Juliana O’Donohue on the Astronomer Blog

Votes: 

• Airflow Providers prepared on December 08, 2023

• [RESULT][VOTE] Airflow Providers - release of December 08, 2023

• Remove 'apache.scoop' provider

• Remove 'plexus' provider

• Release Airflow 2.8.0 from 2.8.0rc1

• Release Airflow 2.8.0 from 2.8.0rc2

• Release Airflow 2.8.0 from 2.8.0rc3

• Airflow Providers prepared on December 12, 2023

• [RESULT][VOTE] Airflow Providers - release of 12 December 2023

• Release Airflow 2.8.0 from 2.8.0rc4

• [RESULT][VOTE] Release Airflow 2.8.0 from 2.8.0rc4

• Switch 2.8.0 images to use MariaDB client

• Making Listeners API non-experimental

• Release Airflow Python Client 2.8.0 from 2.8.0rc1

• [RESULT][VOTE] Release Airflow Python Client 2.8.0 from 2.8.0rc1

• Airflow Providers prepared on December 23, 2023

• [RESULT][VOTE] Airflow Providers - release of 23rd of December 2023

• Airflow Providers prepared on December 28, 2023

• [RESULT][VOTE] Airflow Providers - release of 28th December 2023

• Airflow Providers prepared on December 31, 2023

Discussed:

• Serialization in Apache Airflow®: Clarifications and Future Directions

• Helm Chart Version 1.9.0 Unavailable

• Common.sql interface / API

• Allowlist in serializatin [was: Serialization in Apache Airflow®]

• Capturing Architectural decisions (ADRS?)

• Using serde for PyODBC case/other cases [was Serialization in Apache Airflow®]

• Add BigQuery partition sensor for sequential (or just list of) partition

• MappedArgument Object is not Subscriptable

• Do the resources on the Community landing page need reordering/reprioritizing?

• New Airflow Community Provider: Teradata

•  "Require conversation resolution" in our PRs before merge?

• Looking for co-mentor in Major League Hacking Spring 2024 Fellows

• Interested in joining/leaving the triage team?

• Enabling `pre-commit.ci` application for Airflow

• Airflow vs Temporal

Announced:

• New committer: Utkarsh Sharma

• Apache Airflow® Providers prepared on December 08, 2023 are released

• Apache Airflow® Providers prepared on 12 December 2023 are released

• Apache Airflow® 2.8.0 Released

• Airflow Virtual Town Hall- Jan. 11th

• Save the date for Airflow Summit 2024!

• Airflow Survey is Closed!

• Meet our keynote speakers and register to Community Over Code EU!

• Apache Airflow® Python Client 2.8.0 Released

• Apache Airflow® Providers prepared on 23rd December 2023 are released

• Starting experimenting with "Require conversation resolution" setting

• Apache Airflow® Providers prepared on 28th December 2023 are released

• Security team rotation introduction to our process

• NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

• Making Listeners API non-experimental

• AIP-44: Airflow Internal API

• AIP-49: Open Telemetry Support

• AIP-51 Removing Executor Coupling from Core Airflow

• AIP-56: Extensible user management

• AIP-57: Refactor SLA feature

• AIP-58: Airflow ObjectStore (AS)

• AIP-59: Performance test framework

• AIP-60: Standard URI representation

• See the Wiki for more of the project’s approved AIPs.

Did you know that dolphins give names to each other? Research has revealed that the marine mammals use a unique whistle to identify each other. A team from the University of St Andrews in Scotland found that when the animals hear their own call played back to them, they respond.

A humble reminder that humans are just another mammal ;).

Source: BBC News

Please subscribe to this newsletter to ensure you always receive the latest issue:  http://eepurl.com/hXUA3r.

Know of an upcoming event or publication the community should know about? Let us know at viraj@astronomer.io or briana.okyere@astronomer.io.