Strong data orchestration is as much about culture and visibility as it is about technology. As data platforms scale, teams need systems that reduce cognitive load while increasing reliability and observability.

In this episode, Carlos Daniel Puerto Niño, Senior Analytics Engineer and Data Analyst at Addi, joins us to share how Addi uses Airflow to support batch orchestration, manage organizational complexity and improve monitoring across its data platform.

Key Takeaways:

(00:00) Introduction.

(01:25) Changes in company strategy increase data platform complexity over time.

(04:00) Centralized data teams help manage organizational and technical change.

(06:08) Scalable architectures support growing data volumes and use cases.

(09:10) Adopting orchestration tools introduces operational and maintenance challenges.

(14:43) Abstraction layers lower technical barriers for onboarding new team members.

(15:36) Modularity and visibility improve the reliability of data pipelines.

(18:14) Integrated monitoring supports faster incident response and resolution.

(22:19) Limited access to orchestration metadata constrains proactive analysis.

Resources Mentioned:

Carlos Daniel Puerto Niño: https://www.linkedin.com/in/carlospuertoniño/

Addi: https://www.addi.com

Addi on LinkedIn: https://www.linkedin.com/company/addicol/

Apache Airflow: https://airflow.apache.org/

Astronomer: https://www.astronomer.io/

Databricks: https://www.databricks.com/

dbt: https://www.getdbt.com/

Grafana: https://grafana.com/

Slack: https://slack.com/

Thanks for listening to “The Data Flowcast: Mastering Apache Airflow^® for Data Engineering and AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.

#AI #Automation #Airflow