This post has been on my To Do List for over three years. No, I am not a world-class procrastinator. I’m just checking off a task I assigned to myself a while back.
Over the past few years, we’ve maintained a relatively low profile at Astronomer, preferring to keep our heads down and focus on our work. We used this time to work alongside the community that created Airflow, listen to the companies using it everyday, and invest our resources to mold its future. Today’s post is simply a brief stop on a long journey — for one thing, it’s a chance to say thanks to the many who have supported us the past few years.
But it’s also a signal that we are embracing our responsibilities as a thought leader, and turning up the volume just a bit, so you can hear our thoughts on the future of modern data orchestration. We are a much different company from when I made that initial To Do List, and this post will be the first evidence of that. Starting today, you are going to hear a lot more from Astronomer.
For the first two years of our journey, we focused exclusively on a very tight agenda: make Airflow better, help companies around the globe get more out of it, and “earn the right” to have a voice in its future. These goals defined our culture and formed the foundation of the company we have become.
When we raised our Seed Round in early 2019, Airflow was already the de facto tool of choice for data engineers worldwide. Its immense user base and broad utility made it one of the icons of the open-source data community. But its success also brought some controversy, placing it at the center of numerous public debates on methodology. From our perspective, Airflow was on par with other super successful open-source projects such as Spark and Kafka, but it lacked a steward — an entity with the capital, patience, and passion to fully activate its unlimited potential. And there is the thesis for Astronomer.
The Airflow community has been our inspiration and our partner from the beginning. “Earn the right” has always been about our respect for their work and the values they represent. Our first big decision early on was to put our commercial interests on the back burner, and any investor wanting to be associated with Astronomer had to understand the strategic importance of putting Airflow first.
That narrowed the field considerably, but it only took one email to get our Seed Round done. I reached out to Tim Guleri, Managing Director of Sierra Ventures, who fit the profile perfectly. I knew this because we’d been good friends for over a decade, and I’d worked alongside him in the early days of building companies, when things change fast and don’t always go according to plan. Twenty minutes and a glass of wine on the Sierra balcony later, we had a handshake agreement. Bain Capital soon joined in, as well as a number of other very early investors in Astronomer: Tim Schigel of Refinery Ventures, Mike Venerable of CincyTech, Tim Streit of Grand Ventures, and Harsh Patel of WireFrame, to name a few. Special shout out to these investors, all with ties to Cincinnati, who played pivotal roles in the Astronomer story. They all saw the potential in Airflow and Astronomer long before my friends on the West Coast.
Raising the A round was more stressful. Just as we were planning to kick it off, COVID hit. The world was in an uncertain state, but good fortune came to us in an email from Ethan Batraski, a General Partner at Venrock. To say that Ethan was a believer in Airflow is an understatement. He knew more about Airflow than 90% of our company — he truly understood its power and scope. It was an easy decision to have Ethan lead our Series A.
Those first two rounds provided the fuel we needed to deliver the Airflow 2.0 release in December of 2020. And with 2.0, we saw that the world was paying attention! Even though Airflow was already well ahead of other competing projects, the release of 2.0 immediately drove monthly downloads from an average of 600,000 to over 3 million, and the features contained in 2.0 exploded inbound interest in Astronomer from all over the world.
Our investment in the open-source project those first two years gave us privileged access to thousands of Airflow implementations, and we were learning the ways in which companies wanted to work with and consume Airflow. With that, we began moving into the Product Market Fit phase of our journey, and once again fate was smiling on us.
Scott Yara has been my friend for more than 16 years. Anyone who has worked with Scott knows he is a brilliant product strategist, an inspirational leader, and a ton of fun. It was a big day for us when he agreed to personally invest in Astronomer and join our Board of Directors in the early part of 2020. Having Scott as an advisor was a gift, but 10 months later something happened that profoundly changed our trajectory: Scott joined Sutter Hill Ventures (SHV) as a Managing Director, and Astronomer became his first official investment. Series B was done. Scott jumped in full time, transformed our product vision, and brought Sutter Hill’s legendary expertise in developing young companies to Astronomer. Game changer!
For the past 12 months, we’ve maintained our commitment to Airflow. Monthly downloads are now reaching 12 million, the pace of new releases has tripled, and Astronomer is delivering the vast majority of the code to the project. Our team has grown from 60 to over 260, and we are attracting amazing talent from across the data ecosystem. Our vision for the future has Airflow at the center, but we are expanding our product scope to include data lineage as a core component of the modern data orchestrator. Businesses are becoming increasingly dependent on data pipelines to run their day to day operations, and a data outage today is just as consequential to an automotive manufacturer as an assembly line going down. Data Pipelines are not only the connective tissue of almost every business process, they are mission critical. Solving the tough and varied problems faced by data teams, and ensuring their systems are completely reliable, has become our obsession.
That brings us to the series of announcements we made today. When we raised our Series C, we were looking for new perspectives and strong conviction. We found that in George Mathew from Insight Partners, who led the round, along with Rob Ward of Meritech Capital and Paul Drews from Salesforce Ventures. Their support, expertise, and guidance are helping to drive the next leg of our journey, in which data lineage will play a prominent role.
Data lineage is one of the essential building blocks that enables businesses to trust their data, and our decision to acquire Datakin was informed by our belief that untrusted data inherently lacks value. Datakin’s capabilities will become an integral part of the Astronomer platform, and the combined power of orchestration, lineage, and observability will completely transform the way teams work with data.
Building a great company is equal parts talent, capital, culture, hard work, and luck. We have been extraordinarily fortunate in our success thus far, and I am extremely grateful to all who have contributed, from our investors to our partners to our incredible staff. I would be remiss if I did not mention our founding team: Ry Walker, Greg Neiheisel, Brad Kirn, Viraj Parekh, Paola Peraza Calderon, Pete DeJoy, Ash Berlin-Taylor, and Kaxil Naik. Their grit, integrity, diligence, empathy, and dedication set the stage for where we are today and where we’re headed. As for the future, you will indeed be hearing more from us very soon. The great news is we’re just getting started.
Joe Otto is the CEO of Astronomer.
