Tracking Innovation: How Astronomer Streamlined Release Notes with Towncrier

Writing release notes is a deceptively mundane task. At its core, you just have to write a sentence about each of the changes going into a given release. But as with anything in tech, the complexity of a task grows exponentially when you factor in team structure, project structure, and release cycles. Reducing that complexity to a single list of changes becomes a complexity in and of itself.

Astro, our hosted offering for Apache Airflow, is a constantly evolving product with new features releasing every week. In addition to the volume of changes to document, there are a few other circumstances that complicate writing release notes for Astro:

• We have a large team distributed across the globe, and each team member wears multiple hats. 
• Astro has weekly releases. Any change, no matter when it was started, can be included in the release as long as it’s ready to launch by the release deadline.
• Engineers write their documentation for other engineers; there is no official process for capturing user-facing information in the engineering cycle.

In this blog post, we’ll explain how we created an efficient release note system on Astro through a few small process changes. We’ll also explain how we introduced these changes to the developer experience with as little friction as possible.

Capturing changes from a rapidly innovating, distributed team

The Astro development team is remote and distributed across the globe. This is not a problem for release notes in itself; with enough time, you could contact all team members and get a summary of the changes they were releasing. However, Astro’s weekly release cycle has large batches of changes shipped to production every week, which complicates the process of collaborating with every individual member on the team. On this cadence, writing time is at a premium.

Previously, a technical writer had to trawl through each pull request in a release and determine whether a change needed a release note. Not only did this take ages, but it resulted in a lot of missed or incorrectly documented changes that were only caught after publication.

We first tried to create a release note workflow through the Product team. Each product manager was responsible for providing a recap of the changes they knew were releasing each week, and writers were responsible for following up on those leads with engineering. The Product team was great at providing context and details for major features coming to Astro, but they didn’t have as much visibility into small code changes that could have major impact for customers, such as reductions to cloud resource consumption. The Astro team wanted release notes for these changes to champion the many small ways in which we improved Astro from week to week.

Because using the Product relationship to generate release notes still required speaking to engineering and consulting GitHub issues, we thought it was best to connect directly with engineering and build our process as close to the code as possible. But at the same time, we didn’t want to depend on engineers to write their own release notes or turn their pull request (PR) descriptions into a lengthy compliance form. So we found a solution to meet the engineers halfway.

Creating an internal reference changelog with Towncrier

There was a palpable Aha! moment when Kaxil Naik mentioned Towncrier as a potential solution to our release note woes. For those unfamiliar with the tool, Towncrier automatically generates changelogs based on files called “news fragments” that engineers add to PRs to explain their code changes. Towncrier is especially useful for automating changelogs in open source projects where no single person is responsible for summarizing a release. 

We subtly changed the purpose of the Towncrier changelog from a user-facing document to an internal digest for technical writers. Instead of writing directly to users, engineers write their news fragments as informal summaries about how their change impacts users. When the Astro team promotes a week’s release to staging, a GitHub action compiles all news fragments into a single document that a technical writer can reference. The technical writing team uses these statements to write release notes and, if needed, to follow up and get more information from specific engineers. 

With this implementation, we solved a few of our core problems: 

• Each week’s changes were filtered to show only the changes that users cared about
• The experts on each change were delivering us information exactly when and where we needed it.
• The time to write release notes shrunk from multiple hours to as little as 30 minutes. 

But this also introduced new problems: How do we enforce the process without bogging down engineering workflows, and how do we check for changes that don’t have a news fragment but should?

Automating and iterating the process

We first introduced the process to engineers as an opt-in process. Engineers submitted their news fragments based on the honor system, and there were no checks to determine whether a given change required a news fragment or not. That’s because as-is, the process introduced a lot of friction for engineers.

While the out-of-the-box Towncrier workflow is great, it’s a big ask to have engineers manually create a news fragment for each PR, especially for engineers who are already diligently noting user impact in their PR descriptions. The first way we improved the process was to create a section in the Astro GitHub PR template that gave engineers a space to note the user impact of their change. After the PR was submitted, a GitHub action extracted the text from this section and applied it as a news fragment file to the PR. This greatly reduced the friction of submitting change descriptions and allowed engineers to develop without any direct interaction with Towncrier.

After we received enough adoption and feedback, we switched the process from voluntary opt-in, to opt-out. We now required engineers to complete one of the following actions in each PR:

• Write about the user impact of their change in the PR description.
• Add a doc-skip label to the PR to indicate that the change did not require a release note.

If a developer didn’t interact with this process at all, a CI check failed and they couldn’t merge their PR. This change encouraged engineers to think about their changes from a user’s perspective, and it gave us more information to use for an RCA if we ever missed documenting a change. This process was hugely beneficial to technical writers and only required a few minor actions from engineers without disrupting their core work.

Conclusion

This project was a lesson in creating sustainable development workflows. What I find great about the Towncrier solution is that you can fully customize it to find the balance of regulation that works best for your team.

Because Astro is a distributed and rapidly releasing team, we had to add more checks and balances to the default Towncrier workflow to make our release notes accurate. But a smaller team could make use of Towncrier as-is with tremendous results. Beyond Towncrier, this project proved that you can solve some major operational problems on a commercial project with just a little bit of open source tooling and creativity. Keeping docs and code close together has greatly strengthened the relationship between engineering and technical writers.