Semantic Versioning Sucks! Long Live Semantic Versioning

avatar-matt_raible.jpg Matt Raible

Hello, fellow developers.

Have you ever been bitten by transitive dependencies changing in minor releases? I have. Semantic versioning is supposed to prevent this, and sure, semantic versioning is a great idea at its core, but when its guidance is not followed it sucks. People release minor versions without backward compatibility all-the-time. TL;DR? Semantic versioning sucks because humans get involved.

In the early days, I don’t recall having much of a problem with dependencies and their versions. Back then, Java developers would use their IDE for everything, or Ant with their dependencies (aka JAR files) checked into source control. In essence, everything was "locked down" and there were no transitive dependencies.

Before I go any further, let me answer what is semantic versioning?

Semantic Versioning…​

Semantic versioning is a naming system for version numbers of software releases. It’s commonly used by open source projects. According to semver.org, given a version number MAJOR.MINOR.PATCH, you should:

  1. MAJOR version when you make incompatible API changes,

  2. MINOR version when you add functionality in a backward-compatible manner, and

  3. PATCH version when you make backward-compatible bug fixes.

Why Does Semantic Versioning Suck?

Semantic versioning sucks because of transitive dependencies. Back in the day, when you had all your dependencies in source control, there were no additional dependencies pulled in. Today, there are tools like Maven/Gradle (for Java) and npm for Node. These build tools read a set of dependencies, download them (including the dependencies they depend on), and make it so your project can read them and compile everything.

Maven and Gradle have been criticized for "downloading the internet". If you think they’re bad, you should see npm. It downloads the internet and invites all its friends to come along!

This is not the fault of the build tool, but often the library authors who depend on all kinds of third-party libraries.

In Java-land, we learned early that version ranges are a bad idea. If you want them, you can use them, but most people don’t. Here’s how they work:

  • [1.0, 2.0]: all versions >= 1.0 and ⇐ 2.0

  • [1.0, 2.0): all versions >= 1.0 and < 2.0

  • [1.0, ) : all versions >= 1.0

Node, on the other hand, encourages ranges. Even if you don’t want to use them, chances are your downstream dependencies are. This doesn’t happen often in Java libraries.

Node has an advanced range syntax with hyphens, x-ranges, tilde ranges, and caret ranges.

  • Hyphen ranges specify an inclusive set: 1.2.3 - 2.3.4 means >=1.2.3 ⇐2.3.4

  • X-Ranges allow X, x, or * to specify the numeric values in the [major, minor, patch]

  • Tilde ranges allow patch-level changes if a minor version is specified, ~1.2.3 means >=1.2.3 <1.3.0

  • Caret ranges allow changes that do not modify the left-most non-zero element. This means you’ll get patch and minor updates for version 1.0.0 and above.

Why does Node encourage ranges so much? The Online Semver checker says:

Strict constraint (or fully qualified constraint) are those constraints matching only one version. In most cases it is a bad idea to use them.

Why? Because with them you are locking your dependency to a specific patch release which means you won’t ever get bug fixes when updating your dependencies.

I like the ideas behind this statement. It would be nice to get bug fixes and security updates from patch releases. However, using ranges for versions means that incompatible versions can sneak in. And builds might not be reproducible if a downstream patch release happens between builds.

Semantic Versioning Sucked in 2019

I’ve personally experienced some times when semantic versioning killed my productivity in 2019.

JHipster is an application generator that uses Java on the backend (with Maven or Gradle) and npm for the frontend. It doesn’t use version ranges in its package.json, but still had issues caused by downstream releases. The project experienced three issues in 2019:

  1. #9438 React Router (broke React)

  2. #9952 Inquirer (broke all prompts, JHipster unusable for a whole day)

  3. #10371 Terser (broke Angular entity screens)

None of these problems were caused by versions specified in package.json. They were all caused by downstream dependencies releasing new versions and breaking things!

I also got blocked (along with many others) by Ionic CLI v5.4.1. The version had issues caused by a patch release of minipass. This took three days to fix and I was blocked from testing changes to the JHipster Ionic Starter during it. No new release of Ionic CLI was required, you just had to uninstall/reinstall after minipass released another patch release.

A wide-sweeping patch update that affected react-scripts, Angular CLI, and Ionic CLI was a minor release of core-js. I didn’t experience this problem personally, but it does seem like it was fixed in the same day.

I also experienced an issue in 2018 where the TypeScript support for Create React App didn’t work for a two hour period. That period happened to be when I was on stage demoing React at Spring One. You can watch my failure here and find the cause in this Twitter thread.

What Can We Do to Make Semantic Versioning Better in 2020?

The Java projects I use don’t seem to have semantic versioning issues. I believe this is because no one uses version ranges in Java. If there’s a new downstream release that fixes a critical bug, the upstream library does a new release. Builds are always reproducible.

I’m not sure if there’s a good solution in the Node community. There are millions of libraries and most development guides encourage ranges and say strict versions are a bad idea. In theory, it sounds good, but it only works if developers strictly adhere to semantic versioning.

JHipster doesn’t use version ranges for Angular, React, and Vue, yet we still experience issues. Yarn’s selective version resolutions does help, and it’s what JHipster often uses to lock downstream dependency versions. For example, adding the following to your package.json makes sure you get a version of handlebars without a security vulnerability.

"resolutions": {
 "handlebars": "4.5.3"
}

However, this only works with Yarn. The good news is you can use npm-force-resolutions if you’re an npm user. You just need to add a preinstall script.

"scripts": {
 "preinstall": "npx npm-force-resolutions"
}

I’ve started using this in the projects I maintain to pin the version numbers of transitive dependencies. Please note that this has caused some issues, so you might be better off just running npm npm-force-resolutions every-so-often to update your package-lock.json file.

When reviewing this article, Minko Gechev noted that "many projects pin their dependencies to a specific version and use tools like renovatebot to update to latest versions. Combined with a CI and a complete test suit, it’s a good solution."

A Better Future: Automated Semantic Versioning

Semantic versioning is a good idea. It would be a great idea if there were tools that automatically assigned the version based on comparing the current version to a previous version. Assigning the version number for a release seems to be mostly a manual task.

If we could automate the version number assignment, I think Node projects would experience a lot fewer issues. Of course, then you’d have to get all developers to adopt it, so that might be tough. Then again, if it was built into npm (like npm audit for security vulnerabilities), then we might be onto something!

Personally, I don’t know of any tools that do this, so I asked a few friends in the developer community.

How Experts Do Semantic Versioning

I contacted the following developers that maintain open source projects I use and love.

I asked them how they manage semantic versioning for their projects and if it’s automated. If their process is manual, I asked them how they validate patch and minor releases to verify they don’t break anything. Specifically, these were my questions:

  • How do you do semantic versioning? Is it manual or automated?

  • If automated, what tools do you use to validate a patch or minor release doesn’t break anything?

  • If manual, how do you guarantee you aren’t breaking backward compatibility?

Phil Webb, Spring Boot

Phil Webb

With Spring Boot we actually decided to intentionally not use semantic versioning. We found it too rigid for our needs because we’d need to bump the major version too frequently. Instead, we try to take a more pragmatic approach where we’ll try to ensure back-compatibility, but will occasionally choose to break things if we feel like that’s ultimately the best option.

Instead of true semantic versioning, we instead use the version number to indicate the amount of pain users might expect with an upgrade. The general rule is:

  • A patch version should be a drop-in replacement (e.g. v2.2.1 → v.2.2.2 should just work). Very very rarely we might choose to break something if there’s no way to deprecate methods

  • A minor version should be relatively easy to upgrade. You will need to make sure you’re not using deprecated methods because we remove them fairly aggressively.

  • A major version might cause some upgrade pain depending on how deeply you integrate with our code. For example, v1.5.x → 2.0.x wasn’t too tricky for most users, but it was hard if you’d written custom actuator endpoints. We take the opportunity on a major version bump to fix deeper problems with APIs where we need to break them because there’s not an obvious way to migrate them.

With that in mind, your questions are still valid if you remove the word "semantic" so here are the answers:

How do you do semantic versioning for Spring Boot? Is it manual or automated?

It’s a manual process. We’ve got quite a bit of experience evolving APIs so we tend to know when a change will cause problems.

If automated, what tools do you use to validate a patch or minor release doesn’t break anything?

We don’t have any automated tools. We do however review each others commits to try and spot issues early. We also have a great user community that lets us know when we break things. :)

If manual, how do you guarantee you aren’t breaking backward compatibility?

We don’t make strong guarantees. We mainly rely on our own experience to ensure we don’t do anything foolish. We only add new features in minor versions so most patch releases are bug fix only and hence API changes are rare.

Juergen Hoeller, Spring Framework

Juergen Hoeller

Our semantic versioning in Spring is not strictly about backward compatibility, it’s rather a form of pragmatic impact guidance along the lines of generation / feature release / maintenance release (e.g. 5.2.3). This is an entirely manual part of our design process where "generation" means a fundamental revision of the codebase (mostly JDK baselining but also e.g. nullable annotations and Kotlin extensions in 5.0) including some pruning and module rearrangements, "feature release" means a rich set of new features (including refactorings and re-implementations of existing features) but all within the existing framework architecture and its structural arrangements, and "maintenance release" means bug fixes and minor enhancements.

We compare API diff reports between releases and run Animal Sniffer for JDK baseline enforcement, but otherwise, there are no tools involved. Backward compatibility (in particular binary compatibility between maintenance releases) is mostly covered through integration tests, no guarantees attached…​ Sometimes we have to fix regressions after the fact, doesn’t happen all that often though, in particular not for maintenance releases where we strongly enforce selective and well-reviewed backporting, to begin with. And in very rare cases, we have to intentionally break strict backward compatibility even within a maintenance line, e.g. in case of tightened rules for vulnerabilities or to fix accidents or recent regressions.

All in all, our versioning is pragmatically semantic with a focus on developer impact. Near-100% backward compatibility is a key goal, mostly covered by backport reviews and integration testing.

Minko Gechev, Angular

Minko Gechev

We have semantic commit messages prefixed with "fix", "refactor", "test", "ci", "feat", etc. If we’re about to release a patch version, we cannot include a feature (feat) PR. Based on these semantic commit messages we can automatically generate the changelog.

Additionally, to make sure we’re not introducing breaking changes we have two (maybe more that I’m not aware of), processes:

  1. We have golden files. These are TypeScript d.ts files which we verify each build against. If we change the public API surface, we’ll generate another set of d.ts files which will not match the current set. We can release backward-incompatible changes in the public API surface only between major releases and the golden files help us verify that.

  2. We run tests for affected Google projects. The d.ts files do not provide 100% guarantee that we haven’t changed anything semantically in Angular (for example, the lifecycle hooks execution order) in a backward-incompatible way. When we introduce a change we run the tests of the affected google projects to make sure we haven’t broken them. This is all automated with our internal CI.

We’ve reached a state in which we can detect (almost?) any breaking change in Angular thanks to the tens of hundreds of projects internally and the hundreds of thousands of tests.

In general, I agree semantic versioning is not ideal. I don’t see a way it could be completely automated. Programming languages are too complicated to verify statically, as part of the build process, which should be the next version a certain project should be released under.

Brian Demers, Okta Java Tools

Brian Demers

The Okta Spring Boot Starter has VERY few public classes to avoid this exact problem. It mostly proves implementations of existing Spring Security interfaces.

The Okta Java SDK is a different story. I heavily rely on the japicmp Maven plugin.

If automated, what tools do you use to validate a patch or minor release doesn’t break anything?

I don’t think I’d be able to follow semver without it. We run the cmp goal during CI and releases which will fail if there is a breaking change (or if a minor version needs to be bumped instead of a patch). Updating the actual version is still a manual process (I use the Maven versions plugin mvn versions:set -DnewVersion={new-version-here}).

Semver and Java don’t 100% line up, the OSGi alliance has a nice guide. Java has a notion of "source" and "binary" compatibility. Japicmp can handle both, but it’s really only binary compatibility that matters.

Adding new "default" methods to a Java interface is technically a breaking change too, japicmp allows for post-processing of the results, so you can allow these changes depending on your use cases.

If manual, how do you guarantee you aren’t breaking backward compatibility?

The Okta Spring Boot Starter public methods (I think there are three total) are easy to manage right now because of the small team size and the public API size. That said it’s very easy to change an API in what you think is a backward compatible way (this recently happened in the Spring Security 5.2 release). Scanning this project to ensure semver is on the TODO list.

Deepu K Sasidharan, JHipster

Deepu K Sasidharan

IMO semantic versioning itself doesn’t suck, I mean the idea of having major.minor.patch releases do work when done correctly.

The problem is actually when using non fixed version ranges. For example, the same problem is present in the Golang community as well even though they don’t necessarily use semver, but the module system supports using ranges or GIT branches for versions, which breaks stuff when you accidentally upgrade transitive dependencies. To an extent, this could also happen in Java IMO. I have experienced it but surely not as much as NodeJS. That is also due to the amount of modularization there.

I was doing the releases for ng-jhipster, react-jhipster, and few other libs, and even wrote the release scripts on the NPM files, I never had issues with semver per se. The issue is obviously when a bad actor doesn’t respect the semver versioning and do breaking changes in a minor version or something like that. If the range support is removed in NPM package resolution, most of the issues will be gone

For your question 2 and 3, I don’t see how it is a server-specific problem, it could be applied to any versioning scheme right.

Julien Dubois, JHipster

Julien Dubois

Concerning the automatic Semver, there is just no way to do this for JHipster…​ so we do it manually.

Basically, the release manager "knows" when there is something breaking, and usually, we never break anything outside of major releases. Of course, we can be wrong sometimes!

The only exception would be for a security patch: then it’s good for us to break the user code, if necessary, as they also needs to fix their own code. But that’s very specific because we are a code generator, and I hope we can remove this as much as possible (I’d like to generate less code, and give more responsibilities to the JHipster libraries, typically because they solve this kind of issue).

Robert Damphousse, Okta JavaScript SDKs

How do you do semantic versioning for the Okta JS SDKs (Auth SDK, Angular, React, Vue, etc.)? Is it manual or automated?

Manual at the moment, as it provides the most about of flexibility. As you know, semver is opinionated about feature vs. patch/bugfix, so we have to take that into account. The interesting thing about semver is that, because of that constraint, it forces you to break up multiple changes, which I don’t think is a bad thing. There seems to be a lot of FUD around releasing too many versions, but I don’t worry about that. We invented numbers to be used.

If automated, what tools do you use to validate a patch or minor release doesn’t break anything?

We have looked at automation, and if we go down this route it would likely rely on well-formatted commit messages, such as conventional commits (which would have to be human vetted during code review). There is some trickiness around how to enforce those messages though, and where we would want to put those assertions (e.g. a GitHub PR hook?)

If manual, how do you guarantee you aren’t breaking backwards compatibility?

That’s done during code review, IMO you’re always going to need to rely on code review to make sure you aren’t breaking. We’ve caught quite a few just through basic code review. Some languages (Java) do have tools for checking method signatures and other things that are statically obvious as breaking a contract/interface, those tools can give you some early warnings. Perhaps there is something for JavaScript but I haven’t looked.

I think the idea that 0.x version ranges can have breaking changes within the 0.x range is silly. Just roll it over to 1.0 if you need to break. What major version is Angular on now anyways?

Maintaining and Releasing Open Source Software is Hard

The general consensus from most of the folks I interviewed is that they set version numbers manually. The Spring framework’s versioning is pragmatically semantic with a focus on developer impact. They use lots of integration tests and reviews to ensure backward compatibility. Spring Boot doesn’t use semantic versioning standards, but focuses on developer pain instead. Patch and minor versions shouldn’t cause any upgrading pain; major versions might give you a bit of trouble. Angular uses golden files with TypeScript, making it possible to guarantee API compatibility. Hundreds of thousands of tests help too.

If you’re a Java developer, Animal Sniffer and the japicmp Maven plugin might be useful for verifying compatibility. These are used by the Spring framework, and Okta’s Java team.

There’s a lot of developers in the world, and some of them work on open source. Many do it after their regular work hours. There’s also several lucky developers that get paid to develop and maintain open source software. It’s possible that the semantic versioning issues I experienced in 2019 are from independent open source developers. Maybe they don’t have the privilege of getting paid to work on their projects and spending time thinking about release version numbers?

Most of the folks I interviewed in this post are paid to work on their respective open source projects. They all seem to have somewhat rigorous processes for maintaining and releasing their projects. This seems to be the magic recipe: spend more time thinking about releases, reviewing your code, and setting version numbers. I love the thought of setting release numbers based on level of developer pain.

How do you succeed at open source releases? I think it’s important to test your libraries as much as you can. If you have a lot of projects depending on yours, join Open Collective and ask for donations. Encourage other developers to join your project and help out! Mentor developers and enter bugs and enhancements in your issuer tracker so folks know what you need help with. Don’t be afraid to increment your minor and major release numbers when you’re creating upgrade pain.

Finally, lock those versions down as much as you can for the sanity of your end users.

Good Luck!

In this post, you learned about the problems with semantic versioning and using ranges for versions. Using version ranges causes a lot of problems for Node developers. This isn’t semantic versioning’s fault. It’s often caused by humans who think they didn’t break anything in a patch or minor release, but they actually did.

You learned how popular projects like Spring Boot, Spring Framework, Angular, and JHipster set their release numbers. Everyone sets increments version numbers manually, and some Java projects use tools to guarantee a baseline of compatibility.

I believe that automated tools that assign version numbers to releases (based on backward compatibility) could be a fix for this problem. Unfortunately, I don’t know of any such tools. It also doesn’t seem to be a problem in the Java community where ranges aren’t really used.

If you have any ideas about how to improve semantic versioning and create truthful release numbers, I’d love to hear about it in the comments.

In the meantime, you might enjoy some of my other blog posts.

For more posts like this one, follow @oktadev on Twitter, follow us on LinkedIn, or subscribe to our YouTube channel.