Semantic Versioning and Release Automation on GitLab

Published in

Level Up Coding

18 min readJan 25, 2020

You have your project hosted on GitLab and use CI/CD, but you still manage branches and releases manually. As soon as your activity increases you lose your mind keeping track of versions, releases, tags, branches etc. It’s time to improve your workflow and automate a few things. It doesn’t matter how big your project or your team is. Just like Git, you don’t need a complex organisation to make things tidy, even if you’re working alone. In this post, I’m going to show you my suggested configuration to stick with the Semantic Versioning specification and how to get it working quickly.

UPDATE AS OF FEB 2022

After writing this post I started working on a new tool called Nyx to overcome the downsides of the solution herein described, coming to a cleaner and streamlined solution.

I also wrote a similar introductory article: Semantic Release Automation with Gradle using Nyx.

UPDATE AS OF DEC 2022

Nyx is now also available as a command line tool and as a Docker image. Another introductory article is: Nyx, the Semantic Release Automation Tool.

Branching model and workflow

This article is not about picking the right branching model for your project and can be used with any as it’s easily configurable.

If you’re hosting your project on GitLab give GitLabFlow a chance. It’s drawn with GitLab’s peculiar features in mind so it’s easy to leverage (long running) merge requests and environments, to name a couple and it’s not imperative about branches as it focuses more on automation.

However, nothing prevents you from using other models like GitFlow, OneFlow or GitHubFlow, to name the most popular. For a quick comparison you can start taking a look at this post. My bit of advice here is that you need to start figuring out what is the best workflow for your project and your team then start looking at those branching models and create your own variation if you need to. As long as you follow a feature-driven development process you can easily find your way to a simple and effective model. Moreover, it’s not a permanent decision and you can change the model at any time so don’t get stuck in over-engineering a workflow. Start simple and grow as needed.

In the rest of this post I’m just assuming you use:

the ubiquitous permanentmaster branch, where releases happen
maintenance branches (i.e. 1.0.x to maintain release 1.0)
preview branches like alpha
feature branches (like feature/NAME, fix/NAME) or any other kind of branch to commit small units of work that do not produce releases (until merged into one of the branches above). These branches are ignored from now on as they don’t have to trigger releases

How and when you create those branches and merge them back depends on your workflow.

Introducing semantic-release

I’m using semantic-release (v16.0.3 as of this writing) because it’s easy to plug, non invasive and, beside the automatic management of version numbers, has all of the extra features you may need like publishing, changelog generation and more. Kudos to these folks for their work! By the way, if you are going to use it please consider showing their badge on your project. Full docs are available here.

There are other several tools out there to automate bumping version numbers but not all also generate the changelog and publish releases or just make different assumptions. A few that I’ve found are go-semrel-gitlab, python-semver, semver-tool and semver.

Although semantic-release is a Node.js package it’s not specific for Node projects nor you need Node installed. You can use it for any kind of project, regardless the language. We are going to use it in a Node Docker container.

Enough words. Let’s get started.

semantic-release pros and cons

I want to save your time in case one of the cons I’ve found make semantic-release a no go in your case.

One downside is that unless you use one of the tweaks shown at the end of this post (see Knowing the version number before the release step) you don’t know the version number in advance so if you need to build your artifacts using the version number even when not actually releasing things get a little cluttered. Even with that tweak, you have to deal yourself with generating a version number when building on branches or commits skipped by semantic-release.

On the other hand, semantic-release lets you put your release management on autopilot so consider using it when none of the above limits applies to your project.

Preparing the environment

You need to generate a new personal access token with push access to your repository. To be more specific, you need to grant the api and the write_repository scopes to the token.

Now copy the generated token and use it for the value of a new CI/CD environment variable named GITLAB_TOKEN(or GL_TOKEN). The variable can be either a project specific or a group variable but it’s important to make it protected, like:

Example of the GITLAB_TOKEN variable configured for the GitLab project or group

From now on all the jobs running within a project pipeline will be able to use the GITLAB_TOKEN variable. If the variable is protected, only jobs running the pipeline for a protected branch will be able to access the variable.

semantic-release will use the token to tag releases and commit to the repository.

Consider that all of the actions taken by semantic-release (like tagging and publishing releases) will appear as performed by the user who issued the token. If you have large teams you probably go for a service user account (or Bot account).

Configuring semantic-release

Now, to configure semantic-release, create a .releaserc.yml file in the root directory of your Git repo (where the .gitlab-ci.yml lives). Edit the file to make it like:

plugins:
  - "@semantic-release/commit-analyzer"
  - "@semantic-release/release-notes-generator"
  - "@semantic-release/gitlab"
branches:
  - "master"
  - "+([0-9])?(.{+([0-9]),x}).x"
  - name: "alpha"
    prerelease: "alpha"

The above instructs semantic-release to treat the master branch as the release branch, while we may have any number of N.M.x maintenance branches plus a pre-release branch (alpha) that we want to trigger releases, separate from regular ones.

There are other options for this if you want to dig into the configuration. You may even skip the configuration file and pass all the options on the command line.

In this article I’m using YAML files for the configuration but you can easily use JSON instead.

The plugins section

Plugins do just what you expect: they extend or customize the behavior of semantic-release.

semantic-release has an internal sequence of steps, representing an internal workflow. At each step all the plugins are invoked for that specific step. When the list of plugins ends, the next step starts. Plugins are invoked at each step in the same order they are defined until all the last plugin has been triggered for the last step.

The plugins section is where we load plugins. It differs from the default plugins configured out of the box in that:

the @semantic-release/npm plugin is removed as we don’t need it (unless working on a Node project)
the @semantic-release/github plugin is replaced by @semantic-release/gitlab

Just for the records, the plugins section could have been replaced by the gitlab-config shareable configuration but that would bring in the npm plugin anyway.

Note that plugins will run in the same order they are declared in the file.

There are plenty of other plugins that you can play with. The full list is available here. If you want to add other plugins just add them in this section and make sure you download them in as part of the pipeline (see below). Just be careful about the order of plugins as it reflects the sequence plugins are executed.

The branches section

This section is where the workflow is configured from the semantic-release standpoint. Before we dive in the configuration details it’s useful to understand a few key points:

from the semantic-release standpoint each configured branch defines a type of release
conceptually, there are three types of releases that semantic-release handles: regular releases, pre-releases (like, alpha, beta, release candidate or rc, preview etc, you have the idea) and post-releases (a.k.a. maintenance releases). Each branch belongs to one of these families. Most of the logic used by semantic-release is meant to grant consistency among these types of releases so that maintenance releases can only issue version numbers within the range of the regular release they are tied to and pre-releases are identified by a version number that represents the early stage (like alpha, rc etc)
not all branches that you use in the project need to be defined to semantic-release: just the ones that define releases. For example feature branches are not configured just because committing in these branches doesn’t trigger a release. semantic-release will take action only when those branches are merged into some release branches (and that’s where the commit history is scrubbed to collect data to feed the change log)
conversely, not all branches configured into semantic-release must be actual branches in your Git repository. Those that do not exist are just ignored because there will be no commit to analyze. This allows you to have a generic configuration (that, for example, you can reflect in multiple projects) and you don’t need to worry about matching branches

Back to the configuration.

The branches: section is a list of branches where action is taken by semantic-release and depends on your branching model. Remember, these branches should be configured as protected branches.

Each item has the attributes:

name: it can be a simple branch name or a glob that will be evaluated to match with current branches. For example master, develop, pre, next are simple branch names while +([0-9])?(.{+([0-9]),x}).x is a glob that will match branches like 1.x, 2.2.x etc. Consider using this tester when using globs. If no actual branches in your repository match a certain item in the branches configuration then that item in the configuration will just be ignored (and considered if and when some actual branch will match).
channel: ignore it unless you know you need a release channel, depending on the artifacts you publish. Know it defaults to name when used. See here for more
range: makes a branch a maintenance branch. The value of range must be unique in the entire project and must be in the form N.M.x or N.x, where N and M are numbers (while x is fixed). When the branch name is defined as a glob the range is inferred from that so all actual branches that match the glob will have a range associated, which makes them maintenance branches. In other words, you don’t need to set the range attribute yourself
prerelease: it’s a string used by semantic-release to append a qualifier to the end of the release name. This attribute makes the branch a pre-release branch. For example, if prerelease is alpha, semantic-release will produce releases named like 1.0.0-alpha.1 (more generally x.y.z-alpha.j). This attribute may also be set to true, in which case the string used for the identifier is name

name is the only mandatory attribute to be defined of a branch or a glob and if other attributes are not used the branch can be defined by just the name, as for the master branch in the configuration example above. When only the name is defined other attributes are defaulted.

Although you don’t set it explicitly, each item has a type (remember the regular, pre, post families) among:

(regular) release: these are the branches where releases are actually evaluated and published. This type of branch is inferred by default for all branches not belonging to the other types. semantic-release needs at least one release branch and allows up to four. Unless you have different flavors, platforms etc you just need one release branch. Also see the Version ranges section below for more
maintenance (post-release): these branches are made to issue releases on top of older releases (like patches and hotfixes). For example a maintenance branch 1.0.x is a maintenance branch for release v1.0.0 through v1.0.N. Maintenance releases exist to avoid incurring in the Version ranges constraint and they are simply inferred by the presence of the range attribute so when a branch defines the range it’s automatically considered to be a maintenance branch
pre-release: these branches are intended to publish releases in advance of regular releases published by release branches. These releases are qualified by a trailing qualifier (the prerelease that is defined in the branch) and should be used for test and preview purposes

Remember: you can’t define the type of the branch in the configuration as it’s always inferred by the other attributes.

When you have multiple release branches (say master and preview, where preview is releasing versions before master) the order in which they are defined matters as versions released on a given branch must always be higher than the last release made on the previous branch. So in this example master must be defined before preview as it will mark a certain release after it’s been marked for preview. See the official docs for more on this.

Configuring the GitLab CI pipeline

All is left to do now is to add a job in the pipeline configuration to run semantic-release.

Open the .gitlab-ci.yml file (or other, if you’re not using the default name) in the root of your repo and add a job like this:

stages:
  # your existing stages here
  [...]
  - release# your existing jobs here
[...]release:
  image: node:13
  stage: release
  only:
    refs:
    - master
    - alpha
    # This matches maintenance branches
    - /^(([0-9]+)\.)?([0-9]+)\.x/
    # This matches pre-releases
    - /^([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?(?:\+[0-9A-Za-z-]+)?$/ 
  script:
    - npm install @semantic-release/gitlab
    - npx semantic-release

What we do here is:

add the release stage (in the stages: section) at the end of the pipeline (it must be at the end, after building, testing etc)
add the release job (in the release: section) to run semantic-release. The job runs in a Node container (image: node:13). Consider that semantic-release needs Node.js version 10.13 or higher. The job is executed in the release stage (stage: release) and only when the pipeline runs for the specific branches we want semantic-release to be triggered for. Then, the actual script: runs the required tasks to install the required plugins and install/run semantic-release

A couple of things to underline here:

the only: section is optional but makes the pipeline faster for all of those branches that do not need to run semantic-release. If you omit the section then the release: job is always executed, even when semantic-release just decides to skip action just because there is nothing to do. However this still implies that the Docker image is downloaded and the commands executed, adding useless work and time to your pipeline. If you use the only: section, make sure the branches and tags you specify here match the ones configured in the branches: section of the semantic-release configuration file (.releaserc.yml) above (using regexp this time)
if you have more plugins (configured in the plugins: section of the semantic-release configuration file) or shareable configurations, you need to install them by modifying the npm install command. You can add as many as you want on the same command line, example: @npm install @semantic-release/gitlab @semantic-release/exec adds the @semantic-release/exec plugin
the npx semantic-release command both downloads and runs semantic-release, so it’s where action takes place

Ready to go

You are ready to go now. Just commit and do merge requests and you’ll see your releases flow.

Remember that semantic-release uses the Angular commit message conventions by default to parse commit messages and produce the changelog in a conventional format so when you commit, just use that format for messages.

In summary, to get semantic-release to manage our releases automatically all we had to do was:

generate a Personal Access Token and set the GITLAB_TOKEN variable to hold its value
create the semantic-release configuration file (.releaserc.yml) in the project root to set up the plugins and the workflow (branches)
add a job in the pipeline configuration (.gitlab-ci.yml) to run semantic-release

Below you can find further discussion about custom cases and the internals.

Behind the scenes

If you’re curious to know what happens when a pipeline is triggered, based on the settings above, here’s the detail:

when a new commit happens(either via git push or a merge) GitLab starts a new pipeline
when the pipeline runs the job that starts semantic-release, the process goes through each release steps for each plugin, in the order they are configured
the latest release is determined by inspecting the tags and messages of commits
the analyzeCommits (the only required step) determines if a new release has to be created and, if so, which type
each plugin performs its core tasks, like generating release notes, changelogs or creating tags. Change logs are interpreted by default based on the Angular commit message conventions
the release is then published and notified by plugins

Note that semantic-release doesn’t store status files in your Git repository to save current versions. Instead, it only inspects the commit history to determine the current version and the next one. This makes it cleaner and less invasive, avoiding clutter in your repo.

How does semantic-release decide when to tag a version or not?

semantic-release makes this decision based on:

the branch that was committed (either via a merge request or a push) must be configured in the workflow configuration (the branches section of the configuration file)
the format of the commit message or the title of the merge request must match one of the formats configured for the @semantic-release/commit-analyzer plugin (which is usually configured to use presets)

These conditions must both be true, otherwise semantic-release will just skip versioning the commit.

When creating merge requests you often squash the commits belonging to the merge. This is a good practice to avoid cluttering your commit history but from the semantic-release point of view it makes the messages of single commits invisible. In other words, if you meant the commits to trigger a release by mean of their commit messages you will see that semantic-release doesn’t version the commit coming from the merge request because it just doesn’t see those messages anymore. This is true unless you set the title of the merge request to be a significant message, in which case semantic-release is able to detect the message and trigger the version.

Version ranges

If you configure multiple branches of the same type you may encounter the EINVALIDNEXTVERSION error, it looks like this:

 [3:54:19 PM] [semantic-release] › ✖  EINVALIDNEXTVERSION The release `x.y.z` on branch `name` cannot be published as it is out of range.
 Based on the releases published on other branches, only versions within the range >=x.y.z <x.y.z can be published from branch name.

This error means that one commit on one branch is supposed to be versioned with a version number that conflicts with an existing version of another branch of the same type, breaking the uniqueness of version numbers across branches.

From the official documentation:

A project must define a minimum of 1 release branch and can have a maximum of 3. The order of the release branch definitions is significant, as versions released on a given branch must always be higher than the last release made on the previous branch. This allow to avoid situation that would lead to an attempt to publish releases with the same version number but different codebase. When multiple release branches are configured and a commit that would create a version conflict is pushed, semantic-release will not perform the release and will throw an EINVALIDNEXTVERSION error, listing the problematic commits and the valid branches on which to move them.

To fix this you can avoid having multiple branches of the same type (i.e. release) or assign specific ranges to branches by mean of the range attribute in the workflow configuration (the branches section of the semantic-release configuration file). More on this can be found here.

Additional use cases

Publish a CHANGELOG file

In case you also want to generate the CHANGELOG file instead of just showing it in the releases of GitLab’s UI, add the changelog plugin to the .releaserc.yml file this way:

plugins:
- "@semantic-release/commit-analyzer"
- "@semantic-release/release-notes-generator"
- - "@semantic-release/changelog"
  - changelogFile: CHANGELOG.md
- "@semantic-release/gitlab"
branches:
- "master"
- "+([0-9])?(.{+([0-9]),x}).x"
- name: "alpha"
  prerelease: "alpha"

This way the CHANGELOG.md file is generated in the root folder of your project. You can change the name of the file or its path, relative to the root.

We need to add the plugin to the .gitlab-ci.yml file as well and by the way, in order not to lose the generated file at the end of the job we also (optionally) publish it among the pipeline artifacts:

release:
  image: node:13
  stage: release
  only:
    refs:
    - master
    - alpha
    # This matches maintenance branches
    - /^(([0-9]+)\.)?([0-9]+)\.x/
    # This matches pre-releases
    - /^([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?(?:\+[0-9A-Za-z-]+)?$/ 
  script:
    - touch CHANGELOG.md
    - npm install @semantic-release/gitlab @semantic-release/changelog
    - npx semantic-release
  artifacts:
    paths:
    - CHANGELOG.md

The touch CHANGELOG.md is useful in order to avoid the job failure because there is no file to publish, when semantic-release determines that there is nothing to publish.

Knowing the version number before the release step

Remember that semantic-release has to run at the end of the pipeline because the release is the last step after building, testing etc. However sometimes you can’t defer knowing the version number until the end of the pipeline because, for example:

you need to build the artifacts including the version number somewhere and somehow
testing requires coherence between the tested and released artifacts
dependency management and artifacts catalog need
and more…

Whatever the case, while still running the release job at the end, you need to fetch the version number in the early stages of your build pipeline. Here we are going to deal with this. Just keep in mind that:

this usage is deprecated by semantic-release, which follows a purist approach and considers version number should be generated during the actual release step
semantic-release will only give us version numbers when running pipelines in one of the branches it’s aware of (release branches, pre-release and maintenance) while in other branches (i.e. feature branches) it will just skip, if invoked, so in this case you need to generate your own numbers (unrelated to version number patterns to avoid clashes). We are going to deal with this as well

So to get this done we are going to extend the semantic-release configuration in order to use an additional plugin, @semantic-release/exec, which allows us to run arbitrary commands orchestrated in the semantic-release workflow steps. With more detail, we’ll use the plugin in the verifyRelease step (which happens after the other plugins in the previous steps have evaluated the commit history) but before others that are ignored when using the --dry-run option. At this stage we use a dry run because we are not issuing any release yet, just sneaking into the API.

Here is how the .releaserc.yml looks like:

plugins:
- "@semantic-release/commit-analyzer"
- "@semantic-release/release-notes-generator"
- - "@semantic-release/exec"
  - verifyReleaseCmd: "echo ${nextRelease.version} > VERSION.txt"
- "@semantic-release/gitlab"
branches:
- "master"
- "+([0-9])?(.{+([0-9]),x}).x"
- name: "alpha"
  prerelease: "alpha"

As you can see the new exec plugin just saves the next version number to a flat file VERSION.txt, that we will later see in the pipeline.

The .gitlab-ci.yml pipeline configuration:

stages:
  - fetch-version
  - build
  - releasefetch-semantic-version:
  # Requires Node >= 10.13 version
  image: node:13
  stage: fetch-version
  only:
    refs:
    - master
    - alpha
    - /^(([0-9]+)\.)?([0-9]+)\.x/ # This matches maintenance branches
    - /^([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?(?:\+[0-9A-Za-z-]+)?$/ # This matches pre-releases
  script:
    - npm install @semantic-release/gitlab @semantic-release/exec
    - npx semantic-release --generate-notes false --dry-run
  artifacts:
    paths:
    - VERSION.txtgenerate-non-semantic-version:
  stage: fetch-version
  except:
    refs:
    - master
    - alpha
    - /^(([0-9]+)\.)?([0-9]+)\.x/ # This matches maintenance branches
    - /^([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?(?:\+[0-9A-Za-z-]+)?$/ # This matches pre-releases
  script:
    - echo build-$CI_PIPELINE_ID > VERSION.txt
  artifacts:
    paths:
    - VERSION.txtbuild:
  stage: build
  script:
    - echo "Version is $(cat VERSION.txt)"release:
  image: node:13
  stage: release
  only:
    refs:
    - master
    - alpha
    # This matches maintenance branches
    - /^(([0-9]+)\.)?([0-9]+)\.x/
    # This matches pre-releases
    - /^([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?(?:\+[0-9A-Za-z-]+)?$/ 
  script:
    - npm install @semantic-release/gitlab
    - npx semantic-release

It looks like a lot of stuff but you can find the few relevant parts in bold. What we do here:

define a new stage in the pipeline named fetch-version. This stage runs at the beginning of the pipeline to fetch or generate the version number and make it available
define a new job fetch-semantic-version that runs at the fetch-version stage only on the branches managed by semantic-release. This time we use the options --generate-notes false --dry-run to avoid applying any change. However this still generates the VERSION.txt file because of the semantic-release configuration we have in place
define another job generate-non-semantic-version that runs at the fetch-version stage too but only when building on non release branches (the opposite of fetch-semantic-version) and generates a VERSION.txt file with a non-semantic version id that doesn’t clash with regular version numbers but we can still use to build and test artifacts. The version id generated here has the form build-<PIPELINE_ID> but you can chose anything else like timestamps, SHAs or random numbers. You can skip defining this job if you don’t need a version number in the pre-release stages but if you do, nest stages cane behave in an idempotent way, counting on the presence of a version id
produce the VERSION.txt file in both jobs (of course you can use any other name) and publish it as an artifact because it needs to be available across different build jobs
define another example job build` that just shows one example of using the value from VERSION.txt regardless of how and when it was generated