TokuMX Versioning

This document describes the process for versioning a new release and the ongoing procedures and workflows (branches and tags) around release engineering.

Definitions and general practices

Versioning

We follow the guidelines of semver.org. Mostly so far this means using the -pre- suffix for pure development versions, and the -rc.N suffix for release candidates.

The version is made of two parts, the "upstream compatibility" part (2.2.5) and the "TokuMX" part (1.0.3). These are stored separately in the binaries but for the most part we care about the TokuMX part. We apply semver to this.

The version information is stored in src/mongo/util/version.cpp.

Tagging

We use annotated git tags to mark releases, where the tag name and annotation is just the TokuMX version string. Some day we may want to sign our tags or provide more information in the annotation.

Branching

The master branch is considered to be a prerelease of the least unreleased minor version, with patch version 0. So when our most recent release is 3.4.5, master will be considered to be on release 3.5.0-pre-.

For each minor version X.Y, there is a branch releases/X.Y that was branched from master at version X.Y.0-rc.0, and contains the history of patches, version bumps, and tags for all patch versions in that series.

The same theory applies to the ft-index repository. When we're ready for a new minor release of TokuMX or TokuDB, the new release branch is done off master. So, avoid merging something to master unless it's ready to go to production some day, and if you have something in ft-index that needs to go in a patch release, follow the procedure below for patch releases, but on the ft-index repository.

NEWS file

We provide a NEWS file that contains a user-friendly log of changes for each version. It's best to update this with new features before making a new release branch, preferably as the features are added. The NEWS file is in asciidoc format.

Release process

Patch version release

We'll use the following example: assume a bug is found that is present in both versions 3.1.4 and 3.4.5. We need to fix the bug on a branch, test the fix, merge it to both the 3.1 series and the 3.4 series, bump the patch versions of both series, and make new builds.

We can use git merge-base to find a commit these branches all have in common, where we can start working.

1. First, make sure we're up to date, and then make a branch from the most recent point in common:

$ git fetch
$ git merge-base origin/master origin/releases/3.1 origin/releases/3.4
d0ac95bf23bcdb2ccfcd1bb22113a4eae57e1815
$ git checkout -b bugs/nasty-bug-1 d0ac95bf23bcdb2ccfcd1bb22113a4eae57e1815

2. Fix the bug on this branch, and do any manual testing you need to do.

$ emacs src/whatever
$ scons something
$ git add src/whatever
$ git commit -m "fixing nasty-bug-1"

3. Merge this bugfix to all affected minor version branches, including master if it's affected:

$ git checkout releases/3.1 && git merge origin/releases/3.1
$ git merge bugs/nasty-bug-1
$ git checkout releases/3.4 && git merge origin/releases/3.4
$ git merge bugs/nasty-bug-1
$ git checkout master && git merge origin/master
$ git merge bugs/nasty-bug-1
$ git push origin releases/3.1; git push origin/releases/3.4; git push origin master

4. For each minor version, make a -rc.0 version bump, including a tag, and do a full build and test (see Testing).

$ git checkout releases/3.1
$ emacs src/mongo/util/version.cpp  # change "3.1.4" (it should really be "3.1.5-pre-" at this point) to "3.1.5-rc.0"
$ git add src/mongo/util/version.cpp
$ git commit -m "tokumx-3.1.5-rc.0"
$ git tag -a -m 3.1.5-rc.0 3.1.5-rc.0
$ git push origin releases/3.1; git push origin 3.1.5-rc.0
$ # ... same for releases/3.4

5. If the release candidates don't pass, fix them in bugs/nasty-bug-1 and return to step 3, this time making -rc.1.
6. When the release candidate is good, do one more version bump, this time to a full patch release, followed by a version bump to the prerelease for the next patch release:

$ git checkout releases/3.1
$ emacs src/mongo/util/version.cpp  # change "3.1.5-rc.0" to "3.1.5"
$ git add src/mongo/util/version.cpp
$ git commit -m "tokumx-3.1.5"
$ git tag -a -m 3.1.5 3.1.5
$ git push origin 3.1.5
$ emacs src/mongo/util/version.cpp  # change "3.1.5" to "3.1.6-pre-"
$ git add src/mongo/util/version.cpp
$ git commit -m "tokumx-3.1.6-pre-"
$ git push origin releases/3.1
$ # ... same for releases/3.4

Moving commits to a release branch

When you decide you need to put a change on a release branch (for the next patch release), you have a few options, depending on when you decide this:

1. If you know before you make the change, follow the procedure above. Make a branch off the most recent common ancestor, and merging will be easy.
2. If you find out after you've started working (let's assume in a branch off master), but before you merge to master, you can rebase onto the most recent common ancestor commit, and continue working as above, and merging will be easy:

$ git merge-base releases/3.1 master
d0ac95bf23bcdb2ccfcd1bb22113a4eae57e1815
$ git checkout features/woozle
$ git rebase --onto d0ac95bf23bcdb2ccfcd1bb22113a4eae57e1815 master...features/woozle

3. If you've already merged to master, it's going to be a pain. We don't want to rebase things that have already been pushed to master, so our only option is to cherry-pick all the relevant commits for the change. This is likely to be the most frustrating option for you and for people to come (because now there are effectively two copies of each of your changes, and it will confuse git's merge algorithm so others will get merge conflicts long after your change has gone in).

$ git log master  # write down all the commit hashes for the commits you need to move to the release branch
$ git checkout releases/3.1
$ git cherry-pick 1234abc abcd123 45678ef ...
...  # resolve conflicts as they come, and use git cherry-pick --continue to move to the next commit

Major/minor version release

This is roughly the same in terms of making a release candidate, tagging, testing, etc. except that we have to make a new releases/X.Y branch from master. Take the example of releasing 3.5.0. We're starting with master at 3.5.0-pre-:

1. Make the new branch:

$ git checkout master
$ git checkout -b releases/3.5

2. Do the first release candidate version bump, and also do a version bump for master:

$ emacs src/mongo/util/version.cpp  # change "3.5.0-pre-" to "3.5.0-rc.0"
$ git add src/mongo/util/version.cpp
$ git commit -m "tokumx-3.5.0-rc.0"
$ git tag -a -m 3.5.0-rc.0 3.5.0-rc.0
$ git push origin releases/3.5; git push origin 3.5.0-rc.0
$ git checkout master
$ emacs src/mongo/util/version.cpp  # change "3.5.0-pre-" to "3.6.0-pre-"
$ git add src/mongo/util/version.cpp
$ git commit -m "tokumx-3.6.0-pre-"
$ git push origin master

3. Continue working on releases/3.5, building and testing, adding fixes on a bugs/ branch that get merged to both master and releases/3.5, and adding new release candidates until all tests pass.
4. Do one more version bump to 3.5.0 and tag it, then do a bump to 3.5.1-pre- to get ready for the first patch release.