Skip to content

GitLab

Last edited by Pierre Paleo
Page history

How to do a release of Nabu

This page is a "cookbook" for doing a release of Nabu.

The release is named 20XX.Y.Z (Nabu uses Calendar versioning).

0. Define a "features scope"

Before "freezing" Nabu in a given state, we have to agree on what will be part of the release.

Define and write the features scope to prepare the changelog.

1. Create a new git branch and Gitlab Merge Request

A release should be based on a clean master.

Tidy your current local repository and create a new branch (based on master):

git checkout -b release_20XX.Y

Create a Merge Request on Gitlab.

All Merge Requests related to this release should have release_20XX.Y branch as a target

2. Update the readme and the documentation

2.1 - Regenerate APIdoc

cd doc
rm -rf apidoc
sphinx-apidoc --separate --ext-autodoc -o apidoc .. ../*/tests/*
rm -f apidoc/modules.rst
rm -f apidoc/setup.rst
sed -i '1s/.*/API reference/' apidoc/nabu.rst
sed -i '2s/.*/==============/' apidoc/nabu.rst

2.2 - Update the relevant pages

When new features are available, dedicated pages might have to be created in the documentation. Other pages have to be updated, for example features.md and the section "Latest news" in index.rst.

2.3 - Update the changelog

The changelog should advertise user-visible features and bugfixes.

2.4 - Publish the documentation

Once the pages are successfully built by Gitlab CI and manually checked, the documentation can be published.

Documentation is uploaded on www.silx.org/pub/nabu/doc

3. Test the software

We distinguish two types of tests:

  • Unit tests: run with nabu-test
  • End-to-end tests: using nabu.app or the CLI commands for reconstructing a dataset.

3.0 - Ensure the CI tests are passing

The CI tests have to pass before merging the current Merge Request.

3.1 - Build the wheel

All the tests should be done on an installed wheel.

This is closer to the real-world setting in which the final user installs Nabu from a wheel.

python setup.py bdist_wheel # build the wheel
python setup.py sdist # build the sources tarball

This step has to be re-done each time a bug is encountered/corrected.

3.2 - Create a fresh new environment for the tests

To spot "missing modules" errors, it is best to create a new python environment.

python3 -m venv /tmp/venv_nabu_release
source /tmp/venv_nabu_release/bin/activate
pip install --upgrade pip setuptools wheel
pip install nabu_wheel.whl

Then, run nabu-test. Many tests should be skipped (no pycuda, no scipy, etc). Progressively install new modules ; and for each module re-run the tests. Example:

nabu-test
pip install scipy
nabu-test
pip install pycuda
nabu-test
# ...

3.3 - Run manually the tests

Ideally, the tests have to be run:

  • On your current developing environment: tests in an environment having a number of pre-installed modules
  • On a clean environment: install from scratch with the dependencies from a wheel
  • On different machines:
    • scisoft10: Ubuntu 20.04, Python 3.8.2
    • scisoft13: Debian 8.8, Python 3.5.3, GTX 1080 Ti
    • scisoft15: Debian 10, Python 3.7.3, Tesla V100 (x2), PowerPC architecture

4. Release

4.1 - Tag the git release branch

git tag --sign -a v20XX.Y.Z
git push origin v20XX.Y.Z

(git push does not transfer tags to remote servers by default, you have to explicitly push the tag)

4.2 - Make a release candidate (optional)

It is strongly recommended to make release candidates to have an additional period of tests. This allows further discovering/fixing of remaining bugs.

Edit nabu/__init__.py to define __version__ = "20XX.Y.Z-rc1" (for release candidate 1).

4.3 - Deploy on pypi

Be careful when doing this step ! A pip release cannot be undone. If there is a bug in the released wheel, then a new release has to be done with a different version. So please ensure that no obvious bugs remain before doing the pip release.

Get the wheel to deploy (the one corresponding to the version you extensively tested on several machines/environments), and the corresponding sources (see section 3.1).
Now upload them on pypi:

First test with the "test" pypi instance:

twine upload --sign --repository testpypi mywheel.whl sources.tar.gz

If everything goes well, do the final deployment on pypi:

twine upload --sign mywheel.whl sources.tar.gz

You can do the above step with release candidate versions (see section 4.2). Once there is nothing more to do in release candidates, you can deploy the final version:

  • Edit nabu/__init__.py to define __version__ = "20XX.Y.Z" (final version).
  • Build the wheels and source (see section 3.1)
  • Perform the above steps. Make sure to first test with testpypi to ensure all goes smoothly.

5. Clean-up

  • Merge the "release pull request" on Gitlab. All the developments/corrections done for the release are now integrated in the master branch.
  • On your local repository, update your master branch with upstream (git checkout master && git pull upstream master)
  • Increment the minor version in version.py (ex. if the released version is 2020.1.0, increment to 2020.1.1)
  • Commit and push to master (incrementing the version does not need a new PR)

6. Nabu release policy

6.1 - Version numbers

Nabu uses Calendar versioning. This means that versions are in the form 20XX.Y.Z, for example 2020.3.0 for third major release of 2020.

6.2 - Major and minor releases

Calendar versioning does not define a direct equivalent of "major" and "minor" versions from semantic versining. In our case, the scheme "version 20XX.Y.Z" is the following:

  • Year: 20XX
  • Major release: Y (one-based)
  • Revision: Z (zero-based)

For example: version 2020.1.0 is the first major release (without revision) in 2020 ; version 2020.2.1 is the first revision of second major release of 2020.

6.3 - Release schedule

The DAU project silx uses a "fixed release rate" (every 3 or 6 months).
Nabu releases are done according to a roadmap, which can include intermediate releases. Therefore, there is no "fixed release rate", although we tend to make at least one major release every 3 months.

6.4 - Definition in the code

The nabu version is defined in one unique place: nabu/__init__.py.

We do not use "modifiers" (ex. "dev", "alpha", ...), except for release candidates. This means that versions usually look like __version__ = "2020.3.0", or __version__ = "2020.3.0-rc1" (release candidate 1).

This means that once a new version is released (ex. 2020.3.0), the revision (number Z) is immediately incremented in nabu/__init__.py (ex. 2020.3.1). For example, installing version 2020.3.0 from pypi provides this final (major) release ; and an installation from gitlab will provide version 2020.3.1 (more recent but not released).

Installations from gitlab should always be considered as a development versions, except if the commit corresponds to a tagged release.