Preparing a release

Rules for release branches:

  • Branch from develop

  • Naming convention: package-* where * is a version number

Release policy and release numbering

We use a MAJOR.MINOR.PATCH scheme to label releases. We adhere to the idea of semantic versioning (semantic versioning was introduced with release 0.9, see Issue 200): Given a version number MAJOR.MINOR.PATCH, we increment the:

  • MAJOR version when we make incompatible API changes,

  • MINOR version when we add functionality in a backwards-compatible manner, and

  • PATCH version when we make backwards-compatible bug fixes.

However, as long as the MAJOR number is 0 (i.e. the API has not stabilized), even MINOR increases may introduce incompatible API changes. As soon as we have a 1.0.0 release, the public API can only be changed in a backward-incompatible manner with an increase in MAJOR version.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

The CHANGELOG lists important changes for each release.

MAJOR, MINOR and PATCH numbers are integers that increase monotonically.

The release number is set in setup.py and in MDAnalysis.__version__ (MDAnalysis/version.py), e.g.

RELEASE = '0.7.5'

While the code is in development (i.e. whenever we are not preparing a release!) the release number gets the suffix -dev0, e.g.

RELEASE = '0.7.6-dev0'

so that people using the develop branch from the source repository can immediately see that it is not a final release. For example, “0.7.6-dev0” is the state before the 0.7.6 release.

Typical workflow for preparing a release

Summary of tasks

  • Declare a feature freeze on develop via discord and/or GitHub Discussions (Announcement)

  • Finalize the CHANGELOG file with the date of release

  • Increment the version across MDAnalysis and MDAnalysisTests (4 places)

  • Merge changes into develop

  • Create a tag from develop named release-<version_number>

  • Create a package-<version_number> branch from develop

  • Add rebuilt C / C++ files to package-<version_number>

  • Create a tag from package-<version_number>

  • Check automated testing of source distribution and wheel generation and upload

  • Create a new release from newly created tag

  • Check that deployment actions have adequately pushed dist and wheels to PyPi

  • Manually upload Cirrus CI wheels (temporary)

  • Update conda-forge packages

  • Create a blog post outlining the release

  • Increment develop branch files ready for the next version

  • Clean up old dev docs builds

Getting the develop branch ready for a release

  1. Declare feature freeze on develop via discord and GitHub Discussions (Announcement)

  2. Create a pre-release feature branch from develop

  3. Finalise the CHANGELOG with the release number and date. Ensure that the CHANGELOG summarizes important changes and includes all authors that contributed to this release.

  4. Make sure the version number matches the release version. The following files need to be updated: package/MDAnalysis/version.py, package/setup.py, testsuite/MDAnalysisTests/__init__.py, and testsuite/setup.py.

  5. Create a pull request against develop from this branch.

Packaging the release

  1. Create a new tag from develop named release-<version_number> where <version_number> is the release version number (this tag contains a snapshot of the Python source files as they were when the release was created):

    git tag -m "release 0.7.5 of MDAnalysis and MDAnalysisTests" release-0.7.5
git push --tags origin

  2. Create a package-<version_number> branch from develop. This branch is automatically protected, so you will also need to create a separate branch to create commits via PR against package-<version_number>.

  3. Generate new C/C++ files and commit them to the package-<version_number> branch via PR. We recommend generate the C/C++ files by building a source distribution tarball so you can check at the same time that there are no issues with the distribution creation process:

    # MDAnalysis
cd package/
pipx run build --sdist

  4. Once committed, create a new tag based on package-<version_number> (this tag will contain a record of all the files as they were deployed to users for that version):

    git tag -m "package 0.7.5 of MDAnalysis and MDAnalysisTests" package-0.7.5
git push --tags origin

  5. Upon creation of the new package-* tag, the deploy github action workflow will be automatically triggered to create source/wheels, upload them to testpypi, re-download them and run tests.

  6. If all the tests come back green, you are good to go for a full release.

    1. If tests fail you will need to work out the cause of the failure.

      1. A temporary github actions failure

        Re-run the action and wait for the tests to complete

      2. An issue with the source code.

        1. Delete the current package-* branch, and the newly created tags

        2. Add the new changes to develop and restart the release process.

        3. If the code had successfully uploaded to testpypi and failed later, you will need to create a test package-* tag which contains a different release number of in the source code (bumpy by a minor release or add a -beta modifier). Note: if the code had not successfully uploaded you can just continue the release process as normal.

        4. If CI comes back green then delete the test tag, and create a normal package-* tag with the correct version number.

        5. The github action will fail, but this is ok since we tested it with the test tag above.

Completing the release

If everything works, you can now complete the release by:

  1. Creating a release on GitHub based on the newly created package-<version_number> tag.

  2. Make sure you include relevant release notes, including any known issues and highlights for the release.

  3. Once published, the deploy github action will be triggered which will upload the source distributions and wheels to PyPI.

    1. If the deploy github action fails and no files have been uploaded, then restart the action.

    2. If the action fails and some files have been uploaded, then you will not be able to re-upload to PyPI. At this point you will need to yank the release from PyPI and create a new minor version and re-deploy it.

Manually upload Cirrus CI wheels (temporary)

Unfortunately the deployment of Cirrus CI generated wheels (for osx-arm64 and linux-aarch64) does not get properly triggered by a release. However, they are properly uploaded to TestPyPi

  1. Go to the recently updated TestPyPi release and download all the .whl files which have the tags arm64 and aarch64.

  2. From a local directory upload these wheels using twine.

    twine upload -r pypi *.whl --verbose

Update conda-forge packages

On push to PyPI, the conda-forge bot should automatically pick up the presense of a new version and create a pull request on the MDAnalysis feedstock and the MDAnalysisTests feedstock. You will need to merge the MDAnalysis feedstock followed by the MDAnalysisTests feedstock in order for the new package to appear on conda-forge.

To do this you will need to:

  1. Update the meta.yaml files as necessary, especially bumping up the python and dependency minimum versions as necessary.

  2. If NumPy pins differ from those conda-forge uses, you will need to update the conda_build_config.yaml accordingly.

  3. Ask the conda-forge bot to re-render, check that CI returns green, approve and merge the pull request.

Create a release of the UserGuide

For now, the UserGuide is released at the same time as the core library. If it’s failing please fix before you do the tag / release. Here is how to update the snapshots

  1. Update the version of MDA used by the UserGuide to the release version.

  2. Re-generate the Syrupy test snapshots, and commit those a to git and confirm the build passes.

  3. Make a Pull Request with a re-generated releases.md which contains a copy of the GitHub release notes. This can be generated by doing:

    cd doc/source/scripts
python gen_release_notes.py

  4. Create a new release tag and upload them for the UserGuide repository.

    git tag -m 'release 2.6.1 of the MDAnalysis UserGuide' release-2.6.1
git push --tags origin

  5. This will automatically trigger a Github Action to build a new set of docs for that release and upload them. Due to the large size of the gh-pages branch on the UserGuide, this can be both slow and flaky, make sure to keep an eye out for any potential failures.

Create a blog post outlining the release

Create a blog post outlining the release notes and publicize it on GitHub Discussions / discord / twitter/ etc…!

Increment develop branch files ready for the next version

Once the release is completed you can go ahead and update the develop branch so that it is ready for the next round of development.

  1. Update the 4 version file locations with the -dev0 appended version of the next release.

  2. Update the CHANGELOG with a new entry for the next release.

  3. Once these changes are merged into the develop branch, message the developers on discord and GitHub Discussions letting them know that the feature freeze is over.

Clean up old developer builds of the documentation

Whilst new docs are automatically deployed on a release, old developer builds (appended with -dev) are not automatically cleaned up. To avoid causing large amounts of files being uploaded to GitHub Pages, we need to delete these old developer builds manually. To do this switch to the gh-pages branch, delete these old files, and push the change directly. You should do this for both the core library and the UserGuide.

While this is still a manual procedure, you should also edit versions.json to remove the old dev links.