Cutting a Release

The release process consists of:

  • the release manager cutting the release (documented below)

  • Members of the Apache Isis PMC verifying and voting on the release

  • the release manager performing post-release tasks, for either a successful or an unsuccessful vote.

Apache Isis itself is released in one go, everything is ultimately a child of isis-parent/pom.xml (relative to the source code root). This section details the process for formally releasing this module.

The subsequent sections describe how other committers can verify a release and how the release manager can then perform post-release activities and set up for the next development iteration.

If you’ve not performed a release before, then note that there are some configuration prerequisites that must be configured first. In particular, you’ll need signed public/private keys, and the ASF Nexus staging repo inlocal ~/.m2/settings.xml file.

These release notes using bash command line tools. They should work on Linux and MacOS; for Windows, use mSysGit.

Preparation

Obtain Consensus

Before releasing the framework, ensure there is consensus on the dev mailing list that this is the right time for a release. The discussion should include confirming the version number to be used, and to confirm content.

These discussions should also confirm the version number of the module being released. This should be in line with our semantic versioning policy.

Make sure you have a JIRA ticket open against which to perform all commits. In most cases a JIRA ticket will have been created at the beginning of the previous release cycle.

Pull down code to release

Set the HEAD of your local git repo to the commit to be released. This will usually be the tip of the origin’s master branch:

git checkout master
git pull --ff-only

License headers

The Apache Release Audit Tool RAT (from the Apache Creadur project) checks for missing license header files. The parent pom.xml of each releasable module specifies the RAT Maven plugin, with a number of custom exclusions.

To run the RAT tool, use:

find unapproved/missing licenses
mvn clean

pushd isis-parent
mvn org.apache.rat:apache-rat-plugin:check -D rat.numUnapprovedLicenses=1000 -Dreleased
popd

for a in `/bin/find . -name rat.txt -print`; do grep '!???' $a; done > /tmp/rat-qn.txt
for a in `/bin/find . -name rat.txt -print`; do grep '!AL' $a; done > /tmp/rat-al.txt
the parent pom.xml in each of these locations has the apache-rat-plugin appropriately configured.

This script runs over all submodules, including non-released modules.

The command writes out a target\rat.txt for each submodule. missing license notes are indicated using the key !???. The for command collates all the errors.

Investigate and fix any reported violations, typically by either:

  • adding genuinely missing license headers from Java (or other) source files, or

  • updating the <excludes> element for the apache-rat-plugin plugin to ignore test files, log files and any other non-source code files

  • also look to remove any stale <exclude> entries

Once you’ve fixed all issues, run the script again to confirm that all license violations have been fixed.

Missing License Check

Although Apache Isis has no dependencies on artifacts with incompatible licenses, the POMs for some of these dependencies (in the Maven central repo) do not necessarily contain the required license information. Without appropriate additional configuration, this would result in the generated DEPENDENCIES file and generated Maven site indicating dependencies as having "unknown" licenses.

Fortunately, Maven allows the missing information to be provided by configuring the maven-remote-resources-plugin. This is stored in the src/main/appended-resources/supplemental-models.xml file, relative to the root of each releasable module.

It’s first necessary to have built the framework locally at least once.

In the root directory:

mvn clean install -o -DskipTests

Once this is done, capture the missing license information using:

mvn license:download-licenses
groovy scripts/checkmissinglicenses.groovy

The Maven plugin creates a license.xml file in the target/generated-resources directory of each module. The script then searches for these licenses.xml files, and compares them against the contents of the supplemental-models.xml file.

For example, the output could be something like:

licenses to add to supplemental-models.xml:

[org.slf4j, slf4j-api, 1.5.7]
[org.codehaus.groovy, groovy-all, 1.7.2]

licenses to remove from supplemental-models.xml (are spurious):

[org.slf4j, slf4j-api, 1.5.2]

If any missing entries are listed or are spurious, then update supplemental-models.xml and try again.

Update and preview website

Generate the website, ensuring that the config, examples, projdoc (system overview and global index) are all updated:

sh preview.sh
this now needs to be run using Java 11 (because of the "projdoc" tooling).

Check for any Asciidoc errors, and fix.

Commit changes

Commit any changes from the preceding steps:

git add ..
git commit -m "$ISISJIRA: updates to pom.xml etc for release"

Releasing the Framework

Set environment variables

We use environment variables to parameterize as many of the steps as possible. For example:

export ISISJIRA=ISIS-9999                           (1)
export ISISTMP=/c/tmp                               (2)
export ISISREL=2.0.0-M6                             (3)
export ISISRC=RC1                                   (4)
export ISISBRANCH=release-$ISISREL-$ISISRC
export ISISART=isis
env | grep ISIS | sort
1 set to an "umbrella" ticket for all release activities. (One should exist already, created at the beginning of the development cycle now completing).
2 adjust by platform
3 adjust as required
4 adjust as necessary if this is not the first attempt to release

The branch name is intentionally not the same as the eventual tag names (eg isis-2.0.0-M6).

Create a release branch and worktree

The release is performed on a branch; if we are successful, this branch will be merged back into master.

We also recommend performing this work in a separate git worktree. The original worktree can stay with the master branch and be used for documentation fixes, etc.

  • create (but don’t checkout) a release branch for the version number being released; eg:

    git branch $ISISBRANCH
  • Create a worktree for this branch:

    git worktree add ../isis-release $ISISBRANCH
  • Switch to the new worktree and push the branch:

    cd ../isis-release
    git push origin $ISISBRANCH -u

Bump code to $ISISREL

Search through the .java files for 2.0.0-M5, and change to 2.0.0-M6.

At the time of writing there are just a couple of files that have the release hard-coded.

Switch to the isis-parent directory

The remainder of the release process is performed from the context of the isis-parent directory:

cd isis-parent

Bump projects to $ISISREL

We use mvn versions:set to manually bump the release version. (After release, there is a similar step at the end to reset back to a -SNAPSHOT version).

(Still in the isis-parent directory), the steps are:

  • First we bump the framework’s pom.xml files:

    mvn versions:set -DnewVersion=$ISISREL
  • next we build the framework so that these versioned pom.xml files are available for the next step:

    mvn install -o -DskipTests -Dreleased

    The released system property is set to only sanity check the modules actually to be released (ignores the incubator modules).

    Partly that’s because there’s no need to check the incubator modules, it’s also because the incubator modules require Java 11 (Java 8 is used for everything else)

  • next we also update the starters (as used by applications built using the framework):

    pushd ../starters
    mvn versions:set -DnewVersion=$ISISREL
    mvn install -DskipTests -o
    popd
  • finally we commit the changes:

    git add ..
    git commit -m "$ISISJIRA: bumps version to $ISISREL"

Sanity check

Perform one last sanity check on the codebase. Delete all Isis artifacts from your local Maven repo, then build using the -o offline flag:

rm -rf ~/.m2/repository/org/apache/isis
mvn clean install -o -Dreleased
git clean -dfx ..

Deploy

Since the <version> has already been updated, we just use mvn deploy to upload the artifacts. We activate the (inherited) apache-release profile to bring in the gpg plugin for code signing.

The build creates a zip of the directory, so before executing the release we remove any other files.

Still in the isis-parent directory, we deploy (upload the artifacts) using:

mvn deploy \
    -Dapache-release \
    -Dgit

When prompted, enter your GPG passphrase. (Or, it might be sufficient to add just -Dgpg.passphrase="…​")

This requires gpg v2.1 or later.

Tag the Release

Finally, tag the release:

git tag $ISISART-$ISISREL
git tag $ISISART-$ISISREL-$ISISRC

Check/Close Staging Repo

The mvn deploy commands will have uploaded all of the release artifacts into a newly created staging repository on the ASF Nexus repository server.

Log onto repository.apache.org (using your ASF LDAP account):

nexus staging 0

And then check that the release has been staged (select staging repositories from left-hand side):

nexus staging 1

If nothing appears in a staging repo you should stop here and work out why.

Assuming that the repo has been populated, make a note of its repo id; this is needed for the voting thread. In the screenshot above the id is org.apache.isis-008.

After checking that the staging repository contains the artifacts that you expect you should close the staging repository. This will make it available so that people can check the release.

Press the Close button and complete the dialog:

nexus staging 2

Nexus should start the process of closing the repository.

nexus staging 2a

All being well, the close should (eventually) complete successfully (keep hitting refresh):

nexus staging 3

The Nexus repository manager will also email you with confirmation of a successful close.

If Nexus has problems with the key signature, however, then the close will be aborted:

nexus staging 4

Use gpg --keyserver hkp://pgp.mit.edu --recv-keys nnnnnnnn to confirm that the key is available.

Unfortunately, Nexus does not seem to allow subkeys to be used for signing. See Key Generation for more details.

Reset revision property

At the beginning of the release process we bumped the version to the release version, ie $ISISREL. With the release now deployed we now need to reset the revision back down to the base snapshot, ie 2.0.0-SNAPSHOT.

Bumping back to 2.0.0-SNAPSHOT (rather than the next development snapshot) simplifies CI/CD scripts for teams that want to building Apache Isis from source using git submodules.

Therefore (still in the isis-parent directory):

mvn versions:set -DnewVersion=2.0.0-SNAPSHOT
mvn install -DskipTests -o -Dreleased

pushd ../starters
mvn versions:set -DnewVersion=2.0.0-SNAPSHOT
mvn install -DskipTests -o
popd

git add ..
git commit -m "$ISISJIRA: resetting version"

Push branch & tag

Push the release branch to origin:

git push -u origin $ISISBRANCH

and also push tag:

git push origin refs/tags/isis-$ISISREL:refs/tags/isis-$ISISREL-$ISISRC
git fetch

The remote tags aren’t visible locally but can be seen online.

Update starter apps

For each of the two starter apps, we maintain four branches:

  • jdo and jpa

    These are intended to reference the most recently released version, the first demonstrating persistence using JDO, the second using JPA. These are the branches referenced from the home page and getting started pages of the website.

  • jdo-SNAPSHOT and jpa-SNAPSHOT

    These reference the most current snapshot nightly build.

The general idea is that a release will fast-forward jdo to jdo-SNAPSHOT and similarly moves jpa up to jpa-SNAPSHOT, bumping to the newly released version of the framework in the process.

In order that we don’t break the starter apps while a release is being voted on, we do the changes in work branches, $ISISBRANCH-jdo and $ISISBRANCH-jpa:

  • for helloworld, we create a release branch for both variants:

    • for jdo:

      git checkout jdo-SNAPSHOT
      git pull --ff-only
      
      git checkout -b $ISISBRANCH-jdo
      
      mvn versions:update-parent -DparentVersion=$ISISREL (1)
      mvn versions:set -DnewVersion=$ISISREL
      
      mvn clean install -o
      mvn spring-boot:run
      1 the top-level pom.xml references the ASF staging repository, so this will pull down the release if not already present in ~/.m2/repository.

      Commit any changes and then push the branch to origin:

      git add .
      git commit -m "$ISISJIRA - updates to $ISISREL (jdo)"
      git push -u origin $ISISBRANCH-jdo
    • repeat for jpa:

      git checkout jpa-SNAPSHOT
      git pull --ff-only
      
      git checkout -b $ISISBRANCH-jpa
      
      mvn versions:update-parent -DparentVersion=$ISISREL
      mvn versions:set -DnewVersion=$ISISREL
      
      mvn clean install -o
      mvn spring-boot:run

      Commit any changes and then push the branch to origin:

      git add .
      git commit -m "$ISISJIRA - updates to $ISISREL (jpa)"
      git push -u origin $ISISBRANCH-jpa
  • for simple app, the steps are almost the same:

    • for jdo:

      git checkout jdo-SNAPSHOT
      git pull --ff-only
      
      git checkout -b $ISISBRANCH-jdo
      
      mvn versions:update-parent -DparentVersion=$ISISREL (1)
      mvn versions:set -DnewVersion=$ISISREL
      
      mvn clean install -Dmetamodel.lockdown -B -o        (2)
      1 the top-level pom.xml references the ASF staging repository, so this will pull down the release if not already present in ~/.m2/repository.
      2 the -B flag is required to avoid control characters in generated output

      Approve any failed lockdown tests (the mavendeps will fail first time around because the dependencies on Apache Isis itself have just be bumped).

      You could also check the output of the Cucumber tests, under:

      • webapp-tests/target/cucumber-reports and

      • webapp-tests/target/cucumber-html-reports.

      Repeat, then run the app as a sanity check:

      mvn -pl webapp spring-boot:run

      Commit any changes and then push the branch to origin:

      git add .
      git commit -m "$ISISJIRA - updates to $ISISREL (jdo)"
      git push -u origin $ISISBRANCH-jdo
    • repeat for jpa:

      git checkout jpa-SNAPSHOT
      git pull --ff-only
      
      git checkout -b $ISISBRANCH-jpa
      
      mvn versions:update-parent -DparentVersion=$ISISREL
      mvn versions:set -DnewVersion=$ISISREL
      
      mvn clean install -Dmetamodel.lockdown -B -o

      Approve any failed lockdown tests (the mavendeps will fail first time around because the dependencies on Apache Isis itself have just be bumped).

      Repeat, then run the app as a sanity check:

      mvn -pl webapp spring-boot:run

      Commit any changes and then push the branch to origin:

      git add .
      git commit -m "$ISISJIRA - updates to $ISISREL (jpa)"
      git push -u origin $ISISBRANCH-jpa

Preview website

We also prepare a preview of the next version of the website, then made accessible from https://isis.staged.apache.org.

  • Prerequisites:

    • clone the https://github.com/apache/isis-site repo, alongside the isis repo:

      git clone https://github.com/apache/isis-site ../isis-site
    • in the isis-site repo, check out the asf-staging branch:

      cd ../isis-site
      
      git checkout asf-staging
      git pull --ff-only
  • still in the isis-site repo, delete all the files in content/ except for the schema and versions directories:

    pushd content
    for a in $(ls -1 | grep -v schema | grep -v versions)
    do
        rm -rf $a
    done
    popd
  • Back in the isis repo’s release worktree, generate the Antora site (from the top-level directory).

    this now needs to be run using Java 11 (because of the "projdoc" tooling).
    cd ../isis-release
    
    sh preview.sh
  • Copy the generated Antora site to isis-site repo’s contents directory:

    cp -Rf antora/target/site/* ../isis-site/content/.
  • Back in the isis-site repo, commit the changes and preview:

    cd ../isis-site
    
    git add .
    git commit -m "$ISISJIRA : staging changes to website"
    
    sh preview.sh
  • If happy, then push the changes:

    git push origin asf-staging -u

Wait a minute or two; the site should be available at https://isis.staged.apache.org (nb: 'staged', not 'staging').

Voting

Once the artifacts have been uploaded, you can call a vote.

In all cases, votes last for 72 hours and require a +3 (binding) vote from members.

Start voting thread on dev mailing list

The following boilerplate is for a release of the Apache Isis Core. Adapt as required:

Use the following subject, eg:

[VOTE] Apache Isis Core release 2.0.0-M6 RC1

And use the following body:

I've just cut a new release of the Apache Isis Framework.

The source code zip artifact has been uploaded to a staging repository on
https://repository.apache.org, along with its corresponding .asc signature.

In the source code repo the code has been tagged as isis-2.0.0-M6-RC1;
see https://github.com/apache/isis/tags

To verify the source code itself, you can use the following commands
(in an empty directory):

----------------------------------------------------------------
VERSION=2.0.0-M6
RC=RC1
NEXUSREPONUM=11xx

curl https://downloads.apache.org/isis/KEYS > /tmp/KEYS
gpg --import /tmp/KEYS

rm -rf isis-$VERSION

curl -O -L https://raw.githubusercontent.com/apache/isis/release-$VERSION-$RC/scripts/verify-isis-release.sh

chmod +x ./verify-isis-release.sh
./verify-isis-release.sh $NEXUSREPONUM $VERSION $RC
----------------------------------------------------------------

You can then test the helloworld or simpleapp starter apps, see:
https://isis.staged.apache.org/comguide/latest/verifying-releases.html.

You can also inspect the website in general, available at:
https://isis.staged.apache.org.

Please verify the release and cast your vote.
The vote will be open for a minimum of 72 hours.

[ ] +1
[ ]  0
[ ] -1

Remember to update:

  • the version number (2.0.0-M6 or whatever)

  • the release candidate number (RC1 or whatever)

  • the NEXUSREPONUM to the repository id as provided by Nexus earlier (11xx or whatever)

Note that the email also references the procedure for other committers to verify the release.