<
Fork me on GitHub

Committers' Guide

This committers' guide is for committers of Apache Isis itself who want guidance on release process, publishing documents and other related procedures.

Apache Isis documentation is broken out into a number of user, reference and "supporting procedures" guides.

The user guides available are:

The reference guides are:

The "supporting procedures" guides are:

This guide provides guidance for Apache Isis' own committers.

Merging a Pull Request

The process for merging in github pull requests (so that they can be tested locally before committing) has been scripted in the github-pr.sh script.

The script will merge the fork into a temporary branch, and then run a build. Once you are happy, you can commit.

The script requires jq to parse JSON; see the section below on installing pre-requisites.

Process and Usage

The overall process is as follows:

  • locate/raise corresponding JIRA ticket, eg ISIS-1162

  • checkout branch from which PR was forked (usually just 'master')

  • merge PR into temporary branch using the github-pr.sh script

  • test the change locally (run the app, rebuild, manual regression tests etc)

  • if required, tidy up/refactor code as required

  • merge temporary branch into mainline, and commit

For example:

github-pr.sh isis 1162 31

where:

  • isis is the JIRA project and repo

  • 1162 is the JIRA ticket number

  • 31 is the gthub PR issue number

Example transcript

The listing below shows the steps taken by the script:

$ sh github-pr.sh isis 1162 31

Found JIRA ticket
Found github PR
branch_name_local: master
username         : sebadiaz
repo_full_name   : sebadiaz/isis
repo_clone_url   : https://github.com/sebadiaz/isis.git
branch_name_fork : master

merging into: ISIS-1162_pr-31

Deleting branch 'ISIS-1162_pr-31'
Deleted branch ISIS-1162_pr-31 (was bd2e3c2).
Creating the branch ISIS-1162_pr-31
Switched to a new branch 'ISIS-1162_pr-31'
Pulling the changes from https://github.com/sebadiaz/isis.git master
From https://github.com/sebadiaz/isis
 * branch            master     -> FETCH_HEAD
Auto-merging core/pom.xml
Merge made by the 'recursive' strategy.
 core/pom.xml                                       |   3 +-
 .../apache/isis/security/shiro/IsisLdapRealm.java  | 198 +++++++++++++++++++--
 2 files changed, 186 insertions(+), 15 deletions(-)

Merged the PR; hit enter to build

The build now commences. Once done, the script continues:

If build successful and happy to merge, execute:

git checkout master && git merge --no-ff ISIS-1162_pr-31 && git branch -d ISIS-1162_pr-31

The screenshot belows shows the history we end up with:

github pr history

This shows the fork being merged into the temporary branch ("ISIS-1162_pr-31"), then some further tidy-up, and finally the merging of the temporary branch into mainline.

Note that there is no rebasing in this model. This is intentional: when the merged branch is pushed, github will automatically close the original pull request.

Prerequisites

The script uses 'jq' to parse JSON. To install:

  • on Linux:

    aptitude install jq
  • on MacOS:

    brew install jq
  • on Windows:

    Download exe from website

Cutting a Release

The release process consists of:

  • the release manager cutting the release (documented below)

  • Members of the Apache Isis PMC verifying and voting on the release

  • the release manager performing post-release tasks, for either a successful or an unsuccessful vote.

Apache Isis itself consists of two separately releasable modules; relative to the source code root there are:

  • core

  • component/example/archetypes/simpleapp

This section details the process for formally releasing Isis modules. It describes the process for both core and then the archetype. The subsequent sections describe how other committers can verify a release and how the release manager can then perform post-release activities and set up for the next development iteration.

If you’ve not performed a release before, then note that there are some configuration prerequisites that must be configured first. In particular, you’ll need signed public/private keys, and the ASF Nexus staging repo inlocal ~/.m2/settings.xml file.

These release notes using bash command line tools. They should work on Linux and MacOS; for Windows, use mSysGit.

Obtain Consensus

Before releasing core, ensure there is consensus on the dev mailing list that this is the right time for a release. The discussion should include confirming the version number to be used, and to confirm content.

These discussions should also confirm the version number of the module being released. This should be in line with our semantic versioning policy.

Make sure you have a JIRA ticket open against which to perform all commits. In most cases a JIRA ticket will have been created at the beginning of the previous release cycle.

Set environment variables

We use environment variables to parameterize as many of the steps as possible. For example:

cd core
export ISISTMP=/c/tmp              (1)
export ISISDEV=1.13.0-SNAPSHOT
export ISISREL=1.12.0
export ISISRC=RC1
export ISISBRANCH=release-$ISISREL-$ISISRC
export ISISJIRA=ISIS-9999          (2)

env | grep ISIS | sort
1 adjust by platform
2 set to an "umbrella" ticket for all release activities. (One should exist already, created at the beginning of the development cycle now completing).

Obviously, alter $ISISDEV and $ISISREL as required, and bump $ISISRC for re-releasing following an unsuccessful releases.

Note that the branch name is not the same any of the eventual tag names (eg isis-1.12.0 or simpleapp-archetype-1.12.0).

If they did have the same name, then what would happen is that the maven-release-plugin would checkout the (HEAD of the) branch and thus upload a SNAPSHOT to the snapshot repository. What it should of course do is checkout the tag and then upload that to the release staging repository.

Pull down code to release

Set the HEAD of your local git repo to the commit to be released. This will usually be the tip of the origin’s master branch. Then, create a release branch for the version number being released; eg:

git checkout master
git pull --ff-only
git checkout -b $ISISBRANCH

All release preparation is done locally; if we are successful, this branch will be merged back into master.

Double check that the version number of the parent pom should reflect the branch name that you are now on (with a -SNAPSHOT suffix). his will normally have been done already during earlier development; but confirm that it has been updated. If it has not, make the change.

Double check that the version number of the core POM (core/pom.xml) should reflect the branch name that you are now on. For example, if releasing version 1.12.0, the POM should read:

<groupId>org.apache.isis.core</groupId>
<artifactId>isis</artifactId>
<version>1.12.0-SNAPSHOT</version>

Also, check that there are no snapshot dependencies:

grep SNAPSHOT `/bin/find . -name pom.xml | grep -v target | grep -v mothball | sort`

The only mention of SNAPSHOT should be for the Isis modules about to be released.

Obviously, don’t update Apache Isis' SNAPSHOT references; these get updated by the mvn release:prepare command we run later.

Releasing Core

First, we release core. Switch to the appropriate directory:

cd core

Set environment variables

Set additional environment variables for the core "artefact":

export ISISART=isis
export ISISCOR="Y"

env | grep ISIS | sort

License headers

The Apache Release Audit Tool RAT (from the Apache Creadur project) checks for missing license header files. The parent pom.xml of each releasable module specifies the RAT Maven plugin, with a number of custom exclusions.

To run the RAT tool, use:

mvn org.apache.rat:apache-rat-plugin:check -D rat.numUnapprovedLicenses=50 -o && \
for a in `/bin/find . -name rat.txt -print`; do grep '!???' $a; done || \
for a in `/bin/find . -name rat.txt -print`; do grep '!AL' $a; done

where rat.numUnapprovedLicenses property is set to a high figure, temporarily overriding the default value of 0. This will allow the command to run over all submodules, rather than failing after the first one. The command writes out a target\rat.txt for each submodule. missing license notes are indicated using the key !???. The for command collates all the errors.

Investigate and fix any reported violations, typically by either:

  • adding genuinely missing license headers from Java (or other) source files, or

  • updating the <excludes> element for the apache-rat-plugin plugin to ignore test files, log files and any other non-source code files

  • also look to remove any stale <exclude> entries

To add missing headers, use the groovy script addmissinglicenses.groovy (in the scripts directory) to automatically insert missing headers for certain file types. The actual files checked are those with extensions specified in the line def fileEndings = [".java", ".htm"]:

groovy ../scripts/addmissinglicenses.groovy -x

(If the -x is omitted then the script is run in "dry run" mode). Once you’ve fixed all issues, confirm once more that apache-rat-plugin no longer reports any license violations, this time leaving the rat.numUnapprovedLicenses property to its default, 0:

mvn org.apache.rat:apache-rat-plugin:check -D rat.numUnapprovedLicenses=0 -o && \
for a in `find . -name rat.txt -print`; do grep '!???' $a; done

Missing License Check

Although Apache Isis has no dependencies on artifacts with incompatible licenses, the POMs for some of these dependencies (in the Maven central repo) do not necessarily contain the required license information. Without appropriate additional configuration, this would result in the generated DEPENDENCIES file and generated Maven site indicating dependencies as having "unknown" licenses.

Fortunately, Maven allows the missing information to be provided by configuring the maven-remote-resources-plugin. This is stored in the src/main/appended-resources/supplemental-models.xml file, relative to the root of each releasable module.

To capture the missing license information, use:

mvn license:download-licenses && \
groovy ../scripts/checkmissinglicenses.groovy

The Maven plugin creates a license.xml file in the target/generated-resources directory of each module. The script then searches for these licenses.xml files, and compares them against the contents of the supplemental-models.xml file.

For example, the output could be something like:

licenses to add to supplemental-models.xml:

[org.slf4j, slf4j-api, 1.5.7]
[org.codehaus.groovy, groovy-all, 1.7.2]

licenses to remove from supplemental-models.xml (are spurious):

[org.slf4j, slf4j-api, 1.5.2]

If any missing entries are listed or are spurious, then update supplemental-models.xml and try again.

Commit changes

Commit any changes from the preceding steps:

git commit -am "$ISISJIRA: updates to pom.xml etc for release"

Sanity check

Perform one last sanity check on the codebase. Delete all Isis artifacts from your local Maven repo, then build using the -o offline flag:

rm -rf ~/.m2/repository/org/apache/isis
mvn clean install -o

Release prepare "dry run"

Most of the work is done using the mvn release:prepare goal. Since this makes a lot of changes, we run it first in "dry run" mode; only if that works do we run the goal for real.

Run the dry-run as follows:

mvn release:prepare -P apache-release -D dryRun=true \
    -DreleaseVersion=$ISISREL \
    -Dtag=$ISISART-$ISISREL \
    -DdevelopmentVersion=$ISISDEV

You may be prompted for the gpg passphrase.

Experiments in using --batch-mode -Dgpg.passphrase="…​" to fully automate this didn’t work; for more info, see here (maven release plugin docs) and here (maven gpg plugin docs).

Release prepare "proper"

Assuming this completes successfully, re-run the command, but without the dryRun flag and specifying resume=false (to ignore the generated release.properties file that gets generated as a side-effect of using git). You can also set the skipTests flag since they would have been run during the previous dry run:

mvn release:prepare -P apache-release -D resume=false -DskipTests=true \
        -DreleaseVersion=$ISISREL \
        -Dtag=$ISISART-$ISISREL \
        -DdevelopmentVersion=$ISISDEV

If there are any snags at this stage, then explicitly delete the generated release.properties file first before trying again.

Post-prepare sanity check

You should end up with artifacts in your local repo with the new version (eg 1.12.0). This is a good time to do some quick sanity checks; nothing has yet been uploaded:

  • unzip the source-release ZIP and check it builds.

  • Inspect the DEPENDENCIES file, and check it looks correct.

These steps can be performed using the following script:

rm -rf $ISISTMP/$ISISART-$ISISREL
mkdir $ISISTMP/$ISISART-$ISISREL

if [ "$ISISCOR" == "Y" ]; then
    ZIPDIR="$M2_REPO/repository/org/apache/isis/core/$ISISART/$ISISREL"
else
    ZIPDIR="$M2_REPO/repository/org/apache/isis/$ISISCPT/$ISISART/$ISISREL"
fi
echo "cp \"$ZIPDIR/$ISISART-$ISISREL-source-release.zip\" $ISISTMP/$ISISART-$ISISREL/."
cp "$ZIPDIR/$ISISART-$ISISREL-source-release.zip" $ISISTMP/$ISISART-$ISISREL/.

pushd $ISISTMP/$ISISART-$ISISREL
unzip $ISISART-$ISISREL-source-release.zip

cd $ISISART-$ISISREL
mvn clean install

cat DEPENDENCIES

popd

Release perform (Upload)

Once the release has been built locally, it should be uploaded for voting. This is done by deploying the Maven artifacts to a staging directory (this includes the source release ZIP file which will be voted upon).

The Apache staging repository runs on Nexus server, hosted at repository.apache.org. The process of uploading will create a staging repository that is associated with the host (IP address) performing the release. Once the repository is staged, the newly created staging repository is "closed" in order to make it available to others.

Use:

mvn release:perform -P apache-release \
    -DworkingDirectory=$ISISTMP/$ISISART-$ISISREL/checkout

The custom workingDirectory prevents file path issues if releasing on Windows. The command checks out the codebase from the tag, then builds the artifacts, then uploads them to the Apache staging repository:

...
[INFO] --- maven-release-plugin:2.3.2:perform (default-cli) @ isis ---
[INFO] Performing a LOCAL checkout from scm:git:file:///C:\APACHE\isis-git-rw\co
re
[INFO] Checking out the project to perform the release ...
[INFO] Executing: cmd.exe /X /C "git clone --branch release-1.12.0 file:///C:\APACHE\isis-git-rw\core C:\APACHE\isis-git-rw\core\target\checkout"
[INFO] Working directory: C:\APACHE\isis-git-rw\core\target
[INFO] Performing a LOCAL checkout from scm:git:file:///C:\APACHE\isis-git-rw
[INFO] Checking out the project to perform the release ...
[INFO] Executing: cmd.exe /X /C "git clone --branch release-1.12.0 file:///C:\APACHE\isis-git-rw C:\APACHE\isis-git-rw\core\target\checkout"
[INFO] Working directory: C:\APACHE\isis-git-rw\core\target
[INFO] Executing: cmd.exe /X /C "git ls-remote file:///C:\APACHE\isis-git-rw"
[INFO] Working directory: C:\Users\ADMINI~1\AppData\Local\Temp
[INFO] Executing: cmd.exe /X /C "git fetch file:///C:\APACHE\isis-git-rw"
[INFO] Working directory: C:\APACHE\isis-git-rw\core\target\checkout
[INFO] Executing: cmd.exe /X /C "git checkout release-1.12.0"
[INFO] Working directory: C:\APACHE\isis-git-rw\core\target\checkout
[INFO] Executing: cmd.exe /X /C "git ls-files"
[INFO] Working directory: C:\APACHE\isis-git-rw\core\target\checkout
[INFO] Invoking perform goals in directory C:\APACHE\isis-git-rw\core\target\checkout\core
[INFO] Executing goals 'deploy'...
...

You may (again) be prompted for gpg passphrase. All being well this command will complete successfully. Given that it is uploading code artifacts, it could take a while to complete.

Releasing the Archetype

Apache Isis archetypes are reverse engineered from example applications. Once reverse engineered, the source is checked into git (replacing any earlier version of the archetype) and released.

Switch to the directory containing the simpleapp example:

cd ../example/application/simpleapp

Setup environment variables

Set additional environment variables for the simpleapp-archetype artefact:

export ISISART=simpleapp-archetype
export ISISPAR=$ISISREL                 (1)

export ISISCPT=$(echo $ISISART | cut -d- -f2)
export ISISCPN=$(echo $ISISART | cut -d- -f1)

env | grep ISIS | sort
1 $ISISPAR is the version of the Apache Isis core that will act as the archetype’s parent. Usually this is the same as $ISISREL.

Check the example app

Update the parent pom.xml to reference the released version of Apache Isis core, eg:

<properties>
    <isis.version>1.12.0</isis.version>
    ...
</properties>

Check for and fix any missing license header notices:

mvn org.apache.rat:apache-rat-plugin:check -D rat.numUnapprovedLicenses=50 -o && \
for a in `/bin/find . -name rat.txt -print`; do grep '!???' $a; done || \
for a in `/bin/find . -name rat.txt -print`; do grep '!AL' $a; done

Finally, double check that the app is running satisfactorily:

Check the about page and confirm built against non-SNAPSHOT versions of the Apache Isis jars.

Create the archetype

Make sure you are in the correct directory and environment variables are correct.

To recreate the simpleapp archetype:

cd example/application/simpleapp
env | grep ISIS | sort

Then, run the script:

sh ../../../scripts/recreate-archetype.sh $ISISJIRA

The script automatically commits changes; if you wish use git log and git diff (or a tool such as SourceTree) to review changes made.

Release prepare

Switch to the archetype directory and execute the release:prepare:

cd ../../../example/archetype/$ISISCPN
rm -rf $ISISTMP/checkout
mvn release:prepare -P apache-release \
                -DreleaseVersion=$ISISREL \
                -DdevelopmentVersion=$ISISDEV \
                -Dtag=$ISISART-$ISISREL

Post-prepare sanity check

This is a good point to test the archetype; nothing has yet been uploaded.

In a different session, create a new app from the archetype. First set up environment variables:

export ISISTMP=/c/tmp    # or as required
export ISISCPN=simpleapp
env | grep ISIS | sort

Then generate a new app from the archetype:

rm -rf $ISISTMP/test-$ISISCPN

mkdir $ISISTMP/test-$ISISCPN
cd $ISISTMP/test-$ISISCPN
mvn archetype:generate  \
    -D archetypeCatalog=local \
    -D groupId=com.mycompany \
    -D artifactId=myapp \
    -D archetypeGroupId=org.apache.isis.archetype \
    -D archetypeArtifactId=$ISISCPN-archetype

Build the newly generated app and test:

cd myapp
mvn clean install
mvn antrun:run -P self-host    # runs as standalone app using webconsole
cd webapp
mvn jetty:run                  # runs as mvn jetty plugin

Release Perform (upload)

Back in the original session (in the archetype directory, example/archetype/$ISISCPN), execute release:perform:

mvn release:perform -P apache-release \
                -DworkingDirectory=$ISISTMP/checkout

This will upload the artifacts to the ASF Nexus repository.

Check/Close Staging Repo

The mvn release:perform commands will have put release artifacts for both core and the simpleapp archetype into a newly created staging repository on the ASF Nexus repository server.

Log onto repository.apache.org (using your ASF LDAP account):

nexus staging 0

And then check that the release has been staged (select staging repositories from left-hand side):

nexus staging 1

If nothing appears in a staging repo you should stop here and work out why.

Assuming that the repo has been populated, make a note of its repo id; this is needed for the voting thread. In the screenshot above the id is org.apache.isis-008.

After checking that the staging repository contains the artifacts that you expect you should close the staging repository. This will make it available so that people can check the release.

Press the Close button and complete the dialog:

nexus staging 2

Nexus should start the process of closing the repository.

nexus staging 2a

All being well, the close should (eventually) complete successfully (keep hitting refresh):

nexus staging 3

The Nexus repository manager will also email you with confirmation of a successful close.

If Nexus has problems with the key signature, however, then the close will be aborted:

nexus staging 4

Use gpg --keyserver hkp://pgp.mit.edu --recv-keys nnnnnnnn to confirm that the key is available.

Unfortunately, Nexus does not seem to allow subkeys to be used for signing. See Key Generation for more details.

Push branches

Push the release branch to origin:

git push -u origin $ISISBRANCH

and also push tags for both core and the archetype:

git push origin refs/tags/isis-$ISISREL:refs/tags/isis-$ISISREL-$ISISRC
git push origin refs/tags/simpleapp-archetype-$ISISREL:refs/tags/simpleapp-archetype-$ISISREL-$ISISRC
git fetch

The remote tag isn’t visible locally but can be seen online.

Voting

Once the artifacts have been uploaded, you can call a vote.

In all cases, votes last for 72 hours and require a +3 (binding) vote from members.

Start voting thread on dev@isis.apache.org

The following boilerplate is for a release of the Apache Isis Core. Adapt as required:

Use the following subject, eg:

[VOTE] Apache Isis Core release 1.12.0 RC1

And use the following body:

I've cut a release for Apache Isis Core and the simpleapp archetype:

* Core 1.12.0
* SimpleApp Archetype 1.12.0

The source code artifacts have been uploaded to staging repositories on repository.apache.org:

* http://repository.apache.org/content/repositories/orgapacheisis-10xx/org/apache/isis/core/isis/1.12.0/isis-1.12.0-source-release.zip
* http://repository.apache.org/content/repositories/orgapacheisis-10xx/org/apache/isis/archetype/simpleapp-archetype/1.12.0/simpleapp-archetype-1.12.0-source-release.zip

For each zip there is a corresponding signature file (append .asc to the zip's url).

In the source code repo the code has been tagged as isis-1.12.0-RC1 and simpleapp-archetype-1.12.0-RC1; see https://git-wip-us.apache.org/repos/asf?p=isis.git

For instructions on how to verify the release (build from binaries and/or use in Maven directly), see http://isis.apache.org/guides/cgcom.html#_cgcom_verifying-releases

Please verify the release and cast your vote.  The vote will be open for a minimum of 72 hours.

[ ] +1
[ ]  0
[ ] -1

Remember to update:

  • the version number (1.12.0 or whatever)

  • the release candidate number (RC1 or whatever)

  • the repository id, as provided by Nexus earlier (orgapacheisis-10xx or whatever)

Note that the email also references the procedure for other committers to verify the release.

Verifying a Release

The release process consists of:

  • the release manager cutting the release

  • members of the Apache Isis PMC verifying and voting on the release (documented below)

  • the release manager performing post-release tasks, for either a successful or an unsuccessful vote.

This section describes some guidance on what a voter (members of the Apache Isis PMC and anyone else who wishes) is expected to do before casting their vote in order to verify a release.

Background

Whenever a release manager announces a vote on a release (as per the release process) on the dev mailing list, it is the responsibility of the project’s PMC to cast their vote on the release. Anyone else can also vote, but only members of the Apache Isis PMC’s vote are binding.

Per this ASF documentation, the legal requirements for an ASF release are:

  • a source zip file + corresponding signature (signed by the release manager, which is in the ASF web of trust and in our KEYS file)

  • all source files have the Apache license (this is ensured by the running of the rat plugin prior to release; you could run it on the unzipped source)

  • all dependencies are appropriately licensed; see the DEPENDENCIES file which is automatically generated from the POMs plus the supplemental-models.xml file

Note that the binaries are not an ASF release, they merely exist on the Maven central repo as a convenience. That said, you might also want to verify the release by pulling the binaries from the Maven staging repository. Details of how to do this are also documented below.

Prerequisites

To verify the source ZIP files, you will need to have imported the public keys used for signing Apache Isis releases. These can be downloaded from the root of the Apache Isis source tree.

Since the Apache Isis source is mirrored on github.com, you can just use:

curl http://www.apache.org/dist/isis/KEYS > /tmp/KEYS
gpg --import /tmp/KEYS

Also, we will be rebuilding Isis from source. Therefore delete all Isis artifacts from your local Maven repo:

rm -rf ~/.m2/repository/org/apache/isis

Verifying source artifacts

You can either verify the source artifacts manuall, or use a script that automates the steps.

Manual procedure

The following section describes the steps to perform to manually verify a release.

Download the artifacts

Download both the ZIP and .ASC files from the location specified in the voting email. To verify that the signature is correct, use:

gpg --verify isis-x.y.z.zip.asc isis-x.y.z.zip
Building source artifacts

Assuming the ZIP file verifies, it should be unpacked, and then the artifact built from source.

To build Apache Isis core, first download any dependencies:

mvn dependency:go-offline

Check that no Isis artifacts have yet been downloaded, ie there is no ~/.m2/org/repository/org/apache/isis directory. If there are, it could indicate that the release being verified incorrectly references previous versions of Apache Isis

Assuming all is ok, build using the -o offline flag:

mvn clean install -o

Confirm that the versions of the Isis artifacts now cached in your local repository are correct.

Verifying binary artifacts

You can verify the binary releases by configuring your local Maven install to point to the Maven staging repository (or repositories) and then using them, eg to run the simpleapp archetype and running the resultant app.

Configuring your local Maven install amounts to updating the ~/.m2/settings.xml file:

<profiles>
    <profile>
        <id>verify-isis</id>
        <repositories>
            <repository>
                <id>isis-core-staging</id>
                <name>Isis Core Staging</name>
                <releases>
                    <enabled>true</enabled>
                    <updatePolicy>always</updatePolicy>
                    <checksumPolicy>warn</checksumPolicy>
                </releases>
                <url>http://repository.apache.org/content/repositories/orgapacheisis-10xx</url>
                <layout>default</layout>
            </repository>
            ...
        </repositories>
    </profile>
    ...
</profiles>
<activeProfiles>
    <activeProfile>verify-isis</activeProfile>
    ...
</activeProfiles>

where the repository URL is as provided in the VOTE email. If there is more than one repository (as is sometimes the case if multiple components have been released), then repeat the <repository> section for each.

Once the vote has completed, the staging repositories will be removed and so you should deactive the profile (comment out the <activeProfile> element). If you forget to deactive the profile, there should be no adverse effects; Maven will just spend unnecessary cycles attempting to hit a non-existent repo.

Automated procedure

To save some time in verifying an Apache Isis release we’ve assembled a script to automate the process. The script is tested on Mac OSX and Linux. Windows users can use Cygwin or msysgit.

It’s recommended that you start this process in an empty directory:

mkdir ~/verify-isis-release
cd ~/verify-isis-release
Copy script to local machine

Copy the following script, save to verify-isis-release.sh:

#!/bin/bash
# Instructions:
# -Create an empty directory
# -Put a .txt file in it containing a list of all the urls of the zip files
# -Run this script
# TODO: enhance this script so it will stop when something is broken
_download(){
    for fil in `cat *.txt`
    do
       echo 'Downloading '$fil
       curl  -L -O $fil
       curl  -L -O $fil.asc
    done
}
_verify(){
    for zip in *.zip
    do
       echo 'Verifying '$zip
       gpg --verify $zip.asc $zip
    done
}
_unpack(){
    echo 'Unpacking '
    unzip -q '*.zip'
}
_build(){
    echo 'Removing Apache Isis from local repo '$module
    rm -rf ~/.m2/repository/org/apache/isis
    COUNTER=0
    for module in ./*/
    do
       COUNTER=$[COUNTER+1]
       if [ $COUNTER -eq 1 ]
       then
         cd $module
         echo 'Building Core '$module
         mvn clean install -o
         cd ..
       else
         cd $module
         echo 'Building Module '$module
         mvn clean install
         cd ..
       fi
    done
}
# The work starts here
_download
_verify
_unpack
_build

Make sure the script is executable:

chmod +x verify-isis-release.sh

The script could be enhanced in many ways, feel free to contribute improvements!

Create an input file

The input file is a plain .txt file containing all urls to the packages to be verified. Here’s a sample of the release of Apache Isis 1.8.0:

https://repository.apache.org/content/repositories/orgapacheisis-063/org/apache/isis/core/isis/1.8.0/isis-1.8.0-source-release.zip
https://repository.apache.org/content/repositories/orgapacheisis-065/org/apache/isis/archetype/simpleapp-archetype/1.8.0/simpleapp-archetype-1.8.0-source-release.zip

The actual list of packages to be verified will be provided through the mailing list.

Execute the script

Execute…​

./verify-isis-release.sh

... and get yourself a cup of coffee.

(Optional) Creadur Tools

The Apache Creadur project exists to provide a set of tools to ensure compliance with Apache’s licensing standards.

The main release auditing tool, Apache RAT is used in the release process.

Creadur’s remaining tools - Tentacles and Whisker - are to support the verification process.

For example, Tentacles generates a report called archives.html. This lists all of the top-level binaires, their LICENSE and NOTICE files and any LICENSE and NOTICE files of any binaries they may contain.

Validation of the output at this point is all still manual. Things to check include:

  • any binaries that contain no LICENSE and NOTICE files

  • any binaries that contain more than one LICENSE or NOTICE file

In this report, each binary will have three links listed after its name '(licenses, notices, contents)'

Test the archetype

Assuming that everything builds ok, then test the archetypes (adjust version as necessary):

mvn archetype:generate  \
    -D archetypeGroupId=org.apache.isis.archetype \
    -D archetypeArtifactId=simpleapp-archetype \
    -D groupId=com.mycompany \
    -D artifactId=myapp \
    -D version=1.0-SNAPSHOT \
    -B \
    -o \
    -D archetypeVersion=1.12.0   # adjust version as necessary

cd myapp
mvn clean install -o
mvn -P self-host antrun:run

If it runs up ok, then it’s time to vote!

Casting a Vote

When you have made the above checks (and any other checks you think may be relevant), cast your vote by replying to the email thread on the mailing list.

If you are casting -1, please provide details of the problem(s) you have found.

Post Release (Successful)

The release process consists of:

  • the release manager cutting the release

  • members of the Apache Isis PMC verifying and voting on the release

  • the release manager performing post-release tasks, for either a successful or an unsuccessful vote (former documented below)

For a vote to succeed, there must be +3 votes from PMC members, and the vote must have been open at least 72 hours. If there are not +3 votes after this time then it is perfectly permissible to keep the vote open longer.

This section describes the steps to perform if the vote has been successful.

Inform dev ML

Post the results to the dev@isis.a.o mailing list:

[RESULT] [VOTE] Apache Isis Core release 1.12.0

using the body (alter last line as appropriate):

The vote has completed with the following result :

  +1 (binding): <i>list of names</i>
  +1 (non binding): <i>list of names</i>

  -1 (binding): <i>list of names</i>
  -1 (non binding): <i>list of names</i>

The vote is SUCCESSFUL.

Update tags

Replace the -RCn tag with another without the qualifier.

You can do this using the scripts/promoterctag.sh script; for example:

sh scripts/promoterctag.sh isis-1.12.0 RC1
sh scripts/promoterctag.sh simpleapp-archetype-1.12.0 RC1

Then, continue onto the next section for the steps to promote and announce the release.

Release to Maven Central

From the ASF Nexus repository, select the staging repository and select 'release' from the top menu.

nexus release 1

This moves the release artifacts into an Apache releases repository; from there they will be automatically moved to the Maven repository.

Release Source Zip

As described in the Apache documentation, each Apache TLP has a release/TLP-name directory in the distribution Subversion repository at https://dist.apache.org/repos/dist. Once a release vote passes, the release manager should svn add the artifacts (plus signature and hash files) into this location. The release is then automatically pushed to http://www.apache.org/dist/ by svnpubsub. Only the most recent release of each supported release line should be contained here, old versions should be deleted.

Each project is responsible for the structure of its directory. The directory structure of Apache Isis reflects the directory structure in our git source code repo:

isis/
  core/
  example/
    archetype/
      simpleapp/

If necessary, checkout this directory structure:

svn co https://dist.apache.org/repos/dist/release/isis isis-dist

Next, add the new release into the appropriate directory, and delete any previous release. The upd.sh script can be used to automate this:

old_ver=$1
new_ver=$2


# constants
repo_root=https://repository.apache.org/content/repositories/releases/org/apache/isis

zip="source-release.zip"
asc="$zip.asc"
md5="$zip.md5"


#
# isis-core
#
type="core"
fullname="isis"
pushd isis-core

curl -O $repo_root/$type/$fullname/$new_ver/$fullname-$new_ver-$asc
svn add $fullname-$new_ver-$asc
curl -O $repo_root/$type/$fullname/$new_ver/$fullname-$new_ver-$md5
svn add $fullname-$new_ver-$md5
curl -O $repo_root/$type/$fullname/$new_ver/$fullname-$new_ver-$zip
svn add $fullname-$new_ver-$zip

svn delete $fullname-$old_ver-$asc
svn delete $fullname-$old_ver-$md5
svn delete $fullname-$old_ver-$zip

popd


#
# simpleapp-archetype
#
type="archetype"
fullname="simpleapp-archetype"
pushd $type/$fullname

curl -O $repo_root/$type/$fullname/$new_ver/$fullname-$new_ver-$md5
svn add $fullname-$new_ver-$md5
curl -O $repo_root/$type/$fullname/$new_ver/$fullname-$new_ver-$asc
svn add $fullname-$new_ver-$asc
curl -O $repo_root/$type/$fullname/$new_ver/$fullname-$new_ver-$zip
svn add $fullname-$new_ver-$zip

svn delete $fullname-$old_ver-$md5
svn delete $fullname-$old_ver-$asc
svn delete $fullname-$old_ver-$zip

popd

For example:

sh upd.sh 1.11.0 1.12.0

The script downloads the artefacts from the Nexus release repository, adds the artefacts to subsversion and deletes the previous version.

At the end, commit the changes:

svn commit -m "publishing isis source releases to dist.apache.org"

Update JIRA

Generate Release Notes

From the root directory, generate the release notes for the current release, in Asciidoc format; eg:

sh scripts/jira-release-notes.sh ISIS 1.12.0 > /tmp/1

Close tickets

Close all JIRA tickets for the release, or moved to future releases if not yet addressed. Any tickets that were partially implemented should be closed, and new tickets created for the functionality on the ticket not yet implemented.

Mark the version as released

In JIRA, go to the administration section for the Apache Isis project and update the version as being released.

In the Kanban view this will have the effect of marking all tickets as released (clearing the "done" column).

Create new JIRA

Create a new JIRA ticket as a catch-all for the next release.

Update the ASF Reporter website

Log the new release in the ASF Reporter website.

Update website

Update the Apache Isis (asciidoc) website:

  • Paste in the JIRA-generated release notes generated above, adding to top of adocs/documentation/src/main/asciidoc/release-notes.adoc. Also add a summary line for the release.

  • Search for any -SNAPSHOT suffices, and remove

  • Search these release procedures, and update any hard-coded reference to the release to the next release (so when they are followed next time the text will be correct).

  • Update the downloads page with a link to the source release zip file (under https://dist.apache.org/repos/dist/release/isis)

  • Update any pages (.adoc, .md, .html etc) that describe how to run the archetype, and ensure they reference the correct version.

    A search for archetypeGroupId=org.apache.isis.archetype should find these pages.

  • update the DOAP RDF file (which provides a machine-parseable description of the project) should also be updated with details of the new release. Validate using the W3C RDF Validator service.

    For more information on DOAP files, see these Apache policy docs.

  • Update the STATUS file (in root of Apache Isis' source) should be updated with details of the new release.

Don’t forget to commit the .adoc changes and publish to the isis-site repo.

Announce the release

Announce the release to users mailing list.

For example, for a release of Apache Isis Core, use the following subject:

[ANN] Apache Isis version 1.12.0 Released

And use the following body (summarizing the main points as required):

The Apache Isis team is pleased to announce the release of Apache Isis v1.12.0.

New features in this release include:
* ...

Full release notes are available on the Apache Isis website at [1].  Please also read the migration notes [2].

You can access this release directly from the Maven central repo [3], or download the release and build it from
source [4].

Enjoy!

--The Apache Isis team

[1] http://isis.apache.org/release-notes.html#r1.12.0
[2] http://isis.apache.org/migration-notes.html#_migration-notes_1.11.0-to-1.12.0
[3] http://search.maven.org
[4] http://isis.apache.org/downloads.html

Blog post

Log onto the Apache blog and create a new post. Copy-n-paste the above mailing list announcement should suffice.

Merge in release branch

Because we release from a branch, the changes made in the branch (changes to pom.xml made by the maven-release-plugin, or any manual edits) should be merged back from the release branch back into the master branch:

git checkout master                           # update master with latest
git pull
git merge release-1.12.0-RC1                  # merge branch onto master
git branch -d release-1.12.0-RC1              # branch no longer needed
git push origin --delete release-1.12.0-RC1   # remote branch no longer needed

Finally, update the simpleapp’s root pom.xml to reference the next SNAPSHOT release (1.12.0-SNAPSHOT)

Update dependencies

With the release complete, now is a good time to bump versions of dependencies (so that there is a full release cycle to identify any possible issues).

You will probably want to create a new JIRA ticket for these updates (or if minor then use the "catch-all" JIRA ticket raised earlier for the next release).

Update parent of Core

Check (via search.maven.org) whether there is a newer version of the Apache parent org.apache:apache.

If there is, update the <version> in the <parent> element in the parent POM to match the newer version:

<parent>
    <groupId>org.apache</groupId>
    <artifactId>apache</artifactId>
    <version>NN</version>
    <relativePath />
</parent>

where NN is the updated version number.

Update plugin versions

The maven-versions-plugin should be used to determine if there are newer versions of any of the plugins used to build Apache Isis. Since this goes off to the internet, it may take a minute or two to run:

mvn versions:display-plugin-updates > /tmp/foo
grep "\->" /tmp/foo | /bin/sort -u

Review the generated output and make updates as you see fit. (However, if updating, please check by searching for known issues with newer versions).

Update dependency versions

The maven-versions-plugin should be used to determine if there are newer versions of any of Isis' dependencies. Since this goes off to the internet, it may take a minute or two to run:

mvn versions:display-dependency-updates > /tmp/foo
grep "\->" /tmp/foo | /bin/sort -u

Update any of the dependencies that are out-of-date. That said, do note that some dependencies may show up with a new dependency, when in fact the dependency is for an old, badly named version. Also, there may be new dependencies that you do not wish to move to, eg release candidates or milestones.

For example, here is a report showing both of these cases:

[INFO]   asm:asm ..................................... 3.3.1 -> 20041228.180559
[INFO]   commons-httpclient:commons-httpclient .......... 3.1 -> 3.1-jbossorg-1
[INFO]   commons-logging:commons-logging ......... 1.1.1 -> 99.0-does-not-exist
[INFO]   dom4j:dom4j ................................. 1.6.1 -> 20040902.021138
[INFO]   org.datanucleus:datanucleus-api-jdo ................ 3.1.2 -> 3.2.0-m1
[INFO]   org.datanucleus:datanucleus-core ................... 3.1.2 -> 3.2.0-m1
[INFO]   org.datanucleus:datanucleus-jodatime ............... 3.1.1 -> 3.2.0-m1
[INFO]   org.datanucleus:datanucleus-rdbms .................. 3.1.2 -> 3.2.0-m1
[INFO]   org.easymock:easymock ................................... 2.5.2 -> 3.1
[INFO]   org.jboss.resteasy:resteasy-jaxrs ............. 2.3.1.GA -> 3.0-beta-1

For these artifacts you will need to search Maven central repo directly yourself to confirm there are no newer dependencies not shown in this list.

Code formatting

This is also a good time to make source code has been cleaned up and formatted according to the Apache Isis and ASF conventions. Use this Eclipse template and this import order.

Push changes

Finally, push the changes up to origin:

git fetch    # check no new commits on origin/master
git push

Release Isis Addons

Once the Apache Isis release is available, all of the (non-ASF) Isis Addons should also be released.

Using this gist to invoke operations across all (or selected) addons:

  • update its dependency on Apache Isis to reference the newly released version:

    sh forsub.sh sh bumpver_isis.sh 1.12.0
  • update the README for each repository

    • in the "How to Configure/Use" section, the version referenced in the pom.xml, and for the -SNAPSHOT

    • the "Change Log" section

    • the "Release to Maven Central" section at the end

  • release to mvn central (contains a sanity check before hand that everything compiles):

    sh forsub.sh sh release.sh "1.12.0" "1.13.0-SNAPSHOT" "dan@haywood-associates.co.uk" \"this is not really my password\"
  • update its dependency on Apache Isis to reference the next SNAPSHOT version:

    sh forsub.sh sh bumpver_isis.sh "1.13.0-SNAPSHOT"

Post Release (Unsuccessful)

The release process consists of:

  • the release manager cutting the release

  • members of the Apache Isis PMC verifying and voting on the release

  • the release manager performing post-release tasks, for either a successful or an unsuccessful vote (latter documented below).

If the vote did not succeed (did not achieve +3 votes after 72 hours and/or is unlikely to do so), then the vote should be closed and the following steps performed.

Note that a release manager may also decide to cancel a vote before 72 hours has elapsed (for example if an error is quickly discovered).

Inform dev ML

Post the results to the dev@isis.a.o mailing list.

For example, use the following subject for a vote on Apache Isis Core:

[RESULT] [VOTE] Apache Isis Core release 1.12.0

using the body (alter last line as appropriate):

The vote has completed with the following result :

  +1 (binding): _list of names_
  +1 (non binding): _list of names_

  -1 (binding): _list of names_
  -1 (non binding): _list of names_

The vote is UNSUCCESSFUL.

Tidy up branches

Tidy up remote branches in the git repo:

  • delete the remote branch, for example:

    git push --delete origin release-1.12.0-RC1
  • delete the remote origin server’s tags, for example:

    git push --delete origin isis-1.12.0-RC1
    git push --delete origin simpleapp-archetype-1.12.0-RC1
  • delete the tags that were created locally, for example:

    git tag -d isis-1.12.0
    git tag -d isis-1.12.0-RC1
    git tag -d simpleapp-archetype-1.12.0
    git tag -d simpleapp-archetype-1.12.0-RC1

Tidy up the Nexus repo

Drop staging repositories:

  • drop the staging repository in Nexus

Reset

Finally, rewind the release branch to prior to the previous release candidate, and continue from there.

Snapshot Releases

Apache Isis consists of a number of separately releasable modules; see the main release process documentation for full details. All the non-core components depend on the core, and use the core’s parent `pom.xml as their parent pom.

Unless otherwise stated, you should assume that the steps described here are performed in the base directory of the module being released.

Prerequisites

Before you start, make sure you’ve defined the snapshots repo in your local ~/.m2/settings.xml file:

<settings>
  <servers>
    <!-- To publish a snapshot of some part of Maven -->
    <server>
      <id>apache.snapshots.https</id>
      <username>xxxxxxx</username>
      <password>yyyyyyy</password>
    </server>
    ...
  </servers>
  ...
</settings>

where xxxxxxx and yyyyyyy are your Apache LDAP username and password. For more information, see these ASF docs.

{note It is also possible to configure to use .ssh secure keys, and thereby avoid hardcoding your Apache LDAP password into your .m2/settings.xml file. A description of how to do this can be found, for example, here. }

Sanity Check

Before deploying the snapshot, perform a quick sanity check.

First, delete all Isis artifacts from your local Maven repo:

rm -rf ~/.m2/repository/org/apache/isis

Next, check that the framework builds ok:

cd core
mvn clean install -o

Confirm that the versions of the Isis artifacts now cached in your local repository are correct (both those pulled down from Maven central repo, as well as those of the component built locally).

Deploy

Deploy the framework using:

cd core
mvn -D deploy=snapshot deploy

This will deploy all the modules that make up a release.

Expect this to take about 10 minutes, give or take.

To confirm that they are present, browse to Apache’s Nexus repository manager and search for "isis".

Key Generation

In order that a contributor can make a release it is necessary for them to have generated a key and had that key recognized by other members of the Apache Software Foundation.

For further background information on this topic, see the release signing page and the openpgp page on the Apache wiki.

Install and Configure gpg

Download and install GnuPG (gpg), version 1.4.10 or higher.

Then, edit ~/.gnupg/gpg.conf (on Windows, the file to edit is C:\Users\xxx\AppData\Roaming\gnupg\gpg.conf) so that the default is to generate a strong key:

personal-digest-preferences SHA512
cert-digest-algo SHA512
default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed

Key Generation

The Apache Software Foundation requires that keys are signed with a key (or subkey) based on RSA 4096 bits. To do this:

$ gpg --gen-key
gpg (GnuPG) 1.4.11; Copyright (C) 2010 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection?

Specify RSA key:

Your selection? 1

RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048)

Specify key length as 4096 bits:

What keysize do you want? (2048) 4096
Requested keysize is 4096 bits

Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0)

Specify key as non-expiring:

Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y

You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
    "Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"

Real name:

Enter your name, email and comment:

  • use your apache.org email

  • the comment should be "CODE SIGNING KEY"

Real name: Xxx Xxxxxxxxx Email address: xxx@apache.org Comment: CODE SIGNING KEY You selected this USER-ID: "Xxx Xxxxxxxxx (CODE SIGNING KEY) xxx@apache.org"

Change (N)ame, ©omment, (E)mail or (O)kay/(Q)uit? O

You need a Passphrase to protect your secret key. Enter passphrase:

Provide a passphrase to secure your key.

Enter passphrase:
Repeat passphrase:

GPG will goes on to generate your key:

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
...+++++
.........................+++++
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
....+++++
...+++++
gpg: key nnnnnnnn marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
pub   4096R/nnnnnnnn yyyy-mm-dd
      Key fingerprint = xxxx xxxx xxxx xxxx xxxx  xxxx xxxx xxxx xxxx xxxx
uid                  Xxx Xxxxxx <xxx@apache.org>
sub   4096R/kkkkkkkk yyyy-mm-dd

The public key with id nnnnnnnn should now be stored in ~/.gnupg/pubring.pgp (on Windows 7, this is in c:/Users/xxx/AppData/Roaming/gnupg/pubring.pgp).

To confirm the key has been generated, use:

$ gpg --list-keys --fingerprint

The key Id is the one true way to identify the key, and is also the last 8 digits of the fingerprint. The corresponding secret key for id nnnnnnnn is stored in ~/.gnupg/secring.pgp (on Windows 7, this is in c:/Users/xxx/AppData/Roaming/gnupg/secring.pgp).

It’s also worth confirming the key has the correct preference of algorithms (reflecting the initial configuration we did earlier). For this, enter the gpg shell for your new key:

$ gpg --edit-key nnnnnnnnn
>gpg

where nnnnnnnn is your key id. Now, use the 'showpref' subcommand to list details:

gpg> showpref
[ultimate] (1). Xxx Xxxxxxxx (CODE SIGNING KEY) <xxx@apache.org>
     Cipher: AES256, AES192, AES, CAST5, 3DES
     Digest: SHA512, SHA384, SHA256, SHA224, SHA1
     Compression: ZLIB, BZIP2, ZIP, Uncompressed
     Features: MDC, Keyserver no-modify

gpg>

The Digest line should list SHA-512 first and SHA-1 last. If it doesn’t, use the "setpref" command:

setpref SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed

Finally, remember to take a backup of your key and the keyring (ie, backup the .gnupg directory and its contents).

Subkey Generation

It’s recommended to use a subkey with an expiry date to sign releases, rather than your main, non-expiring key. If a subkey is present, then gpg will use it for signing in preference to the main key.

After (binary) release artifacts are created, they are deployed to the ASF’s Nexus staging repository. However, Nexus seems unable to retrieve a subkey from the public key server. Until we find a fix/workaround for this, all releases should be signed just with a regular non-expiring main key.

To create a subkey Enter the gpg shell using (the identifier of) your main key:

gpg --edit-key xxxxxxxx
gpg>

Type 'addkey' to create a subkey, and enter your passphrase for the main key:

gpg> addkey
Key is protected.
[enter your secret passphrase]

You need a passphrase to unlock the secret key for
user: "Dan Haywood (CODE SIGNING KEY) <danhaywood@apache.org>"
4096-bit RSA key, ID xxxxxxxx, created 2011-02-01

Please select what kind of key you want:
   (3) DSA (sign only)
   (4) RSA (sign only)
   (5) Elgamal (encrypt only)
   (6) RSA (encrypt only)
Your selection?

Select (6) to choose an RSA key for encryption:

It would seem that Nexus repository manager does not recognize RSA subkeys with an 'S’ign usage; see this discussion on a mailing list and this issue on Sonatype’s JIRA

Your selection? 6

RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096

Requested keysize is 4096 bits

Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for?

Specify that the key is valid for 1 year:

Key is valid for? (0) 1y

Key expires at yy/MM/dd hh:mm:ss
Is this correct? (y/N) y
Really create? (y/N) y
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
...+++++
.+++++

pub  4096R/xxxxxxxx  created: yyyy-mm-dd  expires: never       usage: SC
                     trust: ultimate      validity: ultimate
sub  4096R/xxxxxxxx  created: yyyy-mm-dd  expires: yyYY-mm-dd  usage: E
[ultimate] (1). Dan Haywood (CODE SIGNING KEY) <danhaywood@apache.org>

gpg>

Quit the gpg shell; you now have a subkey.

Generate a Revocation Certificate

It’s good practice to generate a number of revocation certificates so that the key can be revoked if it happens to be compromised. See the gpg page on the Apache wiki for more background on this topic.

First, generate a "no reason specified" key:

$ gpg --output revoke-nnnnnnnn-0.asc --armor --gen-revoke nnnnnnnn

sec  4096R/nnnnnnnn yyyy-mm-dd Xxx Xxxxxxx (CODE SIGNING KEY) <xx@apache.org>
Create a revocation certificate for this key? (y/N) Y

Please select the reason for the revocation:
  0 = No reason specified
  1 = Key has been compromised
  2 = Key is superseded
  3 = Key is no longer used
  Q = Cancel
(Probably you want to select 1 here)
Your decision?

Select 0.

Your decision? 0

Enter an optional description; end it with an empty line:

Provide a description:

> Generic certificate to revoke key, generated at time of key creation.
>
Reason for revocation: No reason specified
Generic certificate to revoke key, generated at time of key creation.
Is this okay? (y/N)

Confirm this is ok.

Is this okay? y

You need a passphrase to unlock the secret key for
user: "Xxx Xxxxxxx (CODE SIGNING KEY) <xxx@apache.org>"
4096-bit RSA key, ID nnnnnnnn, created yyyy-mm-dd

Enter passphrase:
</pre>

Enter a passphrase:

<pre>
Enter passphrase:
Revocation certificate created.

Please move it to a medium which you can hide away; if Mallory gets
access to this certificate he can use it to make your key unusable.
It is smart to print this certificate and store it away, just in case
your media become unreadable.  But have some caution:  The print system of
your machine might store the data and make it available to others!

The file revoke-nnnnnnnn-0.asc should be created: Then, backup this file.

Now repeat the process to create two further revocation certificates:

gpg --output revoke-nnnnnnnn-1.asc --armor --gen-revoke nnnnnnnn

Specify reason as "1 = Key has been compromised"

and:

gpg --output revoke-nnnnnnnn-3.asc --armor --gen-revoke nnnnnnnn

Specify reason as "3 = Key is no longer used"

Backup these files also.

Publish Key

It is also necessary to publish your key. There are several places where this should be done. In most cases, you’ll need the "armored" " (ie ASCII) representation of your key. This can be generated using:

$ gpg --armor --export nnnnnnnn > nnnnnnnn.asc

where nnnnnnnn is the id of your public key.

You’ll also need the fingerprint of your key. This can be generated using:

$ gpg --fingerprint nnnnnnnn

The output from this command includes a line beginning "Key fingerprint", followed by a (space delimited) 40 character hexadecimal fingerprint. The last 8 characters should be the same as the key id (nnnnnnnn).

Publish to a public key server

To a publish your key to a public key server (eg the MIT key server hosted at http://pgp.mit.edu), use the procedure below. Public key servers synchronize with each other, so publishing to one key server should be sufficient. For background reading on this, see the release signing page on the Apache wiki, and the gpg key page on the Maven wiki.

To send the key up to the key server:

$ gpg --send-keys --keyserver pgp.mit.edu nnnnnnnn

where nnnnnnnn is the key Id.

Alternatively, you can browse to the MIT key server and paste in the armored representation of your key.

Confirm the key has been added by browsing to submitting the following URL:

again, where nnnnnnnn is the key Id.

Publish to your Apache home directory

The armored representation of your public key should be uploaded to your home directory on people.apache.org, and renamed as .pgpkey. Make sure this is readable by all.

Publish to your Apache HTML home directory

The armored representation of your public key should be uploaded to your public_html home directory on people.apache.org, named nnnnnnnn.asc. Make sure this is readable by all.

Check the file is accessible by browsing to:

where

  • xxxxxxxx is your apache LDAP user name

  • nnnnnnnn is your public key id.

FOAF

First, check out the committers/info directory:

Go to Apache FOAF-a-matic web page to generate the FOAF file text (we copy this text out in a minute):

  • enter ASF LDAP user name

  • enter First name, Last name

  • for PGP key fingerprints, add Key

  • paste in the key id

  • paste in the fingerprint

  • press "Create"

In the box below, you should have a FOAF file, something like:

<?xml version="1.0" encoding="UTF-8"?>
<rdf:RDF
      xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
      xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
      xmlns:foaf="http://xmlns.com/foaf/0.1/"
      xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#"
      xmlns:pm="http://www.web-semantics.org/ns/pm#"
      xmlns:wot="http://xmlns.com/wot/0.1/"
      xmlns:rss="http://purl.org/rss/1.0/"
      xmlns:dc="http://purl.org/dc/elements/1.1/"
      xmlns:ical="http://www.w3.org/2002/12/cal/ical#"
      xmlns:doap="http://usefulinc.com/ns/doap#">
  <foaf:Person rdf:ID="danhaywood">
    <foaf:name>Xxx Xxxxxxxx</foaf:name>
    <foaf:givenname>Xxx</foaf:givenname>
    <foaf:family_name>Xxxxxxxx</foaf:family_name>
    <wot:hasKey>
      <wot:PubKey>
        <wot:fingerprint>nnnn nnnn nnnn nnnn nnnn  nnnn nnnn nnnn nnnn nnnn</wot:fingerprint>
        <wot:hex_id>nnnnnnnn</wot:hex_id>
      </wot:PubKey>
    </wot:hasKey>
  </foaf:Person>
</rdf:RDF>

(If you are creating the FOAF file for the first time, you may want to add additional details).

From this, copy out the wot:key, and paste into your FDF file in committers/info:

<wot:hasKey>
  <wot:PubKey>
    <wot:fingerprint>nnnn nnnn nnnn nnnn nnnn  nnnn nnnn nnnn nnnn nnnn</wot:fingerprint>
    <wot:hex_id>nnnnnnnn</wot:hex_id>
  </wot:PubKey>
</wot:hasKey>

Then, manually add in a <wot:pubkeyAddress> element within <wot:PubKey>:

<wot:hasKey>
  <wot:PubKey>
    <wot:fingerprint>nnnn nnnn nnnn nnnn nnnn  nnnn nnnn nnnn nnnn nnnn</wot:fingerprint>
    <wot:hex_id>nnnnnnnn</wot:hex_id>
    <wot:pubkeyAddress rdf:resource="http://people.apache.org/~username/nnnnnnnn.asc/">
  </wot:PubKey>
</wot:hasKey>

ie, referencing your publically exported public key

Finally, commit your changes.

Save to KEYS

The armored representation of the public key should be saved to Apache Isis' KEYS file, http://www.apache.org/dist/isis/KEYS (that is, in the ASF distribution directory for Apache Isis).

First, in a new directory, checkout this file:

svn -N co https://svn.apache.org/repos/asf/isis/ .

This should bring down the KEYS file.

Then, export your signature and armored representation.

gpg --list-sigs nnnnnnnn >>KEYS
gpg --armor --export nnnnnnnn >>KEYS

Then commit.

id.apache.org

Log onto id.apache.org and ensure that the finger print of your public key is correct.

Attend Key Signing Party (Apache web of trust)

It is strongly advised that the contributor attend a key signing party at an Apache event, in order that other Apache committers/members can in person verify their identity against the key. The process for this is described here and here.

Update Maven Settings file (~/.m2/settings.xml)

The Maven release plugin will automatically sign the release, however it is necessary to update the ~/.m2/settings.xml file with your GPG acronym passphrase in order that it can use your secret key. This is defined under a profile so that it is activated only when we perform a release (as defined by [org,apache:apache] parent POM.

Therefore, make the following edits:

<settings>
  ...
  <profiles>
    <profile>
      <id>apache-release</id>
      <properties>
        <gpg.passphrase>xxx xxx xxx xxx xxx xxx xxx</gpg.passphrase>
      </properties>
    </profile>
  </profiles>
</settings>

In addition, to allow the release plugin to tag SVN changes, you must either add in your LDAP username/password or configure .ssh:

<settings>
  ...
  <servers>
    ...
    <server>
      <id>apache.releases.https</id>
      <username>xxxx</username>
      <password>xxxx</password>
    </server>
  </servers>
</settings>

Appendix: Release Prereqs

This section (appendix) describes the prerequisites for the release process.

Public/private key

The most important configuration you require is to set up public/private key pair. This is used by the maven-release-plugin to sign the code artifacts. See the page on key generation for more details.

In order to prepare the release, you’ll (need to) have a ~/.gnupg directory with the relevant files (gpg.conf, pubring.gpg, secring.gpg etc), and have gpg on your operating system PATH.

If on Windows, the equivalent directory is c:\users\xxx\appdata\roaming\gnupg. For gpg, use either cygwin.com or gpg4win.org. Note also that the mSysGit version of gpg (as provided by GitHub’s bash client) is not compatible with that provided by cygwin; move it to one side and check that gpg.exe being used is that from gpg4win.

Maven settings.xml

During the release process the maven-deploy-plugin uploads the generated artifacts to a staging repo on the Apache repository manager. This requires your Apache LDAP credentials to be specified in your ~/.m2/settings.xml file:

<settings>
  <servers>
    <server>
      <id>apache.releases.https</id>
      <username>xxxxxxx</username>
      <password>yyyyyyy</password>
    </server>
    ...
  </servers>
  ...
</settings>

where xxxxxxx and yyyyyyy are your Apache LDAP username and password. For more information, see these ASF docs.

It is also possible to configure to use .ssh secure keys, and thereby avoid hardcoding your Apache LDAP password into your .m2/settings.xml file. A description of how to do this can be found, for example, here.

Also, set up keyphrase for gpg; this avoids being prompted during release:

<profiles>
  <profile>
    <id>gpg</id>
    <properties>
      <gpg.executable>gpg2</gpg.executable>
      <gpg.passphrase>this is not really my passphrase</gpg.passphrase>
    </properties>
  </profile>
  ...
</profiles>

<activeProfiles>
  <activeProfile>gpg</activeProfile>
  ...
</activeProfiles>

Appendix: PMC

Every ASF project has a Project Management Committee, or PMC. This committee is ultimately responsible for the long-term management of the framework. More information about PMCs can be found here

In Apache Isis, every committer is a member of the PMC.

This page contains some general notes on maintenance activities required by PMC members.

Accessing people.apache.org

Must be accessed via ssh.

eg:

ssh danhaywood@people.apache.org

and when prompted, provide passphrase for private key…​ though I’ve forgotten what I did to set this up in the first place, though :-(

LDAP Access (UNIX groups)

Whenever we get a new committer, the ASF LDAP entries must be maintained to grant access to our repos and various other 'karma'.

Log onto people.apache.org, then use:

list_unix_group.pl isis

to list committers

list_committee.pl isis

to list the PMC committee members (in Apache Isis, every committer should be on the PMC committee)

To change membership of either the committers or the PMC, use:

modify_unix_group.pl isis --add joebloggs
modify_unix_group.pl isis --remove joebloggs

and

modify_committee.pl gump --add joebloggs
modify_committee.pl gump --remove joebloggs

respectively.

Further details are in these ASF docs. (They talk about SVN access, but really it is LDAP access).


Copyright © 2010~2015 The Apache Software Foundation, licensed under the Apache License, v2.0.
Apache, the Apache feather logo, Apache Isis, and the Apache Isis project logo are all trademarks of The Apache Software Foundation.