BDD Spec Support

Behaviour-driven design (BDD) redefines testing not as an after-the-fact "let’s check the system works", but instead as a means to specify the required behaviour in conjunction with the domain expert. Once the feature has been implemented, it also provide a means by which the domain expert can verify the feature has been implemented to spec.

Since domain experts are usually non-technical (at least, they are unlikely to be able to read or want to learn how to read JUnit/Java code), then applying BDD typically requires writing specifications in using structured English text and (ASCII) tables. The BDD tooling parses this text and uses it to actually interact with the system under test. As a byproduct the BDD frameworks generate readable output of some form; this is often an annotated version of the original specification, marked up to indicate which specifications passed, which have failed. This readable output is a form of "living documentation"; it captures the actual behaviour of the system, and so is guaranteed to be accurate.

There are many BDD tools out there; Apache Isis provides an integration with Cucumber JVM (see also the github site):

How it works

At a high level, here’s how Cucumber works

  • specifications are written in the Gherkin DSL, following the "given/when/then" format.

  • Cucumber-JVM itself is a JUnit runner, and searches for feature files on the classpath.

  • These in turn are matched to step definitions through regular expressions.

    It is the step definitions (also called "glue") that exercise the system.

The code that goes in step definitions is broadly the same as the code that goes in an integration test method. However one benefit of using step definitions (rather than integration tests) is that the step definitions are reusable across scenarios, so there may be less code overall to maintain.

For example, if you have a step definition that maps to "given an uncompleted todo item", then this can be used for all the scenarios that start with that as their precondition.

Writing a BDD spec

BDD specifications contain:

  • a Xxx.feature file, describing the feature and the scenarios (given/when/then)s that constitute its acceptance criteria

  • a RunCucumberTest class file to run the specifications (all boilerplate). This will run all .feature files in the same package or subpackages, but is basically just boilerplate.

  • one or several XxxStepDef classes constituting the step definitions to be matched against.

    The step definitions are intended to be reused across features. We therefore recommend that they reside in a separate package, and are organized by the entity type upon which they act.

    For example, given a feature that involves Customer and Order, have the step definitions pertaining to Customer reside in CustomerStepDef, and the step definitions pertaining to Order reside in OrderStepDef.

The SimpleApp starter app provides some BDD specs, so we’ll use them to understand how this all works.

Bootstrapping

The RunCucumberSpecs class is annotated with the @Cucumber JUnit 5 platform test engine to discover the features and step defs (glue).

The class itself is trivial:

RunCucumberSpecs.java
package domainapp.webapp.bdd;

import io.cucumber.junit.platform.engine.Cucumber;

@Cucumber                                           (1)
public class RunCucumberSpecs {
}
1 The tests are run through JUnit 5, as a custom platform engine

The Cucumber engine is configured using JUnit 5’s standard mechanism, namely the junit-platform.properties file (in the root package):

junit-platform.properties
cucumber.filter.tags=not @backlog and not @ignore
cucumber.glue=domainapp.webapp.bdd.stepdefs

We also use two "infrastructure" step definitions to bootstrap and configure Spring. These are also boilerplate:

  • The BootstrapStepDef class actually starts the Spring application context:

    BootstrapStepDef.java
    package domainapp.webapp.bdd.stepdefs.infrastructure;       (1)
    ...
    
    public class BootstrapStepDef
                    extends ApplicationIntegTestAbstract {      (2)
    
        @Before(order= OrderPrecedence.FIRST)                   (3)
        public void bootstrap() {
            // empty                                            (4)
        }
    }
    1 in a subpackage of domainapp.webapp.bdd.stepdefs.
    See junit-platform.properties, above.
    2 subclasses from the corresponding integration tests, see integ test support for more on this.
    3 this @Before runs before anything else
    4 there’s not anything to do (the heavy lifting is in the superclass)
  • The TransactionalStepDef simulates Spring’s @Transactional attribute:

    TransactionalStepDef.java
    package domainapp.webapp.bdd.stepdefs.infrastructure;       (1)
    ...
    public class TransactionalStepDef {                         (2)
    
        private Runnable afterScenario;
    
        @Before(order = OrderPrecedence.EARLY)
        public void beforeScenario(){
            val txTemplate = new TransactionTemplate(txMan);    (3)
            val status = txTemplate.getTransactionManager().getTransaction(null);
            afterScenario = () -> {
                txTemplate.getTransactionManager().rollback(status);
            };
    
            status.flush();
        }
    
        @After
        public void afterScenario(){
            if(afterScenario==null) {
                return;
            }
            afterScenario.run();                                (4)
            afterScenario = null;
        }
    
        @Inject private PlatformTransactionManager txMan;       (5)
    }
    1 again, in a subpackage of the stepdefs package.
    2 no need to subclass anything
    3 uses Spring’s TransactionTemplate to wrap up the rest of the steps
    4 rolls back the transaction at the end.
    5 supporting services are automatically injected.

These two "infrastructure" step definitions could be combined into a single class, if desired.

Typical Usage

With the bootstrapping and infrastructure taken care of, let’s look at the actual spec and corresponding step defs.

SimpleObjectSpec_listAllAndCreate.feature
Feature: List and Create New Simple Objects                             (1)

  @DomainAppDemo                                                        (2)
  Scenario: Existing simple objects can be listed and new ones created  (1)
    Given there are initially 10 simple objects                         (3)
    When  I create a new simple object                                  (3)
    Then  there are 11 simple objects                                   (3)
1 Provide context, but not actually executed
2 Tag indicates the fixture to be run
3 Map onto step definitions

We need a step definition to match the Cucumber tag to a fixture script.

DomainAppDemoStepDef.java
package domainapp.webapp.bdd.stepdefs.fixtures;                      (1)
...

public class DomainAppDemoStepDef {

    @Before(value="@DomainAppDemo", order= OrderPrecedence.MIDPOINT) (2)
    public void runDomainAppDemo() {
        fixtureScripts.runFixtureScript(new DomainAppDemo(), null);  (3)
    }

    @Inject private FixtureScripts fixtureScripts;                   (4)
}
1 again, under the stepdefs package
2 specifies the tag to match
3 invokes the similarly named FixtureScript
4 The fixtureScripts service is injected automatically

This will only activate for feature files tagged with "@DomainAppDemo".

Finally, the step definitions pertaining to SimpleObjects domain service residein the SimpleObjectsSpecDef class. This is where the heavy lifting gets done:

package domainapp.webapp.bdd.stepdefs.domain;                           (1)
...
public class SimpleObjectsStepDef {

    @Inject protected SimpleObjects simpleObjects;                      (2)

    @Given("^there (?:is|are).* (\\d+) simple object[s]?$")             (3)
    public void there_are_N_simple_objects(int n) {
        final List<SimpleObject> list = wrap(simpleObjects).listAll();  (4)
        assertThat(list.size(), is(n));
    }

    @When("^.*create (?:a|another) .*simple object$")
    public void create_a_simple_object() {
        wrap(simpleObjects).create(UUID.randomUUID().toString());
    }

    <T> T wrap(T domainObject) {
        return wrapperFactory.wrap(domainObject);
    }

    @Inject protected WrapperFactory wrapperFactory;                    (5)
}
1 again, under the stepdefs package
2 injected domain service being interacted with
3 regex to match to feature file specification.
4 code that interacts with the domain service. This is done using the WrapperFactory to simulate the UI.
5 supporting domain services

The Scratchpad domain service is one way in which glue classes can pass state between each other. Or, for more type safety, you could develop your own custom domain services for each scenario, and inject these in as regular services. See this blog post for more details.

Running from the IDE

IntelliJ IDEA (ultimate edition) has built-in support for running individual features:

intellij idea run feature

Running the feature will automatically create a Run Configuration. It may however be necessary to tweak this Run Configuration before the feature file runs successfully:

intellij idea feature run configuration

There are (up to) three things to change:

  • the "Glue" property can be simplified to just the domainapp.webapp.bdd.stepdefs package

  • the "Working directory" property should be set to ${MODULE_WORKING_DIR}

    Note: at the time of writing this doesn’t seem to be in the drop-down, so just type it in

  • in the "Before launch", make sure that the domain entities are enhanced.

Maven Configuration

Dependencies

Apache Isis' BDD spec support is most easily configured through a dependency on the isis-mavendeps-integspecs module:

<dependency>
    <groupId>org.apache.isis.mavendeps</groupId>
    <artifactId>isis-mavendeps-integspecs</artifactId>
    <scope>test</scope>                                (1)
    <type>pom</type>
</dependency>
1 Normally test; usual Maven scoping rules apply.

This will set up dependencies for BDD specs support libraries, along with a number of libraries. There is no need to specify the version if you inherit from from the Parent POM.

If you just want to set up BDD spec support, then use:

<dependency>
    <groupId>org.apache.isis.core</groupId>
    <artifactId>isis-core-specsupport</artifactId>
    <scope>test</scope>
</dependency>

Cucumber CLI

At the time of writing, the Maven Surefire does not support custom JUnit platform test engines. As a workaround, we use the Antrun plugin to execute the Cucumber CLI.

webapp/pom.xml
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-antrun-plugin</artifactId>
  <executions>
    <execution>
      <id>cucumber-cli</id>
      <phase>integration-test</phase>
      <goals>
        <goal>run</goal>
      </goals>
      <configuration>
        <target>
          <echo message="Running Cucumber CLI" />
          <java classname="io.cucumber.core.cli.Main"
                fork="true"
                failonerror="true"
                newenvironment="true"
                maxmemory="1024m"
                classpathref="maven.test.classpath">
            <arg value="--plugin" />
            <arg value="json:${project.build.directory}/cucumber-no-clobber.json" />
            <arg value="--glue" />
            <arg value="domainapp.webapp.bdd.stepdefs" />
            <arg value="${project.build.directory}/test-classes/domainapp/webapp/bdd/specs" />
          </java>
        </target>
      </configuration>
    </execution>
  </executions>
</plugin>

This uses all of the step definitions found in the stepdefs package, and writes the results to the cucumber-no-clobber.json file.

see Cucumber-JVM documentat for the full set of arguments.

Generated Report

BDD is all about creating a conversation with the domain expert, and that includes providing meaningful feedback as to whether the spec is passing or failing.

The SimpleApp's webapp module uses a Maven plugin to generate a snazzy HTML website based on the contents of the .json file emitted by the Cucumber CLI.

The plugin’s configuration is:

webapp/pom.xml
<plugin>
  <groupId>net.masterthought</groupId>
  <artifactId>maven-cucumber-reporting</artifactId>
  <version>${maven-cucumber-reporting.version}</version>
  <executions>
    <execution>
      <id>default</id>
      <phase>post-integration-test</phase>
      <goals>
        <goal>generate</goal>
      </goals>
      <configuration>
        <projectName>SimpleApp</projectName>
        <outputDirectory>${project.build.directory}</outputDirectory>
        <inputDirectory>${project.build.directory}</inputDirectory>
        <jsonFiles>
          <param>**/cucumber-no-clobber.json</param>
        </jsonFiles>
        <skip>${skipBDD}</skip>
      </configuration>
    </execution>
  </executions>
</plugin>

Note how this reads the same file that was generated by Cucumber CLI.

The report generated by SimpleApp looks like this:

bdd report

The idea is that this could then be published to a webserver to create an information radiator.