Running tycho tests on multiple machines

Recently I ran into a situation where I wanted to run the tests of an RCP application on different machines. Obviously, the simplest solution consists in running the full build on each machines. This means that every bundle and test bundle will be compiled on each machine, and the tests executed on each machine.

Of course, this works. However I'm not a fan of this approach because I like to test what I ship. What I'm getting at is that when the code is compiled on different machines, there is a chance that different binaries can be produced, which means that from a purist point of view the testing done is invalid since you have not tested the binaries that are finding their way into the final distribution of the product.

What I'm describing below is a way to structure the build that allows:

• tests to be executed on multiple machines without recompiling the tested code;
• developers to keep on running mvn clean install and get the code to compile and the tests to run;
• developers to have minimal understanding of the overall build setup to add a bundle and its tests.

First, let's review the relevant modules of the setup:

• code : bundle that needs to be tested
• test : bundle that contains the tests
• repo : p2 repository containing the bundles being built
• allTests : parent and aggregator that build and execute the tests
• parent : main parent and aggregator for the build that includes code, repo and allTests.

The main idea behind this setup is that it provides two main entry points. One that allows for the complete build to run and a second one that just executes the tests. With these abilities it becomes very easy to create a setup where:

• a main build machine would run a complete build and publish the p2 repository to a shared drive or a repository manager. This is done by executing mvn clean install in top level aggregator here called parent;
• multiple test machines would be configured to just execute the tests and obtain the artifacts to tests from the repository created by the main build. This is done by executing mvn clean install -DtestedRepo=<pathtorepo>
• the coordination between the main build and the tests would be done using a CI.

Details:
The repo used to obtain the artifacts needs to be setup when the test build is run. In case of a full build, we don't want this repo to be used since it could cause interference. Therefore, the repository is defined in a test specific profile, in the allTests aggregator that also becomes the parent of all the tests projects.
Rather than requesting the activation of the profile and passing the repository parameter, this profile is only activated when the testedRepo property is set.

Et voila, it is that simple. This approach also has the following additional benefits:

• catch errors early without wasting multiple build executors since the main build will first run to completion by executing the tests;
• potential build time reduction, if download jars from a repository is faster than building them.

The complete code for this example can be found at https://bitbucket.org/prapicault/tycho-tests-multiple-machines.