Browser-based functional tests are a great way to verify a Grails application works fully end-to-end. But since they use a browser, they are slower to run than unit or integration tests. And depending on how large your functional test suite is, it can take quite a long time to run all your functional tests. This long test suite time can slow down your ability to turn around quick code changes or can make developers less likely to run the full test suite before committing code. We’ll walk through how we can reduce the overall time it takes to run a functional test suite by running the tests in parallel.

In this example, we’re going to test a Grails app using the Geb functional test framework and run the tests with Gradle. The full working example from this post is available on GitHub at https://github.com/craigatk/geb-parallel-gradle

Configuring Gradle dependencies

Since we’re compiling and running the tests with Gradle, we need to declare all the dependencies required to compile and run the tests in Gradle’s build.gradle file. Also, we need to tell Gradle which tests to compile by pointing the source sets directly to the functional test directory.

String seleniumVersion = "2.35.0"

dependencies {
  testCompile "org.codehaus.groovy:groovy-all:2.0.8"

  testCompile "org.gebish:geb-spock:0.9.1"
  testCompile "org.seleniumhq.selenium:selenium-firefox-driver:${seleniumVersion}"
  testCompile "org.seleniumhq.selenium:selenium-support:${seleniumVersion}"

  testCompile 'org.codehaus.groovy.modules.remote:remote-transport-http:0.5'

  testCompile "org.spockframework:spock-core:0.7-groovy-2.0"
}

sourceSets {
  test {
    groovy {
      srcDirs = ['test/functional']
    }
  }
}

Note: We’re directly referencing the Groovy Remote Control library, which is the underpinnings of the Grails Remote Control plugin. We can’t use the remote control Grails plugin directly since we’re compiling and running the tests with Gradle instead of Grails. But it’s not a big deal to configure our own remote control to use from the tests:

class AppRemoteControl extends groovyx.remote.client.RemoteControl {
  AppRemoteControl() {
    super(new groovyx.remote.transport.http.HttpTransport(getRemoteControlUrl()))
  }

  static String getRemoteControlUrl() {
    String port = System.getProperty('server.port', '8080')

    return "http://localhost:${port}/geb-parallel-gradle/grails-remote-control"
  }
}

Starting the app

To start the app we’re using the Grails wrapper and some custom code from Tomas Lin to start the wrapper process. This code waits until Grails prints out the status message that the server is ready so we can wait for that and not start the tests before the app is up and running.

Process grailsTestAppProcess

test.doFirst {
  // Using the Grails wrapper to start the Grails application in the 'test' environment so the remote control
  // plugin is enabled and my remote util bean is injected from resources.groovy
  println "Starting grails app"

  String grailsRunAppCommand

  if (Os.isFamily(Os.FAMILY_WINDOWS)) {
    grailsRunAppCommand = "grailsw.bat test run-app"
  } else {
    grailsRunAppCommand = "./grailsw test run-app"
  }

  grailsTestAppProcess = execCommandAndWaitForStatusString(grailsRunAppCommand, 'Server running.')
}

/**
 * Credit for this method goes to Tomas Lin
 * http://fbflex.wordpress.com/2013/03/14/gradle-madness-execwait-a-task-that-waits-for-commandline-calls-to-be-ready/
 */
Process execCommandAndWaitForStatusString(String command, String readyStatusString) {
  ProcessBuilder builder = new ProcessBuilder(command.split(' '))
  builder.redirectErrorStream(true)
  builder.directory(new File("."))
  Process process = builder.start()

  InputStream processStdout = process.getInputStream()
  BufferedReader reader = new BufferedReader(new InputStreamReader(processStdout))

  def line
  while ((line = reader.readLine()) != null) {
    println line
    if (line.contains(readyStatusString)) {
      println "'${command}' is ready"
      break;
    }
  }

  process.consumeProcessOutput(System.out, System.err)

  return process
}

Running the tests in parallel

The Gradle test task is initially configured to run with two parallel forks. If you have the necessary CPU cores and RAM to support more forks, you can set the number higher as needed. Though there is some startup lag when creating a new testing fork, so for smaller test suites the overhead of more forks may not actually save overall test suite time. It’s worth some testing with your specific test suite to find the sweet spot for the number of parallel forks.

test {
  // Disable Gradle's up-to-date checking so it always runs the tests, even if I didn't update any test files
  outputs.upToDateWhen { false }

  maxParallelForks = 2

  String grailsServerPort = System.getProperty('server.port', '8080')

  systemProperty "geb.build.reportsDir", file("$buildDir/test-results-functional/geb")
  systemProperty "geb.build.baseUrl", "http://localhost:${grailsServerPort}/geb-parallel-gradle/"
}

Stopping the app

When I initially worked on this project, I called process.destroy() on the Grails wrapper process, but that didn’t work on all operating systems. For example, on Windows calling .destroy() killed the Grails wrapper bat script but left the underlying Java process running and orphaned.

Thankfully there is a little-known way to kill a Grails app that’s built into the framework. Grails will kill the server if it finds a file named “.kill-run-app” in the root of the project. So here we create that file when the functional tests are finished, and Grails will stop the server for us.

task stopGrails << {
  // Grails will shut down when it sees a file called ".kill-run-app"
  // See http://stackoverflow.com/a/4665331
  new File(".kill-run-app").text = ""

  grailsTestAppProcess.waitFor()
}

test.finalizedBy stopGrails

Another key piece in the Gradle code that stops the app - we’re using test.finalizedBy instead of test.doLast because .doLast isn’t executed if any of the tests fail.

Designing tests to support parallel execution

In addition to setting up the build infrastructure to support running functional tests in parallel, you’ll need to ensure your tests themselves can handle running in parallel. For example, your tests must meet criteria like these:

1. No shared DB data, such as users. Each test must set up all of its own data.
2. No changes to global state, such as config value changes
3. No system-wide destructive operations, such as dropping DB tables between tests or clearing caches
4. No reliance on expected numbers of records, such as creating a user, going to the user list page and verifying only one user is present. Another test running in parallel may have just created a second user.

Is parallel testing worth the cost?

As you can see, running parallel tests is more complex than just firing up multiple browsers at once. You’ll need to spend time both updating your build infrastructure and potentially changing data setup and verification in many of your tests. Depending on the number of existing tests in your test suite, updating them all to work in parallel can take a long time. If you are starting a new project, the easiest and least expensive way to get started with parallel testing is to design your tests to work in parallel from the start. (This is the approach I took on my latest project) But if your test suite takes a very long time to run and you have a large team who are each running it frequently, updating it to work in parallel may be worth the cost.

The full sample project is on GitHub at https://github.com/craigatk/geb-parallel-gradle

Happy testing!