Parallel Docker builds for the wp-browser project - 03
August 23, 2019
This post is the third in a series; find the first one here and the second one here.
In this series, I'm documenting the steps to add parallel, container-based, builds to the wp-browser project.
To set expectations: as a result of this research and effort, I might add a container-dedicated command to wp-browser in the future, but this is not my primary goal now.
My main goal is being able to run wp-browser own CI builds (on Travis CI, at the moment) faster and more reliably.
Each step, reasoning, and assumption in these posts is part of my discovery process; as such, these might be confused, out of order, or outright wrong. Ye be warned.
You can find the final post shown in this post here.
Running make tasks in parallel
After the round-robin information and data collection I've done in the second post, before I spend any time developing further, I want to make sure the docker-compose based approach to parallel execution works.
I'm using make to run the scripts, and I do not know, at this stage, if this is the best way, but it's what I can immediately work with now.
The make utility comes with parallel execution support out of the box, I've used it a couple of times, but never with intent.
I set up a small test script and a new target in the project Makefile to test this out:
# Define the list of scripts to run
SCRIPTS = foo baz bar
# Generate the targets from the list of scripts.
targets = $(addprefix php_script_, $(SCRIPTS))
# Generate the targets.
$(targets): php_script_%:
@php -r '$$i=0; while( $$i<5 ){ echo "PHP Script $@, run " . $$i++ . PHP_EOL; sleep( 5 / random_int( 1,10 ) ); }'
# Define a target in charge of running the generated targets.
pll: $(targets)
There's a bit of make syntax and constructs here, but the general catch is that I'm generating a list of "targets" ("tasks to run" in make terminology) from a list and running them with the pll (short for "parallel" command).
Each PHP script runs 5 times and sleeps a random number of tenths of a second before resuming.
Please note the double $$ symbol: without going into too much detail into how make works, any single $ symbol would be interpreted, by make as a make variable; since PHP variable sign of choice is the same (the $ symbol) I've doubly escaped it. Each target is executing a bash command like this:
<?php
$i = 0;
while( $i<5 ){
echo 'PHP Script $@, run ' . $i++ . PHP_EOL;
sleep( 5 / random_int( 1,10 ) );
}
I execute the three recipes in parallel using:
make -j pll
To get the typical parallel processing output where the output from each command is mixed with all the others.
[
](https://theaveragedev.com/wp-content/uploads/2019/08/make-parallel-1.png)
To order the output a bit, I can use the -O flag (GNU version of make only, I use that on my Mac machine):
make -j -O pll
The target order might, in this case, be random, but the output is grouped by the target, depending on their completion order.
[
](https://theaveragedev.com/wp-content/uploads/2019/08/make-parallel-2.png)
Failing some parallels task
The next step is making sure the parallel execution does not break when one or more, of the tasks, fail.
With the objective of speeding up wp-browser builds by running each suite in parallel, really running any Codeception command in parallel, failing builds are a reality I have to cope with and react to.
I've added a second group of test targets running this randomly failing PHP script:
<?php
$i = 0;
while( $i<5 ){
echo 'PHP Script $@, run ' . $i++ . PHP_EOL;
$random_int = random_int(1,10);
if($random_int < 3){
// Fail if the value is less than 3.
exit( 1 );
}
sleep( 5 / $random_int );
}
echo 'PHP Script $@ completed.' . PHP_EOL;
Running the scripts with, and without, the -O option yields the following results:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/make-parallel-3.png)
[
](https://theaveragedev.com/wp-content/uploads/2019/08/make-parallel-4.png)
In the last screenshot not only I'm running the parallel "builds" using the -O flag, but I'm also printing the exit status code of the whole make run.
The exit code of 2 indicates an error in the Makefile; since all the build systems I know of take any non-zero value as an error, this is fine as an indication the build failed.
The problems of parallelism
As anything in development, parallelism comes with its own set of problems.
The first and most obvious one is that the order of termination is not guaranteed, while the second, and less obvious one, is that parallel processes accessing non-parallel resources generating race conditions and possible inconsistencies.
I can better explain this with an example.
I've added another target to the Makefile:
RUNS = 1 2 3
docker_runs = $(addprefix run_, $(RUNS))
$(docker_runs): run_%:
@docker-compose run --rm wpbrowser run test_suite -f -q --no-rebuild
pll_docker_builds: $(docker_runs)
The target is running the same test suite, the test_suite one I've created to test this behavior, three times.
The test_suite suite contains only the following test case:
<?php
class FileTest extends \Codeception\Test\Unit
{
protected static function getFiles()
{
$files = [ 'one', 'two', 'three', 'four', 'five', 'six', 'seven' ];
return $files;
}
protected static function cleanFiles()
{
foreach (static::getFiles() as $file) {
$filePath = codecept_output_dir($file);
if (file_exists($filePath)) {
unlink($filePath);
}
}
}
public function setUp()
{
static::cleanFiles();
}
public function tearDown()
{
static::cleanFiles();
}
public function filesDataSet()
{
foreach (static::getFiles() as $file) {
yield $file => [ $file ];
}
}
/**
* @dataProvider filesDataSet
*/
public function test_files_can_be_created_and_removed($file)
{
// Make sure no file initially exists.
$filePath = codecept_output_dir($file);
$this->assertFileNotExists($filePath);
touch($filePath);
$this->assertFileExists($filePath);
}
}
The test case only has one test method: that test method iterates on seven file names, make sure each file does not exist already, create each file in the Codeception output directory, and make sure that file exists.
All tests pass when running one after the other:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/parallelism-issues-1.png)
But fail when running in parallel, seemingly at random:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/parallelism-issues-2.png)
In its simplicity, it showcases the issue of parallelism: shared resources, the host machine shared filesystem in this specific case.
In the docker-compose.yml file I'm using I'm binding the host machine local filesystem, specifically the wp-browser folder, in the /project folder of each container:
version: '3.2'
services:
wpbrowser:
# Instead of using a pre-built image, let's build it from the file.
build:
context: ./docker/wpbrowser
dockerfile: Dockerfile
args:
# By default use PHP 7.3 but allow overriding the version.
- BUILD_PHP_VERSION=${BUILD_PHP_VERSION:-7.3}
- BUILD_PHP_TIMEZONE=${BUILD_PHP_TIMEZONE:-UTC}
- BUILD_PHP_IDEKEY=${BUILD_PHP_IDEKEY:-PHPSTORM}
- BUILD_XDEBUG_ENABLE=${BUILD_XDEBUG_ENABLE:-1}
volumes:
# Bind the project files into the /project directory.
- ./:/project
environment:
# As XDebug remote host use the host hostname on Docker for Mac or Windows, but allow overriding it.
# Build systems are usually Linux-based.
# Use default port, 9000, by default.
XDEBUG_CONFIG: "remote_host=${XDEBUG_REMOTE_HOST:-host.docker.internal} remote_port=${XDEBUG_REMOTE_PORT:-9000}"
# I use PHPStorm as IDE and this makes the handshake easier.
PHP_IDE_CONFIG: "serverName=wp.test"
That binding means each container has its own, independent, filesystem, but each container synchronizes, in real-time, its filesystem modifications to the host machine filesystem; this affects, implicitly, all other containers synchronizing their filesystem with the host machine filesystem. In even shorter terms: all containers are reading, and writing, from the same source.
Why would this be a problem for the wp-browser project? Or for any project using wp-browser to run WordPress tests?
Because wp-browser has extensions and modules using the filesystem.
The reason being that, at times, the only way to affect WordPress behavior is to scaffold themes or plugins (or must-use plugins) "on the fly" and remove them after the tests.
This need to modify the installation files could be involved in something simpler, like testing media attachments.
Long story short: the containers should not share their files; ideally each should work on its copy of them.
Can this filesystem isolation be achieved?
Resolving the container filesystem issues
The docker-compose file syntax allows limiting what containers can do with bound host directories by specifying that a bound directory is read-only.
While I do not want the containers affecting the host, I do need the containers to be able to write on their files, so the read-only approach is not the correct one.
The simple solution is modifying the test container Dockerfile to copy over the project files and rebuilding the container before each parallel run.
I've modified the docker-compose.yml file to remove the volume entry and update the wpbrowser.build.context to the project root folder:
version: '3.2'
services:
wpbrowser:
# Instead of using a pre-built image, let's build it from the file.
build:
# The context is the root folder any relative host machine path wil refer to in the Dockerfile.
context: .
dockerfile: docker/wpbrowser/Dockerfile
args:
# By default use PHP 7.3 but allow overriding the version.
- BUILD_PHP_VERSION=${BUILD_PHP_VERSION:-7.3}
- BUILD_PHP_TIMEZONE=${BUILD_PHP_TIMEZONE:-UTC}
- BUILD_PHP_IDEKEY=${BUILD_PHP_IDEKEY:-PHPSTORM}
- BUILD_XDEBUG_ENABLE=${BUILD_XDEBUG_ENABLE:-1}
environment:
# As XDebug remote host use the host hostname on Docker for Mac or Windows, but allow overriding it.
# Build systems are usually Linux-based.
# Use default port, 9000, by default.
XDEBUG_CONFIG: "remote_host=${XDEBUG_REMOTE_HOST:-host.docker.internal} remote_port=${XDEBUG_REMOTE_PORT:-9000}"
# I use PHPStorm as IDE and this makes the handshake easier.
PHP_IDE_CONFIG: "serverName=wp.test"
I've also modified the docker/wpbrowser/Dockerfile to COPY over the project files during build:
# Take the BUILD_PHP_VERSION argument into account, default to 7.3.
ARG BUILD_PHP_VERSION=7.3
# Allow for the customization of other PHP parameters.
ARG BUILD_PHP_TIMEZONE=UTC
ARG BUILD_PHP_IDEKEY=PHPSTORM
ARG BUILD_XDEBUG_ENABLE=1
# Build from the php-cli image based on Debian Buster.
FROM php:${BUILD_PHP_VERSION}-cli-buster
LABEL maintainer="luca@theaveragedev.com"
# Install XDebug extension.
RUN pecl install xdebug
# Create the /project directory and change to it.
WORKDIR /project
# Create an executable wrapper to execute the project vendor/bin/codecept binary appending command arguments to it.
RUN echo '#! /usr/bin/env bash\n\
# Run the command arguments on the project Codeception binary.\n\
vendor/bin/codecept $@\n' \
> /usr/local/bin/codecept \
&& chmod +x /usr/local/bin/codecept
# By default run the wrapper when the container runs.
ENTRYPOINT ["/usr/local/bin/codecept"]
# At the end to leverage Docker build cache.
COPY . /project
# Set some ini values for PHP.
# Among them configure the XDebug extension.
RUN echo "date.timezone = ${BUILD_PHP_TIMEZONE}\n\
\n\
[xdebug]\n\
zend_extension=xdebug.so\n\
xdebug.idekey=${BUILD_PHP_IDEKEY}\n\
xdebug.remote_enable=${BUILD_XDEBUG_ENABLE}\n\
" >> /usr/local/etc/php/conf.d/99-overrides.ini
I've finally modified the Makefile to force a build, that benefitting from Docker build cache, before running the tests:
RUNS = 1 2 3
docker_runs = $(addprefix run_, $(RUNS))
$(docker_runs): run_%:
@docker-compose run --rm wpbrowser run test_suite -f -q --no-rebuild
build_test_container:
@docker-compose build
pll_docker_builds: $(docker_runs)
Since parallel execution, in make applies to any specified target, I have to ensure the build_test_container target runs before the pll_docker_builds one.
It's as simple as concatenating two calls to make:
make build_test_container && make -j -O pll_docker_builds
[
](https://theaveragedev.com/wp-content/uploads/2019/08/good-parallel-1.png)
You can find the code I've shown in this post here
Next
I need to put all this together to try and run all the suites in parallel.
New challenges await me as not all suites have no infrastructure requirements as the unit suite does, and many require a database, a web-server, and a Chromedriver container to run acceptance tests.