Docker and docker-compose for WordPress testing - 03

Previously, in this series

In the first post in this series, I've covered the basic stuff: what Docker and docker-compose are and how they could enable a portable environment for development, and more interesting to me, testing WordPress projects.
In the second post, I've detailed the behavior of the same docker-compose.yml stack on macOS, Windows, and Linux machines.
I've concluded the post in a situation where files created by the WordPress container, on a Linux host, would not belong to my user, but another user.

In this third article, I will work to mitigate the "portability" issues arising from a stack that will behave differently depending on the host operating system (macOS, Windows or Linux) to move closer to the ideal of a portable testing environment for WordPress projects.

The docker-compose.yml file in question, the one I'm starting this post from, is this one:

version: '3.1'

volumes:

  db:

services:

  wordpress:
    image: wordpress
    ports:
      - 8080:80
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: exampleuser
      WORDPRESS_DB_PASSWORD: examplepass
      WORDPRESS_DB_NAME: exampledb
    volumes:
      # The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
      - ./_wordpress:/var/www/html

  db:
    image: mysql:5.7
    environment:
      MYSQL_DATABASE: exampledb
      MYSQL_USER: exampleuser
      MYSQL_PASSWORD: examplepass
      MYSQL_RANDOM_ROOT_PASSWORD: '1'
    volumes:
      - db:/var/lib/mysql

The difference between host users and container users in Linux

The main difference between how Docker works on macOS and Windows versus how works on Linux is, without delving too deep into technical aspects I'm not expert enough to discuss, there is an additional layer (allow me to call it a "virtual machine") between the macOS or Windows host and the containers that is not present on Linux.

Containers are a Linux technology ported over to macOS and Windows. The ports solve several compatibility problems across different operating systems and mitigate, leveraging the use of "virtual machine" of sorts, some of the key differences between the OSes. Containers ignore the underlying implementation; in a way, Docker (for Windows, for macOS and Linux) acts as Java: create an image, build, publish it, share it, and run wherever Docker runs.

The differences become something to deal with and put some care into when sharing files between the host machine and the container. With some exceptions, most images are based on Linux OSes, where file permissions are serious business and not a small part of why Linux is the de-facto standard when it comes to servers.

When containers run on Linux, file ownership of shared files is maintained.
What this means is that:

If a file belongs to the user luca on the host machine, then that file will belong to luca in the container.

• Vice-versa, if a file belongs to www-data in the container, it will belong to www-data on the host machine.

The second would explain why, in my previous post, I could not delete files created by the container from the host machine: my user on the host, luca is not the owner of the files created by the www-data user in the container.
When I tried to do that I would get the following output:

Repos/wp-docker » whoami
luca
Repos/wp-docker » ls -la _wordpress/wp-content/plugins/hello-dolly
total 16
drwxr-xr-x 2 www-data www-data 4096 Jun  6 16:58 .
drwxr-xr-x 4 www-data www-data 4096 Jun  6 16:58 ..
-rw-r--r-- 1 www-data www-data 2593 Jun  6 16:58 hello.php
-rw-r--r-- 1 www-data www-data  623 Jun  6 16:58 readme.txt
Repos/wp-docker » id -u
1002
Repos/wp-docker » id -u www-data
33

To delete the files from the host I am forced to use sudo:

Repos/wp-docker » sudo rm -rf _wordpress/wp-content/plugins/hello-dolly 
[sudo] password for luca: ***********

Using sudo is a dangerous habit and a requirement I would like to avoid in this instance.

Fixing the file ownership issue in the WordPress container

The official WordPress image published on Dockerhub supports running Apache as a different user.
Looking at the docker-entrypoint.sh file that will run when the container starts, I see I could set the APACHE_RUN_USER and APACHE_RUN_GROUP environment variables to have the Apache web-server process belong to a specific user.

What user? If on my host Linux machine my user is luca, and I want my user to be able to modify and delete the files shared with the container without requiring the use of sudo, then Apache must be started by luca.
I will specify the user by its user ID (uid) and group ID (gid).

I've shown it before, but, as a refresher, I can fetch the current user ID and group ID from the terminal:

Repos/wp-docker » id -u
1002
Repos/wp-docker » id -g
1002

Usually, on Linux, those same values will be available on the UID and GID environment variables, but I prefer to rely on commands I run to make sure I'm getting the values I need.

I modify the docker-compose.yml file to define the APACHE_RUN_USER and APACHE_RUN_GROUP environment variables:

version: '3.1'

volumes:

  db:

services:

  wordpress:
    image: wordpress
    ports:
      - 8080:80
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: exampleuser
      WORDPRESS_DB_PASSWORD: examplepass
      WORDPRESS_DB_NAME: exampledb
      APACHE_RUN_USER: ${APACHE_RUN_USER}
      APACHE_RUN_GROUP: ${APACHE_RUN_GROUP}
    volumes:
      # The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
      - ./_wordpress:/var/www/html

  db:
    image: mysql:5.7
    environment:
      MYSQL_DATABASE: exampledb
      MYSQL_USER: exampleuser
      MYSQL_PASSWORD: examplepass
      MYSQL_RANDOM_ROOT_PASSWORD: '1'
    volumes:
      - db:/var/lib/mysql

The FOO: ${BAR} notation means "at runtime set the value of the FOO environment variable in the container to the current value of the BAR environment variable on the host". There is no requirement for the two variables to have the same name; it's just convenient.

I try to spin up the stack again, specifying the two environment variables:

APACHE_RUN_USER=$(id -u) \
APACHE_RUN_GROUP=$(id -g) \
docker-compose up

Recreating wp-docker_wordpress_1 ... done
Attaching to wp-docker_wordpress_1
wordpress_1  | AH00543: apache2: bad user name 1002
wp-docker_wordpress_1 exited with code 1

Ok, this makes sense. The user luca does not exist in the container; the Apache process cannot belong to a user that does not exist.

The docker-compose specification allows specifying a user, this is the equivalent of [using the -u|--user option when using the docker run command.

The documentation says:

When passing a numeric ID, the user does not have to exist in the container.

Which is what I need, I want to have a user somehow automagically created for me and I want to run Apache as that user.

I modify the docker-compose.yml file again:

version: '3.1'

volumes:

  db:

services:

  wordpress:
    image: wordpress
    ports:
      - 8080:80
    user: ${WORDPRESS_RUN_USER}:${WORDPRESS_RUN_GROUP}
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: exampleuser
      WORDPRESS_DB_PASSWORD: examplepass
      WORDPRESS_DB_NAME: exampledb
      APACHE_RUN_USER: ${APACHE_RUN_USER}
      APACHE_RUN_GROUP: ${APACHE_RUN_GROUP}
    volumes:
      # The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
      - ./_wordpress:/var/www/html

  db:
    image: mysql:5.7
    environment:
      MYSQL_DATABASE: exampledb
      MYSQL_USER: exampleuser
      MYSQL_PASSWORD: examplepass
      MYSQL_RANDOM_ROOT_PASSWORD: '1'
    volumes:
      - db:/var/lib/mysql

I try again setting the two new variables as well:

APACHE_RUN_USER=$(id -u) \
APACHE_RUN_GROUP=$(id -g) \
WORDPRESS_RUN_USER=$(id -u) \
WORDPRESS_RUN_GROUP=$(id -g) \
docker-compose up

wp-docker_wordpress_1 is up-to-date
Creating wp-docker_db_1 ... done
Attaching to wp-docker_wordpress_1, wp-docker_db_1
db_1         | 2020-06-09 11:44:26+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.30-1debian10 started.
wordpress_1  | sed: couldn't open temporary file ./sedEJZmHq: Permission denied
wordpress_1  | sed: couldn't open temporary file ./sedLW2kUW: Permission denied
wordpress_1  | sed: couldn't open temporary file ./sedbvCEiP: Permission denied
wordpress_1  | sed: couldn't open temporary file ./sedEf5PRI: Permission denied
wordpress_1  | sed: couldn't open temporary file ./sed8AOfWS: Permission denied
wordpress_1  | sed: couldn't open temporary file ./sedo8VJU5: Permission denied
wordpress_1  | sed: couldn't open temporary file ./sedpTR0UV: Permission denied
wordpress_1  | sed: couldn't open temporary file ./sed0jXAY3: Permission denied
wordpress_1  | sed: couldn't open temporary file ./sedxBnvZX: Permission denied
wordpress_1  | sed: couldn't open temporary file ./sedgVhJmU: Permission denied

Not what I had expected.

When I run the docker-compose up command, I'm just starting a series of connected containers and running commands in each.

What does "running a command" means?

Containers are a "technology" to run commands in an isolated, self-contained environment.
As an example, the Nginx container will run the nginx -g daemon off command, and the WordPress container I'm using in the current stack will run the apache2-foreground command.
The commands, as is the case with the two I've just listed, might be commands to start a server, but any container image (from which containers are built) is essentially a way to prepare for, and run, a single command. The fact that a command might do something and return in a second or run for weeks does not change how containers work.

It's essential to understand this to appreciate why the user running the command is so important: if the user running the command, or any of the commands required to set up the environment for that command to run, is not authorized, then the container will exit.

By default, Docker containers will run using the root user (id=0).
As is the custom in Linux, the root user is the one who can do everything.

The change, in the docker-compose.yml file, that is causing the issue is the one where I specify a user other than the root one; that user is not authorized to write to /tmp.
In this case, writing to the /tmp directory is required while setting up the WordPress container.

I'm in a bit of a conundrum here: I want to run the final container command, apache2-foreground, to start the Apache web-server as my user (luca) but I cannot run the container as that user (what I just did), and I cannot specify, as APACHE_RUN_USER, a user that does not exist in the container.

Modifying the WordPress container command

What I got from the previous section, is that each container is a way to run a command.
To run the command, in the WordPress container, I can modify its command.

Note: specifying the command parameter in a service description in the docker-compose.yml file is the equivalent of specifying a CMD directive in the image Dockerfile.

On Linux, the user name, e.g. luca is just a local (to the machine) name, and what really has value is the user ID and the ID of the user group.
To make a parallel with WordPress, the user ID is the post ID, and the user name is the post_title: a post will keep being uniquely identifiable as long as the post ID stays the same, the title can change over time.
Knowing this, and keeping the objective of portability in mind, I call my user, the user that has the same user and group ID in the container that I have on my Linux machine (id=1002 gid=1002), docker. The user naming choice follows a common standard.

I modify the docker-compose.yml file section related to the wordpress container to:

1. Remove the user parameter and let the container run using root.
2. Create a docker user with the same uid and gid as my luca user on my Linux machine.

version: '3.1'

volumes:

  db:

services:

  wordpress:
    image: wordpress
    ports:
      - "8080:80"
    # The default entypoint is `/usr/local/bin/docker-entrypoint.sh`. Not overridden here.
    # The default command is `apache2-foreground`, to start the Apache web-server.
    # The command is overridden to create the `docker:docker` user and group mapped to the host machine
    # user and group if they do not exist already.
    # After creating the user and group, run the original command.
    # The `$$` is to escape the `$` in YAML and avoid docker-compose trying to replace it.
    command: 
        - /bin/sh
        - -c
        - |
            test $$(getent group docker) || addgroup --gid ${APACHE_RUN_GROUP} docker
            test $$(id -u docker) || adduser --uid ${APACHE_RUN_USER} --ingroup docker \
            --home /home/docker --disabled-password --gecos '' docker
            /usr/local/bin/docker-entrypoint.sh apache2-foreground
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: exampleuser
      WORDPRESS_DB_PASSWORD: examplepass
      WORDPRESS_DB_NAME: exampledb
      # Start the Apache server as the `docker` user.
      # I've created the user in the `command` section, and this will work now.
      APACHE_RUN_USER: "docker"
      APACHE_RUN_GROUP: "docker"
    volumes:
      # The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
      - ./_wordpress:/var/www/html

  db:
    image: mysql:5.7
    environment:
      MYSQL_DATABASE: exampledb
      MYSQL_USER: exampleuser
      MYSQL_PASSWORD: examplepass
      MYSQL_RANDOM_ROOT_PASSWORD: '1'
    volumes:
      - db:/var/lib/mysql

When I run the docker-compose up command again, everything will work.

APACHE_RUN_USER=$(id -u) \
APACHE_RUN_GROUP=$(id -g) \
docker-compose up

The WordPress installation files belong to my user:

Repos/wp-docker » ls -la _wordpress
total 224
drwxr-xr-x  5 luca luca  4096 Jun 10 15:39 .
drwxr-xr-x  3 luca luca  4096 Jun 10 15:39 ..
-rw-r--r--  1 luca luca   234 Jun 10 15:39 .htaccess
-rw-r--r--  1 luca luca   420 Dec  1  2017 index.php
-rw-r--r--  1 luca luca 19935 Jan  1  2019 license.txt
-rw-r--r--  1 luca luca  7368 Sep  2  2019 readme.html
-rw-r--r--  1 luca luca  6939 Sep  3  2019 wp-activate.php
drwxr-xr-x  9 luca luca  4096 Dec 18 23:16 wp-admin
-rw-r--r--  1 luca luca   369 Dec  1  2017 wp-blog-header.php
-rw-r--r--  1 luca luca  2283 Jan 21  2019 wp-comments-post.php
-rw-r--r--  1 luca luca  3183 Jun 10 15:39 wp-config.php
-rw-r--r--  1 luca luca  2808 Jun 10 15:39 wp-config-sample.php
drwxr-xr-x  4 luca luca  4096 Jun 10 15:39 wp-content
-rw-r--r--  1 luca luca  3955 Oct 11  2019 wp-cron.php
drwxr-xr-x 20 luca luca 12288 Dec 18 23:16 wp-includes
-rw-r--r--  1 luca luca  2504 Sep  3  2019 wp-links-opml.php
-rw-r--r--  1 luca luca  3326 Sep  3  2019 wp-load.php
-rw-r--r--  1 luca luca 47597 Dec  9  2019 wp-login.php
-rw-r--r--  1 luca luca  8483 Sep  3  2019 wp-mail.php
-rw-r--r--  1 luca luca 19120 Oct 15  2019 wp-settings.php
-rw-r--r--  1 luca luca 31112 Sep  3  2019 wp-signup.php
-rw-r--r--  1 luca luca  4764 Dec  1  2017 wp-trackback.php
-rw-r--r--  1 luca luca  3150 Jul  1  2019 xmlrpc.php

I can now delete the files from my host machine without using sudo:

Repos/wp-docker » rm -rf _wordpress/wp-content/plugins/hello.php 
Repos/wp-docker » ls -la _wordpress/wp-content/plugins 
total 16
drwxr-xr-x 3 luca luca 4096 Jun 10 15:56 .
drwxr-xr-x 4 luca luca 4096 Jun 10 15:39 ..
drwxr-xr-x 4 luca luca 4096 Dec 18 23:16 akismet
-rw-r--r-- 1 luca luca   28 Jun  5  2014 index.php

And, after I've installed WordPress using its UI at http://localhost:8080, if I install a plugin from the WordPress installation, tat plugin files too will be accessible to my user.

The docker-compose.yml file, in its wordpress service section especially, could be reduced by building a dedicated image customized to my needs. Currently, I prefer the compact form of the single docker-compose.yml file and will leave that for later, if required.

Thinning the command to start the stack

It's a bit cumbersome to have to start the stack using this command:

APACHE_RUN_USER=$(id -u) \
APACHE_RUN_GROUP=$(id -g) \
docker-compose up

It's not impossible to remember, but it's error-prone, and, as the stack evolves, it will become more and more complex.

To get back the ease-of-use I've created a PHP script that will take care of running the command correctly every time with awareness of the current operating system:

#! /usr/bin/env php
<?php
// Default to `up` if no arguments are provided.
// $argv[0] is the script name itself, drop it.
$unescapedArgs = isset($argv[1]) ? (array)array_slice($argv,1) : ['up'];
// Escape the command arguments before using them.
$escapedArgs = array_map('escapeshellarg', $unescapedArgs);

// Get the current OS; `dar` is macOS, `lin` is Linux, `win` is Windows
// Other OSes (*nix, BSD...) require, probably, the same setup steps as for Linux. 
$os = substr(strtolower(PHP_OS), 0, 3);

$binary = 'docker-compose';

switch ($os){
        case 'win':
            $binary = 'docker-compose.exe';
            break;
        case 'dar':
            // There is nothing to do here.
            break;
        default:
            // If we cannot get the current user and group ID use `0` (`root` in the container).
            putenv('DOCKER_RUN_USER=' . (int)getmyuid());
            putenv('DOCKER_RUN_GROUP=' . (int)getmygid());
            break;
}

// Build the command line.
$commandLine = sprintf('%s %s', $binary, implode(' ', $escapedArgs));

// Debug the command line.
echo "\nCommand: {$commandLine}\n\n";

// Let's not apply the time limit to the processes launched by the script.
set_time_limit(0);

// Execute the command.
passthru($commandLine, $status);

// Print a blank line after the command output to clear the terminal a bit.
echo "\n";

// Exit the same status as the command.
exit($status);

I've called the script stack, so I can now run the equivalent of the previous command by running:

./stack up

I've modified the stack to work on Linux, will it be the same on Windows And macOS?

In my next post, I will test the current solution out on both to move a step closer to a portable WordPress testing environment and iterate on the CLI wrapper script.