Previously
In the previous post, I got the plugin to make real HTTP requests to a llama.cpp server and receiving completions. The code worked, but it was all crammed into the Service class: HTTP connection management, JSON parsing, error handling, notifications, everything.
Time to clean up the mess.
The problem with the current code
Let me look at what I had in Service.kt:
private suspend fun getCompletion(prefix: String, suffix: String): String {
return withContext(Dispatchers.IO) {
try {
val settings = Settings.getInstance()
val state = settings.state
val endpointUrl = state.endpointUrl
// Create the HTTP connection
val url = URI(endpointUrl).toURL()
val connection = url.openConnection() as HttpURLConnection
connection.requestMethod = "POST"
connection.setRequestProperty("Content-Type", "application/json")
connection.doOutput = true
// Add API key if configured
if (state.apiKey.isNotEmpty()) {
connection.setRequestProperty("Authorization", "Bearer ${state.apiKey}")
}
// Build the request body
val requestBody = JSONObject()
requestBody.put("input_prefix", prefix)
requestBody.put("input_suffix", suffix)
// Send the request
val writer = OutputStreamWriter(connection.outputStream)
writer.write(requestBody.toString())
writer.flush()
writer.close()
// Read the response
val responseCode = connection.responseCode
if (responseCode == HttpURLConnection.HTTP_OK) {
val response = connection.inputStream.bufferedReader().use { it.readText() }
val jsonResponse = JSONObject(response)
val content = jsonResponse.optString("content", "")
logger.info("Got completion: $content")
content
} else {
logger.warn("HTTP request failed with code: $responseCode")
showErrorNotification("Failed to get completion: HTTP $responseCode")
""
}
} catch (e: Exception) {
logger.warn("Failed to get completion", e)
showErrorNotification("Failed to connect to LLM endpoint: ${e.message}")
""
}
}
}
This function does everything: HTTP, JSON, error handling, logging, notifications. And what if I want to reuse this HTTP logic elsewhere? Copy-paste? No thanks.
The plan
I asked myself: what should this refactoring look like?
After some thinking, I settled on:
- Extract HTTP logic to a dedicated class (
InfillHttpClient) that only handles raw HTTP requests
- Create a result type (
HttpResponse) that represents success vs. error responses
- Keep JSON parsing in Service - that's application logic, not HTTP logic
- Let HTTP exceptions bubble up - connection failures are exceptional, not normal flow
- Test the HTTP client thoroughly using a real test HTTP server
The HTTP client should know nothing about JSON, completions, or IntelliJ. It just sends bytes over the wire and tells you what came back.
Here's what the refactoring looks like (credit to AI for this piece of ASCII art, I would not have done it manually):
Before:
┌─────────────────────────────────────┐
│ Service.kt │
│ │
│ ┌───────────────────────────────┐ │
│ │ HTTP connection management │ │
│ │ JSON parsing │ │
│ │ Error handling │ │
│ │ Notifications │ │
│ │ Logging │ │
│ │ Settings access │ │
│ │ ... everything mixed ... │ │
│ └───────────────────────────────┘ │
└─────────────────────────────────────┘
After:
┌─────────────────────────────────────┐
│ Service.kt │
│ (Application Logic) │
│ │
│ • JSON parsing │
│ • Notifications │
│ • Logging │
│ • Settings access │
│ │
│ │ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ InfillHttpClient │ │
│ │ (HTTP Only) │ │
│ │ │ │
│ │ • POST requests │ │
│ │ • Headers │ │
│ │ • Timeouts │ │
│ │ • Cancellation │ │
│ └─────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ HttpResponse │ │
│ │ (Result Types) │ │
│ │ │ │
│ │ • Success (2xx) │ │
│ │ • Error (others) │ │
│ └─────────────────────┘ │
└─────────────────────────────────────┘
An aside: among all the good things one can do with AI, custom ASCII art is still the one that amazes me the most.
Building a response type
First, I needed a way to represent HTTP responses. I wanted to distinguish between successful responses (2xx status codes) and error responses (everything else).
In Kotlin, sealed classes are perfect for this:
sealed class HttpResponse(
open val statusCode: Int,
open val body: String,
open val headers: Map<String, List<String>>
) {
data class Success(
override val statusCode: Int,
override val body: String,
override val headers: Map<String, List<String>>
) : HttpResponse(statusCode, body, headers)
data class Error(
override val statusCode: Int,
override val body: String,
override val headers: Map<String, List<String>>
) : HttpResponse(statusCode, body, headers)
}
What are "sealed classes"? They do not really have a corresponding in PHP.
They are like abstract in the sense that they must be extended; they are like final classes in the sense that they cannot be extended. Differently from PHP the final* comes with a condition that says "it cannot be extended save for X and Y".
Here the HttpResponse class can only be extended by the Success and Error classes.
This gives me type-safe pattern matching when handling responses. The compiler will remind me if I forget to handle one of the cases. I also chose to use a sealed class (not interface) with a base constructor so I could have common properties like statusCode, body, and headers available on all responses without duplicating them (although the kotlin grammar immediately makes it more verbose than required...).
Here's how the sealed class hierarchy works:
┌──────────────────────────────┐
│ HttpResponse │
│ (Sealed Base Class) │
│ │
│ • statusCode: Int │
│ • body: String │
│ • headers: Map<String, ...> │
└──────────────┬───────────────┘
│
┌───────────────┴───────────────┐
│ │
▼ ▼
┌──────────────────────────┐ ┌──────────────────────────┐
│ HttpResponse.Success │ │ HttpResponse.Error │
│ │ │ │
│ When statusCode is 2xx │ │ When statusCode is not │
│ (200, 201, 204, etc.) │ │ 2xx (404, 500, etc.) │
│ │ │ │
│ Inherits all properties │ │ Inherits all properties │
│ from HttpResponse │ │ from HttpResponse │
└──────────────────────────┘ └──────────────────────────┘
When you pattern match:
when (response) {
is HttpResponse.Success -> // Compiler knows it's 2xx
is HttpResponse.Error -> // Compiler knows it's error
// Compiler error if you forget a case!
}
Did I mention AI does ASCII art really well?
Putting the HTTP client together
Now for the HTTP client itself. I wanted:
- Constructor takes the URL (can be changed later with
setUrl())
post() method for making POST requests- Returns
HttpResponse.Success for 2xx, HttpResponse.Error for other status codes
- Throws exceptions for connection failures
- Automatically cancels previous request. This is important as completion requests will come in bursts as I type and there's no use in getting a completion for a request that will never be used.
Here's what I came up with:
class InfillHttpClient(
private var url: String,
private val connectTimeoutMs: Int = 5000,
private val readTimeoutMs: Int = 30000
) {
private var currentConnection: HttpURLConnection? = null
fun getUrl(): String = url
fun setUrl(newUrl: String) { url = newUrl }
fun cancelPreviousRequest() {
currentConnection?.disconnect()
currentConnection = null
}
fun post(headers: Map<String, String>, body: String): HttpResponse {
// Cancel any previous request
cancelPreviousRequest()
val uri = URI(url).toURL()
val connection = uri.openConnection() as HttpURLConnection
currentConnection = connection
try {
connection.requestMethod = "POST"
connection.connectTimeout = connectTimeoutMs
connection.readTimeout = readTimeoutMs
connection.doOutput = true
// Set headers
headers.forEach { (key, value) ->
connection.setRequestProperty(key, value)
}
// Write request body
val writer = OutputStreamWriter(connection.outputStream)
writer.write(body)
writer.flush()
writer.close()
// Get response
val responseCode = connection.responseCode
val responseHeaders = connection.headerFields
.filterKeys { it != null }
.mapKeys { it.key!! }
// Read response body (from either inputStream or errorStream)
val responseBody = if (responseCode in 200..299) {
connection.inputStream.bufferedReader().use { it.readText() }
} else {
connection.errorStream?.bufferedReader()?.use { it.readText() } ?: ""
}
return if (responseCode in 200..299) {
HttpResponse.Success(responseCode, responseBody, responseHeaders)
} else {
HttpResponse.Error(responseCode, responseBody, responseHeaders)
}
} finally {
if (currentConnection == connection) {
currentConnection = null
}
connection.disconnect()
}
}
}
The key decisions:
- Store the current connection: This lets me cancel it if a new request comes in while the old one is still running. Critical for inline completions where the user might type quickly.
- Read from errorStream for non-2xx responses: HTTP servers should put error details in the error stream, not the output stream.
- Return result types, throw for real failures: If I get a 404, that's an
HttpResponse.Error. If the network is down, that's an exception.
Testing with an actual server
Here's where it got interesting. I wanted to test this HTTP client properly, which meant I needed... an HTTP server. Not a mock, but an actual server that accepts connections and sends responses.
Turns out the JDK ships with com.sun.net.httpserver.HttpServer - a simple HTTP server perfect for testing. I wrapped it in a helper class:
class TestHttpServer(private val port: Int = 0) {
private var server: HttpServer? = null
private var responseHandler: ((HttpExchange) -> Unit)? = null
fun start() {
server = HttpServer.create(InetSocketAddress(port), 0)
server?.createContext("/") { exchange ->
responseHandler?.invoke(exchange) ?: run {
exchange.sendResponseHeaders(200, 0)
exchange.close()
}
}
server?.executor = null
server?.start()
}
fun stop() {
server?.stop(0)
server = null
}
fun getUrl(): String {
val actualPort = server?.address?.port
?: throw IllegalStateException("Server not started")
return "http://localhost:$actualPort"
}
fun respondWith(statusCode: Int, body: String, headers: Map<String, String> = emptyMap()) {
responseHandler = { exchange ->
headers.forEach { (key, value) ->
exchange.responseHeaders.set(key, value)
}
val responseBytes = body.toByteArray()
exchange.sendResponseHeaders(statusCode, responseBytes.size.toLong())
exchange.responseBody.write(responseBytes)
exchange.close()
}
}
// ... more helper methods
}
Using port = 0 tells the OS to pick any available port, which is perfect for tests - no conflicts.
Now I could write real tests:
class InfillHttpClientTest : BasePlatformTestCase() {
private lateinit var server: TestHttpServer
private lateinit var client: InfillHttpClient
override fun setUp() {
super.setUp()
server = TestHttpServer()
server.start()
client = InfillHttpClient(server.getUrl())
}
override fun tearDown() {
try {
server.stop()
} finally {
super.tearDown()
}
}
fun testSuccessfulPostRequest() {
server.respondWith(200, """{"content": "hello world"}""",
mapOf("Content-Type" to "application/json"))
val response = client.post(
headers = mapOf("Content-Type" to "application/json"),
body = """{"input_prefix": "test", "input_suffix": ""}"""
)
assertTrue(response is HttpResponse.Success)
assertEquals(200, response.statusCode)
assertEquals("""{"content": "hello world"}""", response.body)
assertTrue(response.headers.isNotEmpty())
}
fun testErrorResponse() {
server.respondWith(404, """{"error": "Not found"}""")
val response = client.post(
headers = mapOf("Content-Type" to "application/json"),
body = """{"input_prefix": "test"}"""
)
assertTrue(response is HttpResponse.Error)
assertEquals(404, response.statusCode)
}
fun testConnectionFailure() {
server.stop()
try {
client.post(
headers = mapOf("Content-Type" to "application/json"),
body = """{"test": "data"}"""
)
fail("Expected an exception to be thrown")
} catch (e: Exception) {
// Expected - connection failed
}
}
}
The BasePlatformTestCase class is the PHPUnit\Framework\TestCase equivalent of testing things in the context of an IntelliJ plugin.
The header bug
Of course, the first time I ran these tests, they failed. Naturally. The testSuccessfulPostRequest was failing because response.headers didn't contain "Content-Type".
I spent a few minutes debugging before realizing: I was calling exchange.responseHeaders.add(key, value) after exchange.sendResponseHeaders(). HTTP doesn't work that way - headers have to be set before you start sending the body. It's literally in the name. Headers. First. Body. Second.
Fun fact: that is how headers and body work in PHP as well.
You'd think after all these years of web development I'd know better, but here we are.
Changed it to:
headers.forEach { (key, value) ->
exchange.responseHeaders.set(key, value) // BEFORE sendResponseHeaders!
}
val responseBytes = body.toByteArray()
exchange.sendResponseHeaders(statusCode, responseBytes.size.toLong())
And the tests passed. This is exactly why you write tests - you catch these silly mistakes before they become production bugs.
Progress.
Refactoring Service to use InfillHttpClient
With the HTTP client tested and working, I could refactor Service:
class Service : InlineCompletionProvider {
private val logger = Logger.getInstance(Service::class.java)
private var httpClient: InfillHttpClient? = null
// ... other methods ...
private suspend fun getCompletion(prefix: String, suffix: String): String {
return withContext(Dispatchers.IO) {
try {
val settings = Settings.getInstance()
val state = settings.state
val endpointUrl = state.endpointUrl
// Initialize or update HTTP client
if (httpClient == null) {
httpClient = InfillHttpClient(endpointUrl)
} else if (httpClient!!.getUrl() != endpointUrl) {
// Udpate the URL if it changed in the settings.
httpClient!!.setUrl(endpointUrl)
}
// Build headers
val headers = mutableMapOf("Content-Type" to "application/json")
if (state.apiKey.isNotEmpty()) {
headers["Authorization"] = "Bearer ${state.apiKey}"
}
// Build the request body
val requestBody = JSONObject()
requestBody.put("input_prefix", prefix)
requestBody.put("input_suffix", suffix)
// Make the HTTP request
val response = httpClient!!.post(headers, requestBody.toString())
// Handle the response
when (response) {
is HttpResponse.Success -> {
val jsonResponse = JSONObject(response.body)
val content = jsonResponse.optString("content", "")
logger.info("Got completion: $content")
content
}
is HttpResponse.Error -> {
logger.warn("HTTP request failed with code: ${response.statusCode}")
showErrorNotification("Failed to get completion: HTTP ${response.statusCode}")
""
}
}
} catch (e: Exception) {
logger.warn("Failed to get completion", e)
showErrorNotification("Failed to connect to LLM endpoint: ${e.message}")
""
}
}
}
}
Much cleaner! The HTTP logic is gone. The when expression for handling HttpResponse is type-safe and explicit. And the best part? All existing tests still pass - I didn't break anything.
Next steps
The plugin now has a solid HTTP foundation. But there's more to do:
- The completion logic is still naive - I'm sending the entire file as prefix/suffix
- The request is not properly configured
- There is no context to the request if not for the surrounding lines
But those are problems for future posts. For now, the code is cleaner, tested, and ready to build on.
Introduction
In the previous post, I managed to get a "Hello World!" completion working in my IntelliJ plugin, "completamente". The completion was hardcoded, always returning the same string, but it worked. Progress.
The next logical step is connecting to an actual LLM to get real completions. I'm following the llama.vim approach, which uses the llama.cpp server's infill endpoint. The idea is simple: send the code before the cursor (prefix) and after the cursor (suffix) to the endpoint, and it returns a completion suggestion.
This post documents my attempt at implementing settings, HTTP integration, and error handling for the plugin. I'm sure someone with more experience would develop a better solution, but I'm stuck with my knowledge of the subject, and I'm doing this to learn.
Understanding the llama.cpp infill endpoint
Before writing any Kotlin code, I need to understand what the llama.cpp infill endpoint expects and returns. The llama.vim plugin source code shows it uses a POST request to /infill with input_prefix and input_suffix parameters.
I'm running a llama.cpp server locally on port 8012 with a small model (Qwen2.5-Coder-3B).
The command to start the server locally is llama-server --fim-qwen-3b-default --port 8012.
First, a simple test with just a prefix:
curl -X POST http://127.0.0.1:8012/infill \
-H "Content-Type: application/json" \
-d '{"input_prefix": "<?php\nfunction hello(", "input_suffix": ""}' \
-s | jq '.content'
The response comes back quickly enough on my laptop:
")\n{\n echo \"hello world\";\n}\n?>\n"
All fine and dandy. The key field here is content - that's the completion text I need to extract and show to the user. The response also includes a bunch of metadata about the generation settings, which I might use later for debugging and customization of the request, but for now I just need the content field.
Let me test with both prefix and suffix to see how the model handles Fill In the Middle (FIM):
curl -X POST http://127.0.0.1:8012/infill \
-H "Content-Type: application/json" \
-d '{"input_prefix": "<?php\nfunction calculateSum($a, $b) {", "input_suffix": "}"}' \
-s | jq '.content'
The response:
"\n return $a + $b;\n"
Perfect! The model understands that it needs to fill in the middle between the opening brace and the closing brace and that the code is PHP. This is exactly what I need for inline completions in the IDE.
Or rather: this is what I need to start my completion work. I will customize the request and the context later.
One more test - what happens when the endpoint is unreachable?
curl -X POST http://127.0.0.1:9999/infill \
-H "Content-Type: application/json" \
-d '{"input_prefix": "<?php\nfunction greet(", "input_suffix": ""}' \
-v
As expected, curl fails with:
* Trying 127.0.0.1:9999...
* connect to 127.0.0.1 port 9999 from 127.0.0.1 port 56266 failed: Connection refused
* Failed to connect to 127.0.0.1 port 9999 after 0 ms: Couldn't connect to server
* Closing connection
This means my plugin needs to handle connection failures gracefully - I don't want the whole thing to crash just because the server isn't running.
Implementing Settings
Before I can make HTTP requests, I need a way to configure the endpoint URL. I could hardcode it to http://127.0.0.1:8012/infill, but that would make the plugin less flexible. Different users (which I currently do not care about: scratching an itch here) might run their llama.cpp server on different ports, or even on different machines.
I need to implement a settings page where users can configure:
- The endpoint URL (default:
http://127.0.0.1:8012/infill)
- An optional API key (for when the server requires authentication, I do not need it yet, but I might someday)
Let me start by looking at how IntelliJ handles settings.
The Settings class
In IntelliJ, settings are typically implemented using the PersistentStateComponent interface. This interface provides automatic persistence - the IDE takes care of loading and saving the settings to disk. I just need to define what data to store.
Here's my Settings class:
package com.github.lucatume.completamente.settings
import com.intellij.openapi.components.PersistentStateComponent
import com.intellij.openapi.components.State
import com.intellij.openapi.components.Storage
import com.intellij.openapi.components.Service
import com.intellij.openapi.application.ApplicationManager
@State(
name = "com.github.lucatume.completamente.settings.Settings",
storages = [Storage("CompletamenteSettings.xml")]
)
@Service
class Settings : PersistentStateComponent<Settings.State> {
// The state class holds the actual settings values
data class State(
var endpointUrl: String = "http://127.0.0.1:8012/infill",
var apiKey: String = ""
)
private var myState = State()
override fun getState(): State {
return myState
}
override fun loadState(state: State) {
myState = state
}
companion object {
// Get the application-level instance of Settings
fun getInstance(): Settings {
return ApplicationManager.getApplication().getService(Settings::class.java)
}
}
}
The @State annotation tells IntelliJ where to store the settings (in an XML file called CompletamenteSettings.xml). The @Service annotation makes this a service that can be retrieved using Settings.getInstance().
The State data class is what gets serialized to disk. It's a simple data class with two fields: endpointUrl and apiKey, both with sensible defaults.
The SettingsConfigurable class
Now I need a UI for users to edit these settings. IntelliJ provides the Configurable interface for this:
package com.github.lucatume.completamente.settings
import com.intellij.openapi.options.Configurable
import javax.swing.JComponent
import javax.swing.JPanel
import javax.swing.JLabel
import javax.swing.JTextField
import java.awt.GridBagLayout
import java.awt.GridBagConstraints
import java.awt.Insets
class SettingsConfigurable : Configurable {
private var endpointUrlField: JTextField? = null
private var apiKeyField: JTextField? = null
override fun getDisplayName(): String {
return "Completamente"
}
override fun createComponent(): JComponent {
val panel = JPanel(GridBagLayout())
val constraints = GridBagConstraints()
// Endpoint URL label and field
constraints.gridx = 0
constraints.gridy = 0
constraints.anchor = GridBagConstraints.WEST
constraints.insets = Insets(0, 0, 5, 10)
panel.add(JLabel("Endpoint URL:"), constraints)
endpointUrlField = JTextField(40)
constraints.gridx = 1
constraints.gridy = 0
constraints.fill = GridBagConstraints.HORIZONTAL
constraints.weightx = 1.0
panel.add(endpointUrlField, constraints)
// API Key label and field
constraints.gridx = 0
constraints.gridy = 1
constraints.fill = GridBagConstraints.NONE
constraints.weightx = 0.0
panel.add(JLabel("API Key:"), constraints)
apiKeyField = JTextField(40)
constraints.gridx = 1
constraints.gridy = 1
constraints.fill = GridBagConstraints.HORIZONTAL
constraints.weightx = 1.0
panel.add(apiKeyField, constraints)
return panel
}
override fun isModified(): Boolean {
val settings = Settings.getInstance()
val state = settings.state ?: return false
return endpointUrlField?.text != state.endpointUrl ||
apiKeyField?.text != state.apiKey
}
override fun apply() {
val settings = Settings.getInstance()
val state = settings.state ?: Settings.State()
state.endpointUrl = endpointUrlField?.text ?: state.endpointUrl
state.apiKey = apiKeyField?.text ?: state.apiKey
settings.loadState(state)
}
override fun reset() {
val settings = Settings.getInstance()
val state = settings.state ?: return
endpointUrlField?.text = state.endpointUrl
apiKeyField?.text = state.apiKey
}
}
I'm not a Swing expert (or even a Swing beginner, really), so this code is... functional. It uses GridBagLayout to arrange the labels and text fields. The best way I can explain it is: it's like CSS grid, but more verbose and from the 1990s.
The key methods are:
createComponent(): Creates the UIisModified(): Checks if the user changed anythingapply(): Saves the changesreset(): Reverts to the saved values
Registering the settings
Finally, I need to register both the service and the configurable in plugin.xml:
<extensions defaultExtensionNs="com.intellij">
<inline.completion.provider implementation="com.github.lucatume.completamente.completion.Service"/>
<applicationService serviceImplementation="com.github.lucatume.completamente.settings.Settings"/>
<applicationConfigurable
parentId="tools"
instance="com.github.lucatume.completamente.settings.SettingsConfigurable"
id="com.github.lucatume.completamente.settings.SettingsConfigurable"
displayName="Completamente"/>
</extensions>
The applicationService entry makes the Settings service available, and the applicationConfigurable entry adds a "Completamente" page under "Tools" in the IDE settings dialog.
I ran ./gradlew build and it compiled successfully and the plugin settings section appears in all its brutalistic glory:
Implementing HTTP Client Integration
Now that I have settings configured, I need to update the Service class to actually use them.
Instead of returning "Hello World!" every time, the service should:
- Extract the text before and after the cursor (prefix and suffix)
- Make an HTTP POST request to the configured endpoint
- Parse the JSON response and extract the
content field
- Return the completion to the user
The JDK provides HttpURLConnection which should work. Let me update the Service class:
package com.github.lucatume.completamente.completion
import com.intellij.codeInsight.inline.completion.InlineCompletionEvent
import com.intellij.codeInsight.inline.completion.InlineCompletionProvider
import com.intellij.codeInsight.inline.completion.InlineCompletionProviderID
import com.intellij.codeInsight.inline.completion.InlineCompletionRequest
import com.intellij.codeInsight.inline.completion.suggestion.InlineCompletionSuggestion
import com.intellij.notification.NotificationGroupManager
import com.intellij.notification.NotificationType
import com.intellij.openapi.diagnostic.Logger
import com.github.lucatume.completamente.settings.Settings
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import org.json.JSONObject
import java.io.OutputStreamWriter
import java.net.HttpURLConnection
import java.net.URI
class Service : InlineCompletionProvider {
private val logger = Logger.getInstance(Service::class.java)
override val id: InlineCompletionProviderID
get() = InlineCompletionProviderID("completamente")
override suspend fun getSuggestion(request: InlineCompletionRequest): InlineCompletionSuggestion {
// Get the text before and after the cursor
val document = request.document
val offset = request.startOffset
val text = document.text
val prefix = text.take(offset)
val suffix = text.substring(offset)
// Get the completion from the LLM
val completion = getCompletion(prefix, suffix)
return StringSuggestion(completion)
}
override fun isEnabled(event: InlineCompletionEvent): Boolean {
return true
}
/**
* Make an HTTP POST request to the llama.cpp infill endpoint.
* Returns the completion text, or an empty string if the request fails.
*/
private suspend fun getCompletion(prefix: String, suffix: String): String {
return withContext(Dispatchers.IO) {
try {
val settings = Settings.getInstance()
val state = settings.state
val endpointUrl = state.endpointUrl
// Create the HTTP connection
val url = URI(endpointUrl).toURL()
val connection = url.openConnection() as HttpURLConnection
connection.requestMethod = "POST"
connection.setRequestProperty("Content-Type", "application/json")
connection.doOutput = true
// Add API key if configured
if (state.apiKey.isNotEmpty()) {
connection.setRequestProperty("Authorization", "Bearer ${state.apiKey}")
}
// Build the request body
val requestBody = JSONObject()
requestBody.put("input_prefix", prefix)
requestBody.put("input_suffix", suffix)
// Send the request
val writer = OutputStreamWriter(connection.outputStream)
writer.write(requestBody.toString())
writer.flush()
writer.close()
// Read the response
val responseCode = connection.responseCode
if (responseCode == HttpURLConnection.HTTP_OK) {
val response = connection.inputStream.bufferedReader().use { it.readText() }
val jsonResponse = JSONObject(response)
val content = jsonResponse.optString("content", "")
logger.info("Got completion: $content")
content
} else {
logger.warn("HTTP request failed with code: $responseCode")
showErrorNotification("Failed to get completion: HTTP $responseCode")
""
}
} catch (e: Exception) {
logger.warn("Failed to get completion", e)
showErrorNotification("Failed to connect to LLM endpoint: ${e.message}")
""
}
}
}
/**
* Show an error notification to the user.
*/
private fun showErrorNotification(message: String) {
NotificationGroupManager.getInstance()
.getNotificationGroup("Completamente")
.createNotification(message, NotificationType.ERROR)
.notify(null)
}
}
There's a lot going on here, so let me break it down:
-
Coroutines and Dispatchers: The getCompletion function is wrapped in withContext(Dispatchers.IO) which tells Kotlin to run this code on a background thread suitable for I/O operations. This is important because HTTP requests can be slow and we don't want to block the UI thread. The JavaScript mantra of not blocking the main thread is as fundamental here, especially to keep the snappy IDE experience going.
-
HTTP Request: I'm using the built-in HttpURLConnection class. It's not the most modern HTTP client (there are libraries like OkHttp that are nicer to use), but it works and doesn't require additional dependencies... wait, I already added org.json:json as a dependency because the JDK doesn't include a JSON parser. I guess I'm halfway to using modern libraries anyway. I will eventually refactor this into a dedicated requests object, so this is not as relevant now.
-
JSON Parsing: I'm using org.json.JSONObject to parse the response. The optString method returns an empty string if the field is missing, which is convenient for error handling.
-
Error Handling: I'm catching all exceptions and showing a notification to the user. This is important because if the endpoint is unreachable, I don't want the plugin to crash - I want to show a friendly error message to the user. To me.
Error Handling with Notifications
When the HTTP request fails (either due to a connection error or a non-200 status code), the plugin shows a balloon notification in the IDE.
To make this work, I had to register a notification group in plugin.xml:
<notificationGroup id="Completamente" displayType="BALLOON"/>
This creates a notification group called "Completamente" that displays as a balloon in the bottom-right corner of the IDE (the same place where build notifications appear).
Adding the JSON dependency
When I tried to build, I got compilation errors because the JDK doesn't include a JSON parser. I added the org.json library to build.gradle.kts:
dependencies {
implementation("org.json:json:20240303")
// ... other dependencies
}
The number after it, 20240303 is the version number? Or date? It works.
After that, ./gradlew build succeeded. The tests still pass (well, I had to update one test that was checking for "Hello World!" because now the service makes HTTP requests).
I have played around a bit and and it's mostly working.
Next
In the next post I will concentrate over the HTTP request part of the code:
- refactoring to an abstracted API
- handling concurrent requests correctly
- testing it
Introduction
I've been using LLMs for code assistance, in the form of completion/copilot suggestions or through a CLI
like Claude Code, for some time now, and while I've got a running list of things I like and do not like, I can see
value in the idea of LLM-assisted code writing.
Some things keep nagging at me:
- the tools do not work on a plane i.e., when there is no internet connection or when I'm not otherwise connected
- I've got a powerful laptop: why do I have to use it like a terminal in the original sense of the word? The compute
power is not local to my machine, it's in a server somewhere I have to connect to, and all my RAM and CPU/GPU is
wasted
- I have only a superficial understanding of how these tools work in the context of the IDE I use for work all-day
long
The last point, in particular, triggers me into wanting to build my own copilot plugin for my IDE of choice: an IntelliJ
one and, more specifically, PHPStorm.
Previous attempts and art
I've tried to switch away from PHPStorm to other Visual-Studio-Code-like forks and variations, but the muscle memory,
the tooling and the code inspection capabilities are not where I would like them to be.
I've tried setting up the VS-Code-likes using all the recommended plugins and tools (including licensed ones), I've
tried other IDEs as well (e.g. Zed or neovim) and all of them are great.
But they are not great for me, my idea of IDE perfection that I get out of the box with PHPStorm always another
plugin/extension away.
The one thing that really surprised me is the llama.vim plugins for vim/neovim.
With a relatively small model (Qwen2.5-Coder-3B) running on my laptop, it provides excellent suggestion with a "
look-around" quality that other tools lack.
The lack of an IntelliJ version is the inspiration for this project of mine.
Can I build an IntelliJ plugin providing auto-completions inspired by the llama.vim plugin?
It should be noted I know nothing of Kotlin, the language I will use to build the plugin.
Level 0
I've done some research, and if I'm building a plugin for an IntelliJ-based IDE, I should use
the plugin template.
Once cloned and configured I can run build the plugin using the gradlew buildPlugin task and install it in my IDE of
choice (PHPStorm) from the Plugins section of the IDE using the Install Plugin from Disk option.
I've customized the project and set its name: "completamente".
It means "completely" in Italian and, again in Italian is a composition of the words "complete" and "mind".
It's meant to be a word play around the concept of completion and mind.
Naming things is hard; I'm not a poet.
Since building the plugin and installing it in my work IDE (i.e. the IDE I work with all-day) is not ideal, I've set up
a run configuration to run the plugin in PHPStorm to debug it.
The CLI command to start a different copy of PHPStorm with my plugin installed in it would be:
./gradlew runIdeForUiTests -PplatformType=PS -PplatformVersion=2025.1.3
This is convenient, but I'm still not getting a step debugging environment with this.
To start a debug version of PHPStorm including my plugin I've added a run configuration to the project in the
.run/Run Plugin in PHPStorm.run.xml file:
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Run Plugin in PHPStorm" type="GradleRunConfiguration" factoryName="Gradle">
<log_file alias="IDE logs" path="$PROJECT_DIR$/build/idea-sandbox/*/log/idea.log" show_all="true"/>
<ExternalSystemSettings>
<option name="executionName"/>
<option name="externalProjectPath" value="$PROJECT_DIR$"/>
<option name="externalSystemIdString" value="GRADLE"/>
<option name="scriptParameters" value=""/>
<option name="taskDescriptions">
<list/>
</option>
<option name="taskNames">
<list>
<option value="runIde"/>
</list>
</option>
<option name="vmOptions" value=""/>
</ExternalSystemSettings>
<ExternalSystemDebugServerProcess>false</ExternalSystemDebugServerProcess>
<ExternalSystemReattachDebugProcess>true</ExternalSystemReattachDebugProcess>
<DebugAllEnabled>false</DebugAllEnabled>
<RunAsTest>false</RunAsTest>
<method v="2"/>
</configuration>
</component>
Now I can run or debug a copy of PHPStorm with my plugin installed:
Tapping into the completion API
The first task is connecting my plugin to the reccomended IDE completion API through the plugin configuration file (
src/main/resources/META-INF/plugin.xml):
<!-- Plugin Configuration File. Read more: https://plugins.jetbrains.com/docs/intellij/plugin-configuration-file.html -->
<idea-plugin>
<id>com.github.lucatume.completamente</id>
<name>completamente</name>
<vendor>lucatume</vendor>
<depends>com.intellij.modules.platform</depends>
<resource-bundle>messages.MyBundle</resource-bundle>
<extensions defaultExtensionNs="com.intellij">
<inline.completion.provider implementation="com.github.lucatume.completamente.completion.Service"/>
</extensions>
</idea-plugin>
The Service class is, at this stage, not doing much if not providing the same completion, "Hello World!" over and
over:
package com.github.lucatume.completamente.completion
import com.intellij.codeInsight.inline.completion.InlineCompletionEvent
import com.intellij.codeInsight.inline.completion.InlineCompletionProvider
import com.intellij.codeInsight.inline.completion.InlineCompletionProviderID
import com.intellij.codeInsight.inline.completion.InlineCompletionRequest
import com.intellij.codeInsight.inline.completion.elements.InlineCompletionGrayTextElement
import com.intellij.codeInsight.inline.completion.suggestion.InlineCompletionSingleSuggestion
import com.intellij.codeInsight.inline.completion.suggestion.InlineCompletionSuggestion
import com.intellij.codeInsight.inline.completion.suggestion.InlineCompletionVariant
class Service : InlineCompletionProvider {
// A class to represent a single string suggestion.
class StringSuggestion(private val suggestion: String) : InlineCompletionSingleSuggestion {
override suspend fun getVariant() = InlineCompletionVariant.build {
emit(InlineCompletionGrayTextElement(suggestion))
}
}
override val id: InlineCompletionProviderID
get() = InlineCompletionProviderID("completamente")
override suspend fun getSuggestion(request: InlineCompletionRequest): InlineCompletionSuggestion {
// No computation going on here, the same string every time.
return StringSuggestion("Hello World!")
}
override fun isEnabled(event: InlineCompletionEvent): Boolean {
// It's always enabled.
return true
}
}
I've started the plugin in debug mode and I can see the completion in action in the context of my di52 project:
It works. Badly and poorly, but it compiles and it works.
Tests
I'm doing this project to learn.
The easiest way for me to learn things is to test them.
I've started simple with a test to cover the really basic StringSuggestion class:
// src/test/kotlin/com/github/lucatume/completamente/completion/StringSuggestionTest.kt
package com.github.lucatume.completamente.completion
import com.intellij.testFramework.fixtures.BasePlatformTestCase
import kotlinx.coroutines.flow.count
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.runBlocking
class StringSuggestionTest : BasePlatformTestCase() {
fun testItReturnsTheSameString() {
val stringSuggestion = StringSuggestion("hello")
val variant = runBlocking { stringSuggestion.getVariant() }
val size = runBlocking { variant.elements.count() }
val suggestedString = runBlocking { variant.elements.first().text }
assertEquals(1, size)
assertEquals("hello", suggestedString)
}
}
Things got complicated as I entered the world of Kotlin and found myself immediately staring down the abyss of async, multiprocess code.
The StringSuggestion::getVariant function is marked with the suspend keyword, which means it's going to be executed
in a different thread; asynchronously, that is.
From my meager understanding of Kotlin coroutines, I've worked out that suspend is like JavaScript async and runBlocking is like await.
If I was to write the code in JavaScript, I'd write something like this:
const variant = await stringSuggestion.getVariant();
// The `count` function is a coroutine that returns the number of elements in the flow.
// That too is asynchronous.
const size = await variant.elements.count();
// The `first` function is a coroutine that returns the first element in the flow.
// That too is ... asynchronous.
const suggestedString = await variant.elements.first().text;
Running the tests with gradlew test shows the test passing.
Testing the service itself required me to dig a bit into the IntelliJ API, but I managed to put together a testing
approach that works using real files and with limited mocking.
In the test below, the file src/test/testData/completion/text-file.txt is a real file that I've created for testing
purposes and the makeInlineeCompletionRequest function provides a convenient API to create a InlineCompletionRequest
object from a file path and offsets:
class ServiceTest : BasePlatformTestCase() {
// Where I'm getting tests files from, allow me to use relative paths in the tests.
override fun getTestDataPath() = "src/test/testData/completion"
// Create an inline completion for tests.
fun makeInlineCompletionEvent(): InlineCompletionEvent {
return object : InlineCompletionEvent {
override fun toRequest(): InlineCompletionRequest {
throw UnsupportedOperationException("event.toRequest() is not supported in tests")
}
}
}
// Create an inline completion request for tests.
fun makeInlineCompletionRequest(filePath: String, startOffset: Int, endOffset: Int): InlineCompletionRequest {
val event = makeInlineCompletionEvent()
val file = myFixture.configureByFile(filePath)
val editor = myFixture.editor
val document = myFixture.getDocument(file)
return InlineCompletionRequest(event, file, editor, document, startOffset, endOffset)
}
fun testId() {
val service = Service()
assertEquals("completamente", service.id.id)
}
fun testIsEnabled() {
val service = Service()
assertTrue(service.isEnabled(makeInlineCompletionEvent()))
}
// This function is more about knowing I can write and run tests than about testing the service.
fun testGetSuggestion() {
val request = makeInlineCompletionRequest("text-file.txt", 0, 0)
val service = Service()
val suggestion: InlineCompletionSuggestion = runBlocking { service.getSuggestion(request) }
val variants: List<InlineCompletionVariant> = runBlocking { suggestion.getVariants() }
assertEquals(1, variants.size)
assertEquals(1, runBlocking { variants.first().elements.count() })
assertEquals("Hello World!", runBlocking { variants.first().elements.first().text })
}
}
The tests are not currently testing much, but I'm getting acquainted with the IntelliJ API, and I need to nail the basics
before moving on.
The tests are currently passing. Progress.
Next
Keeping the llama.vim code to the side, I will incrementally reproduce the implementation approach in my plugin.
The next step will be connecting to an external HTTP completion API to obtain my first real completion.
Embracing bare metal
During the rewrite of version 4 of wp-browser, I've used the chance to remove old code that was either required to support older versions of PHP and Codeception, or to support deprecated or little-used features.
Among the "victims" of this cleaning spree was the Symlinker extension: a Codeception extension provided by wp-browser that would create symbolic links from a source to a destination before tests ran.
I was rarely using the extension, always had very little feedback about it (and no data since I've never collected user and usage information in any way), and had seen it rarely used in the wild, so I removed it.
Until recently, I've always dealt with the intricacies of the file and directory structure required by WordPress (e.g., plugins in wp-content/plugins, themes in wp-content/themes) by using containers.
The nature of container bind mounts makes it easy to have plugins and themes live anywhere and just bind them in place.
If a plugin lives in /home/lucatume/vendor/some-plugin on my machine, I can "bind it in place" using a bind mount in /var/www/html/wp-content/plugins/some-plugin and call it a day.
Support for that functionality and setup is not gone from the latest versions of wp-browser, but I've started using "bare metal" solutions more and more where allowed by the nature of the project.
Nginx/Apache on Docker? No, plain PHP built-in web-server with 5 workers.
Chrome in a container? No, Chrome or Chromium already installed on my machine.
MySQL or MariaDB from a container? No, SQLite from a local file, if I can manage it.
Turns out that CI support is pretty solid as well, and wp-browser CI setup itself is much simplified by just using what is available in most CI environments now
There are situations where all the complexities of more complicated, container-based, setups are required, but that is frequently not the case.
Along with the cleaning, I've reworked wp-browser setup template, the one used when running vendor/bin/codecept init wpbrowser, to use a PHP Built-in server, SQLite and Chromium stack by default, allowing plugin, theme and site developers to be up and running in no time.
![Bare metal setup][images/bare-metal-setup.png]
Placing things
In that setup, I'm symbolically linking the plugin or theme under development in the correct location in the WordPress directory.
I wanted to make sure that symbolic link location would work both in integration and end-to-end tests, the two types of tests scaffolded by default by the template, and could not think of a better way to do it than to bring it back the Symlinker extension.
The latest version of the setup makes it clear setting up the main Codeception configuration file, codeception.yml, like this:
namespace: Tests
support_namespace: Support
paths:
tests: tests
output: tests/_output
data: tests/Support/Data
support: tests/Support
envs: tests/_envs
actor_suffix: Tester
params:
- tests/.env
extensions:
enabled:
- Codeception\Extension\RunFailed
- lucatume\WPBrowser\Extension\ChromeDriverController
- lucatume\WPBrowser\Extension\BuiltInServerController
- lucatume\WPBrowser\Extension\Symlinker
config:
lucatume\WPBrowser\Extension\ChromeDriverController:
port: '%CHROMEDRIVER_PORT%'
lucatume\WPBrowser\Extension\BuiltInServerController:
workers: 5
port: '%BUILTIN_SERVER_PORT%'
docroot: '%WORDPRESS_ROOT_DIR%'
env:
DATABASE_TYPE: sqlite
DB_ENGINE: sqlite
DB_DIR: '%codecept_root_dir%/tests/Support/Data'
DB_FILE: db.sqlite
lucatume\WPBrowser\Extension\Symlinker:
wpRootFolder: '%WORDPRESS_ROOT_DIR%'
plugins:
- .
themes: []
commands:
- lucatume\WPBrowser\Command\RunOriginal
- lucatume\WPBrowser\Command\RunAll
- lucatume\WPBrowser\Command\GenerateWPUnit
- lucatume\WPBrowser\Command\DbExport
- lucatume\WPBrowser\Command\DbImport
- lucatume\WPBrowser\Command\MonkeyCachePath
- lucatume\WPBrowser\Command\MonkeyCacheClear
- lucatume\WPBrowser\Command\DevStart
- lucatume\WPBrowser\Command\DevStop
- lucatume\WPBrowser\Command\DevInfo
- lucatume\WPBrowser\Command\DevRestart
- lucatume\WPBrowser\Command\ChromedriverUpdate
So, there and back again; the Symlinker extension is back along with an improved support for "metal" set ups and more to come in wp-browser future.
You can read more about wp-browser in its documentation page.
When I started working on the branch called version-4/minimum-compat-pass in May last year, I thought updating wp-browser to be compatible with Codeception version 5 in a minimal way would have been a matter of a few weeks.
The idea was to fix the most glaring issues of the WordPress testing framework and then move on to a more thorough refactoring of the codebase.
I was wrong.
The first pass took a year and a month and it's now available as a release candidate in the branch v4.
Not because of Codeception, but because of 9 years of code I had to Mary-Kondo through.
My love for the craft of code, which to this day is still raging, made it very difficult for me to overlook the many issues I found in the codebase and I ended up rewriting most of it.
My first commit was on June 16, 2014; I knew so little about PHP I believed all the following to be true:
- I cannot write code without PHPUnit tests or the plugin repository will reject my plugin.
- No company would work with a PHP developer that does not test her code.
- Every one uses XDebug.
There is no judgement here, I was just naive and I had no idea what I was doing.
Being a first pass, it's still not yet all the things I want it to be, but it's a good start and it is, above all, compatible with Codeception version 5, PHP 8.0, 8.1 and 8.2.
Where possible, I tried to keep back compatibility with existing tests: I'm an avid user of my own framework and I didn't want to have to rewrite all my tests to use the new version.
Some tests will need rewriting, though, and I'll try to document them as I go and find out.
Next steps are, in an order I wish I will respect but likely won't:
- Documentation update to reflect the changes in the new version.
- A new first setup experience leveraging containers for a no-hassle setup for theme, plugins and site projects.
- A new test case, provisionally called
SuperWordPressTestCase (no jokes) to leverage the new features of the framework.
I'm also planning to write a series of posts on the new features of the framework and what they bring to the table in terms of new possible use cases.
Previously
This post is part of a series of posts where I'm exploring how to use TLA+ to specify, check and then implement a PHP project that deals with concurrent processes; here are the links to the first post, the second, the third one, and the fourth one.
Fast-failure support
I'm using TLA+ to model a Loop that will be the core of a yet-to-be-implemented testing framework.
One of the features I find very useful is support for fast failure: as soon as a test fails, break out of the Loop, gracefully shut down the other running tests, and exit with an error code.
Since each test "job", whatever that will end up representing, could be running when the first failure comes is, the challenges are the graceful shutdown of the Loop and correct closing of the currently running jobs.
One of the assumptions I had coded in the specification before this step is that all worker processes would run, thus making the worker processes fair; in TLA+ terms, each process is "weakly fair".
Fast failure introduces a drastic change in that paradigm: a worker process could not run if the Loop never starts it because a previous worker's job failed.
This means I should update the worker process by removing the fair keyword from it: while it will run most of the time, it might not run at all if a previous worker job failed triggering the fast-failure handling.
After the update, though, the model checking will fail with Stuttering:
In the image above, without expanding the details of the states, this happened:
- The Loop started with two workers
- When the first worker completed the Loop started a new one (i.e., set the
_startedRegister[J3] value to TRUE).
- The Loop waits for updates from the workers.
- The last worker process started,
J3 stutters: it simply never executes because it is an "unfair" (i.e., not fair) process.
An unfair process simulates a process that might never run: it crashes or takes too much time to produce any noticeable side-effect.
This little experiment shows that making the worker processes not fair is not the right solution.
How should I translate the following from plain English to something that TLA+ would understand?
Beyond PlusCal
The language I've used to write and update the specification is called PlusCal. What I write in PlusCal is then translated into TLA+ specification language proper.
PlusCal is not the specification language; it's merely a comment when it comes to TLA+.
While PlusCal is powerful, it does not allow the kind of access I need to solve my problem: it's time to get my hands dirty.
Before applying my solution to the actual specification, I would like to experiment with a simpler and smaller specification with all the elements required by the real one, but in a more uncomplicated form.
There is a Loop process that will start worker processes, each process should run when activated, and there is the concept of the Loop completing before all worker processes have a chance to start.
Here is the heavily commented version of the PlusCal code, \* starts a comment:
----------------------------- MODULE test_goto -----------------------------
EXTENDS TLC, Integers, FiniteSets
CONSTANTS Loop, Workers, NULL
(*--algorithm test_goto
variables
started = [x \in Workers |-> FALSE],
startedCount = 0,
workTally = 0;
define
NotStartedWorkers == {x \in Workers: started[x] = FALSE}
end define;
fair process loop = Loop
variables proc = NULL;
begin
Loop_StartWorker:
proc := CHOOSE x \in NotStartedWorkers: TRUE; \* Pick the first not started worker.
startedCount := startedCount + 1;
started[proc] := TRUE; \* Mark this process started, to power NotStartedWorkers.
if startedCount /= 2 then \* Purposefully not starting the 3rd worker.
goto Loop_StartWorker;
end if;
Loop_WaitForWorkDone:
await workTally = 2; \* Two workers are done, the 3rd one wil never start.
end process;
fair process worker \in Workers begin \* If not blocked, workers should execute.
Worker_AutostartGuard: \* Always blocked action: workers cannot start on their own.
await FALSE = TRUE;
Worker_Work: \* The Loop will move workers directly here to start them.
workTally := workTally + 1;
end process;
end algorithm;*)
=============================================================================
The model I'm testing the specification with, after translation, is set up as follows:
Translating and running the model will result in a deadlock:
The Loop will start two workers out of 3; both of them blocked by a guard in the following form:
await FALSE = TRUE;
That await will permanently block the worker processes on the Worker_AutostartGuard action.
Instead of setting a flag in a register shared by Loop and workers to coordinate and stimulate the Loop starting the processes, the _startedRegister variable in the specification, I want the Loop to start the workers by moving them to the Worker_Work action directly.
Before I show the code, I would like to make a note about how PlusCal code is translated.
In the TLA+ translation, there is no concept of "processes"; there are simply states the system will transit to that, for convenience of writing, PlusCal will wrap in processes. The TLA+ specification will be one giant state machine, and the model checker will move from state to state, respecting the available nodes.
When translated from PlusCal to the TLA+ language, the translation will add a pc variable that will map each process to its next state; it can be seen in the image above, initially set to this:
<Initial Predicate>
pc
W1 -> Worker_AutostartGuard
W2 -> Worker_AutostartGuard
W3 -> Worker_AutostartGuard
Loop -> Loop_StartWorker
My thinking is that I could move a worker to the Worker_Work phase by setting its entry in the pc variable to Worker_Work to make it look something like this:
<Initial Predicate>
pc
W1 -> Worker_Work
W2 -> Worker_AutostartGuard
W3 -> Worker_AutostartGuard
Loop -> Loop_StartWorker
I've updated my PlusCal code to do that:
----------------------------- MODULE test_goto -----------------------------
EXTENDS TLC, Integers, FiniteSets
CONSTANTS Loop, Workers, NULL
(*--algorithm test_goto
variables
started = [x \in Workers |-> FALSE],
startedCount = 0,
workTally = 0;
define
NotStartedWorkers == {x \in Workers: started[x] = FALSE}
end define;
fair process loop = Loop
variables proc = NULL;
begin
Loop_StartWorker:
proc := CHOOSE x \in NotStartedWorkers: TRUE; \* Pick the first not started worker.
startedCount := startedCount + 1;
started[proc] := TRUE; \* Mark this process started, to power NotStartedWorkers.
pc[proc] := "Worker_Work"; \* Move the worker process to the "Worker_Work" action directly.
if startedCount /= 2 then \* Purposefully not starting the 3rd worker.
goto Loop_StartWorker;
end if;
Loop_WaitForWorkDone:
await workTally = 2; \* Two workers are done, the 3rd one wil never start.
end process;
fair process worker \in Workers begin \* If not blocked, workers should execute.
Worker_AutostartGuard: \* Always blocked action: workers cannot start on their own.
await FALSE = TRUE;
Worker_Work: \* The Loop will move workers directly here to start them.
workTally := workTally + 1;
end process;
end algorithm;*)
=============================================================================
Except that is not allowed, and the translation will fail with the message:
Unrecoverable error:
-- Multiple assignments to pc
I've not done multiple assignments to pc, but the translation did.
Looking at the translated code, I can see the pc variable is modified in each state, its next value (indicated by pc') updated using EXCEPT.
As an example, this is the translation of the Worker_Work action:
Worker_Work(self) == /\ pc[self] = "Worker_Work"
/\ workTally' = workTally + 1
/\ pc' = [pc EXCEPT ![self] = "Done"]
/\ UNCHANGED << started, startedCount, proc >>
The syntax means "when the model checker will run the Worker_Work action, then the next state will be this."; what the model checker will do next is check all the available actions and pick one whose pre-conditions are met and execute it.
In plain English, the above means:
if the action I (a worker) should execute is the "Worker_Work" one
then the next value of workTally will be workTally + 1
and the next value of pc (the program counter) for myself will be Done
and started, startedCount and proc will not change.
Remember /\ is a logic AND, var' = <value> is an assignment applied to the next state, and self means the process itself, used as a key.
The string pc' = [pc EXCEPT ![self] = "Done"] means "the next state of pc is like pc except for the key self that will have a value of Done"; the syntax is a bit weird.
The words "action" and "state" should ring a bell from store-pattern implementations like Redux. It's an excellent concept to keep in mind and wrap my head around the fact that each PlusCal label (e.g., the Worker_Work one) will become an action (an operator, a function in TLA+ terms) that will take a state as an input and return the next state as an output.
Since each translation of a PlusCal label will update the next state of the pc variable, I cannot do that in the context of PlusCal.
Why not allow updates of the variables more than once? I would not know for sure, I did not write the TLA+ model checker, but I can guess to avoid the next state being a function of itself.
Messing with the translation
I've commented the problematic parts of the PlusCal code and translated it to get the bulk of it set up.
In the translation, I've commented the original code out and added the updated version right after it.
----------------------------- MODULE test_goto -----------------------------
EXTENDS TLC, Integers, FiniteSets
CONSTANTS Loop, Workers, NULL
(*--algorithm test_goto
variables
started = [x \in Workers |-> FALSE],
startedCount = 0,
workTally = 0;
define
NotStartedWorkers == {x \in Workers: started[x] = FALSE}
end define;
fair process loop = Loop
variables proc = NULL;
begin
Loop_StartWorker:
proc := CHOOSE x \in NotStartedWorkers: TRUE; \* Pick the first not started worker.
startedCount := startedCount + 1;
started[proc] := TRUE; \* Mark this process started, to power NotStartedWorkers.
\* pc[proc] := "Worker_Work"; >>> TODO IN THE TRANSLATION.
if startedCount /= 2 then \* Purposefully not starting the 3rd worker.
goto Loop_StartWorker;
end if;
Loop_WaitForWorkDone:
await workTally = 2; \* Two workers are done, the 3rd one wil never start.
\* pc := [x \in DOMAIN pc |-> "Done"] >>> TODO IN THE TRANSLATION.
end process;
fair process worker \in Workers begin \* If not blocked, workers should execute.
Worker_AutostartGuard: \* Always blocked action: workers cannot start on their own.
await FALSE = TRUE;
Worker_Work: \* The Loop will move workers directly here to start them.
workTally := workTally + 1;
end process;
end algorithm;*)
\* BEGIN TRANSLATION (chksum(pcal) \in STRING /\ chksum(tla) \in STRING)
VARIABLES started, startedCount, workTally, pc
(* define statement *)
NotStartedWorkers == {x \in Workers: started[x] = FALSE}
VARIABLE proc
vars == << started, startedCount, workTally, pc, proc >>
ProcSet == {Loop} \cup (Workers)
Init == (* Global variables *)
/\ started = [x \in Workers |-> FALSE]
/\ startedCount = 0
/\ workTally = 0
(* Process loop *)
/\ proc = NULL
/\ pc = [self \in ProcSet |-> CASE self = Loop -> "Loop_StartWorker"
[] self \in Workers -> "Worker_AutostartGuard"]
Loop_StartWorker == /\ pc[Loop] = "Loop_StartWorker"
/\ proc' = (CHOOSE x \in NotStartedWorkers: TRUE)
/\ startedCount' = startedCount + 1
/\ started' = [started EXCEPT ![proc'] = TRUE]
/\ IF startedCount' /= 2
\* THEN /\ pc' = [pc EXCEPT ![Loop] = "Loop_StartWorker"]
\* ELSE /\ pc' = [pc EXCEPT ![Loop] = "Loop_WaitForWorkDone"]
THEN /\ pc' = [pc EXCEPT ![Loop] = "Loop_StartWorker", ![proc'] = "Worker_Work"]
ELSE /\ pc' = [pc EXCEPT ![Loop] = "Loop_WaitForWorkDone", ![proc'] = "Worker_Work"]
/\ UNCHANGED workTally
Loop_WaitForWorkDone == /\ pc[Loop] = "Loop_WaitForWorkDone"
/\ workTally = 2
\* /\ pc' = [pc EXCEPT ![Loop] = "Done"]
/\ pc' = [x \in DOMAIN pc |-> "Done"]
/\ UNCHANGED << started, startedCount, workTally, proc >>
loop == Loop_StartWorker \/ Loop_WaitForWorkDone
Worker_AutostartGuard(self) == /\ pc[self] = "Worker_AutostartGuard"
/\ FALSE = TRUE
/\ pc' = [pc EXCEPT ![self] = "Worker_Work"]
/\ UNCHANGED << started, startedCount,
workTally, proc >>
Worker_Work(self) == /\ pc[self] = "Worker_Work"
/\ workTally' = workTally + 1
/\ pc' = [pc EXCEPT ![self] = "Done"]
/\ UNCHANGED << started, startedCount, proc >>
worker(self) == Worker_AutostartGuard(self) \/ Worker_Work(self)
(* Allow infinite stuttering to prevent deadlock on termination. *)
Terminating == /\ \A self \in ProcSet: pc[self] = "Done"
/\ UNCHANGED vars
Next == loop
\/ (\E self \in Workers: worker(self))
\/ Terminating
Spec == /\ Init /\ [][Next]_vars
/\ WF_vars(loop)
/\ \A self \in Workers : WF_vars(worker(self))
Termination == <>(\A self \in ProcSet: pc[self] = "Done")
\* END TRANSLATION
=============================================================================
In the Loop_StartWorker and Loop_WaitForWorkDone actions, I'm updating more than one entry of the pc' map, modeling the idea that the Loop will "start" the worker processes correctly without using a register and avoiding unterminated worker processes by setting the next stage of any process to "Done" in the Loop_WaitForWorkDone phase.
Checking the model for Termination and Deadlock will pass this time:
Next
In my next post, I will apply this newfound "tool" to the actual specification I care about and keep working on it to move it toward its final version.
Previously
This post is part of a series of posts where I'm exploring how to use TLA+ to specify, check and then implement a PHP project that deals with concurrent processes; here are the links to the first post, the second, and the third one.
Checking for output
In the previous post, I have completed a specification of the Loop that would pass model checking in different scenarios with more jobs than parallelism, more parallelism than jobs, and, finally, more of a "serial" case with a parallelism of 1.
I've added two more models to check:
- 3 jobs, parallelism 3
- 1 job, parallelism 1
The specification modified as outlined in the third post passes all model checks without any change.
There is something fundamental to the Loop correct execution that I'm currently not checking: the Loop should collect all the output emitted by the Workers.
In the specification, I've represented the output emitted by the workers as a * char appended to a sequence. I'm keeping that over-simplification in place and leveraging that by ensuring the amount of * chars collected by the Loop is the same emitted by the workers.
Since the workers use an Inter-process Communication pipe to share their output with the Loop, I want to make another assertion that all worker pipes are drained by the end of the Loop.
I've done the first update to the specification to support this new check:
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC, Integers, FiniteSets, Sequences
CONSTANTS Loop, JobSet, Parallelism, NULL
(*--algorithm loop
variables
jobsCount = Cardinality(JobSet),
_startedRegister = [x \in JobSet |-> FALSE],
_processStatusRegister = [x \in JobSet |-> NULL],
_processPipesRegister = [x \in JobSet |-> <<>>],
_emittedOutputLen = 0,
_collectedOutputLen = 0;
define
StartedCount == Cardinality({x \in DOMAIN _startedRegister: _startedRegister[x] = TRUE})
NextNotStarted == CHOOSE x \in DOMAIN _startedRegister: _startedRegister[x] = FALSE
CompletedCount == Cardinality({x \in JobSet: _processStatusRegister[x] /= NULL})
Running == StartedCount - CompletedCount
ParallelismRespected == Running <= Parallelism
StreamSelectUpdates == {x \in JobSet: _processPipesRegister[x] /= <<>>}
end define;
fair process loop = Loop
variables
streamSelectUpdates = {},
updatedProcess = NULL,
processStatus = NULL,
processToOutputMap = [x \in JobSet |-> <<>>],
processToExitStatusMap = [x \in JobSet |-> NULL];
begin
StartInitialBatch:
while StartedCount < Parallelism /\ StartedCount < jobsCount do
with p = NextNotStarted do
_startedRegister[p] := TRUE;
end with;
end while;
WaitForStreamUpdates:
streamSelectUpdates := StreamSelectUpdates;
await Cardinality(StreamSelectUpdates) > 0;
HandleStreamUpdates:
while Cardinality(streamSelectUpdates) > 0 do
updatedProcess := CHOOSE x \in streamSelectUpdates: TRUE;
streamSelectUpdates := streamSelectUpdates \ {updatedProcess};
GetProcessOutput:
with processOutput = _processPipesRegister[updatedProcess] do
processToOutputMap[updatedProcess] := Append(processToOutputMap[updatedProcess], processOutput);
_collectedOutputLen := _collectedOutputLen + Len(processOutput);
end with;
_processPipesRegister[updatedProcess] := <<>>;
GetProcessStatus:
processStatus := _processStatusRegister[updatedProcess];
UpdateTrackedProcessPipes:
if processStatus /= NULL then
processToExitStatusMap[updatedProcess] := processStatus;
end if;
CheckLoopStatus:
if processToExitStatusMap[updatedProcess] /= NULL then
MaybeStartOneMore:
if StartedCount < jobsCount /\ Running < Parallelism then
with p = NextNotStarted do
_startedRegister[p] := TRUE;
end with;
goto WaitForStreamUpdates;
end if;
end if;
CheckAllDone:
if Cardinality({x \in JobSet: processToExitStatusMap[x] /= NULL}) = jobsCount then
assert(_collectedOutputLen = _emittedOutputLen);
goto Done;
else
goto WaitForStreamUpdates;
end if;
end while;
end process
fair process worker \in JobSet
begin
WaitToStart:
await _startedRegister[self] = TRUE;
Work:
either
_processPipesRegister[self] := Append(_processPipesRegister[self], "*");
_emittedOutputLen := _emittedOutputLen + 1
or
skip;
end either;
ExitStatus:
either
_processPipesRegister[self] := Append(_processPipesRegister[self], "*");
_emittedOutputLen := _emittedOutputLen + 1;
_processStatusRegister[self] := 0;
goto Done;
or
_processPipesRegister[self] := Append(_processPipesRegister[self], "*");
_emittedOutputLen := _emittedOutputLen + 1;
_processStatusRegister[self] := 1;
goto Done;
end either;
end process;
end algorithm;*)
\* BEGIN TRANSLATION
\* [...]
\* END TRANSLATION
OneWorkerPerJobStarted == <>[](StartedCount = Cardinality(JobSet))
AllWorkersCompleted == <>[](CompletedCount = Cardinality(JobSet))
=============================================================================
To note in the specification:
- I'm using the
_emittedOutputLen to store the length of the output generated by the workers; only the worker processes will update it.
- The
_collectedOutputLen global variable stores the length of the output collected by the Loop process.
- I've not added any Invariant Property to my model
The specification is failing when checked against a model:
The error message reads as follows (I'm omitting the lines):
The first argument of Assert evaluated to FALSE; the second argument was:
"Failure of assertion at line 69, column 57."
The error occurred when TLC was evaluating the nested
expressions at the following positions: ...
The failing assertion is the one I've added in the Loop CheckAllDone phase; before moving to the Done phase of the Loop, I want to make sure the loop collected all the emitted output.
Instead of using a temporal property, I'm using an assertion. The reason for this approach is that a temporal property would be an "eventually, then always" one.
But the length of the emitted and collected output would be 0 at the start of the specification, and it would be invalidated now and then as the Loop and the workers alternate emitting and collecting it. Changing the temporal condition to just "eventually" would not guarantee me that is true at the end, when the Loop is completed.
And "when the loop is done" is precisely the only moment I want to make this check, so an assertion makes sense.
That assertion fails, telling me the current specification is not correctly collecting all the output emitted by the workers.
Another pass on the specification adds a step in the CheckLoopStatus phase to get the output the process might have emitted before exiting. This update will make the specification pass across all models:
"`tla
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC, Integers, FiniteSets, Sequences
CONSTANTS Loop, JobSet, Parallelism, NULL
(*--algorithm loop
variables
jobsCount = Cardinality(JobSet),
_startedRegister = [x \in JobSet |-> FALSE],
_processStatusRegister = [x \in JobSet |-> NULL],
_processPipesRegister = [x \in JobSet |-> <<>>],
_emittedOutputLen = 0,
_collectedOutputLen = 0;
define
StartedCount == Cardinality({x \in DOMAIN _startedRegister: _startedRegister[x] = TRUE})
NextNotStarted == CHOOSE x \in DOMAIN _startedRegister: _startedRegister[x] = FALSE
CompletedCount == Cardinality({x \in JobSet: _processStatusRegister[x] /= NULL})
Running == StartedCount - CompletedCount
ParallelismRespected == Running <= Parallelism
StreamSelectUpdates == {x \in JobSet: _processPipesRegister[x] /= <<>>}
end define;
macro startProcess() begin
with p = NextNotStarted do
_startedRegister[NextNotStarted] := TRUE;
end with;
end macro;
macro collectProcessOutput(p, processToOutputMap) begin
with processOutput = _processPipesRegister[updatedProcess] do
processToOutputMap[updatedProcess] := Append(processToOutputMap[updatedProcess], processOutput);
_collectedOutputLen := _collectedOutputLen + Len(processOutput);
end with;
_processPipesRegister[updatedProcess] := <<>>;
end macro;
macro emitOutput(p, output) begin
_processPipesRegister[p] := Append(_processPipesRegister[p], output);
_emittedOutputLen := _emittedOutputLen + 1;
end macro;
macro exitWithStatus(p, status) begin
_processStatusRegister[p] := status;
end macro;
fair process loop = Loop
variables
streamSelectUpdates = {},
updatedProcess = NULL,
processStatus = NULL,
processToOutputMap = [x \in JobSet |-> <<>>],
processToExitStatusMap = [x \in JobSet |-> NULL];
begin
StartInitialBatch:
while StartedCount < Parallelism /\ StartedCount < jobsCount do
startProcess();
end while;
WaitForStreamUpdates:
streamSelectUpdates := StreamSelectUpdates;
await Cardinality(StreamSelectUpdates) > 0;
HandleStreamUpdates:
while Cardinality(streamSelectUpdates) > 0 do
updatedProcess := CHOOSE x \in streamSelectUpdates: TRUE;
streamSelectUpdates := streamSelectUpdates \ {updatedProcess};
GetProcessOutput:
collectProcessOutput(updatedProcess, processToOutputMap);
GetProcessStatus:
processStatus := _processStatusRegister[updatedProcess];
UpdateTrackedProcessPipes:
if processStatus /= NULL then
processToExitStatusMap[updatedProcess] := processStatus;
end if;
CheckLoopStatus:
if processToExitStatusMap[updatedProcess] /= NULL then
GetProcessExitOutput:
collectProcessOutput(updatedProcess, processToOutputMap);
MaybeStartOneMore:
if StartedCount < jobsCount /\ Running < Parallelism then
startProcess();
goto WaitForStreamUpdates;
end if;
end if;
CheckAllDone:
if Cardinality({x \in JobSet: processToExitStatusMap[x] /= NULL}) = jobsCount then
assert(_collectedOutputLen = _emittedOutputLen);
goto Done;
else
goto WaitForStreamUpdates;
end if;
end while;
end process
fair process worker \in JobSet
begin
WaitToStart:
await _startedRegister[self] = TRUE;
Work:
either
emitOutput(self, "");
or
skip;
end either;
ExitStatus:
either
emitOutput(self, "");
exitWithStatus(self, 0);
goto Done;
or
_processPipesRegister[self] := Append(_processPipesRegister[self], "*");
_emittedOutputLen := _emittedOutputLen + 1;
_processStatusRegister[self] := 1;
goto Done;
end either;
end process;
end algorithm;*)
* BEGIN TRANSLATION
* [...]
* END TRANSLATION
OneWorkerPerJobStarted == <>[](StartedCount = Cardinality(JobSet))
AllWorkersCompleted == <>[](CompletedCount = Cardinality(JobSet))
Besides putting in place the fix to make sure the specification will pass all models check, I've also refactored the code to extract duplicated code into `macro's: `startProcess`, `collectProcessOutput`, `emitOutput`, and `exitWithStatus`.
What each of them does is pretty easy to understand, and macros work like functions. For the most: the one exception is macros will only be able to change the value of variables that are either local to the macro (but this is pretty common in any programming language) or that are passed to the macros. Once this is taken care of, macros help reduce the verbosity and duplication of the code a bit.
### PHP translation
There are more features I want my Loop-based code to support, but it's worth trying to translate that into PHP code before I move on and find myself having to write too complicated code.
```php
<?php
class Loop
{
private $jobs;
private $jobsCount;
protected $parallelism;
private $procs = [];
private $startedCount = 0;
private $runningCount = 0;
private $readStdoutStreams = [];
private $readStderrStreams = [];
private $jobToExitStatusMap = [];
private $jobToStdoutContents = [];
private $jobToStderrContents = [];
public function __construct($jobs, $parallelism)
{
$this->jobs = $jobs;
$this->jobsCount = count($jobs);
$this->parallelism = $parallelism;
reset($this->jobs);
}
private function collectProcessOutput(stdClass $proc)
{
if (!isset($this->jobToStdoutContents[$proc->job])) {
$this->jobToStdoutContents[$proc->job] = '';
}
if (!isset($this->jobToStderrContents[$proc->job])) {
$this->jobToStderrContents[$proc->job] = '';
}
$this->jobToStdoutContents[$proc->job] .= stream_get_contents($proc->stdoutStream);
$this->jobToStderrContents[$proc->job] .= stream_get_contents($proc->stderrStream);
}
private function startWorker()
{
$job = current($this->jobs);
next($this->jobs);
$command = sprintf("%s %s %s", PHP_BINARY, escapeshellarg(__FILE__), $job);
$desc = [
0 => ['pipe', 'r'], // Proc STDIN.
1 => ['pipe', 'w'], // Proc STDOUT.
2 => ['pipe', 'w'], // Proc STDERR.
];
$procHandle = proc_open($command, $desc, $pipes, null, null, ['bypass_shell']);
$procStdout = $pipes[1];
$procStderr = $pipes[2];
$this->procs[] = (object) [
'procHandle' => $procHandle, 'stdoutStream' => $procStdout, 'stderrStream' => $procStderr, 'job' => $job
];
$this->startedCount++;
$this->runningCount++;
$this->readStdoutStreams[] = $procStdout;
$this->readStderrStreams[] = $procStderr;
}
private function getProcFromStream($stream)
{
foreach ($this->procs as $key => $proc) {
if ($proc->stdoutStream === $stream || $proc->stderrStream === $stream) {
return $proc;
}
}
return null;
}
public function run()
{
// StartInitialBatch
while ($this->startedCount < $this->parallelism && $this->startedCount < $this->jobsCount) {
$this->startWorker();
}
while (true) {
//WaitForStreamUpdates
// Reset this on each run to make sure we only wait for updates from active process streams.
$readStreams = array_merge($this->readStdoutStreams, $this->readStderrStreams);
$read = $readStreams;
$write = [];
$except = [];
$streamUpdates = stream_select($read, $write, $except, 10);
if (!$streamUpdates) {
continue;
}
// HandleStreamUpdates
$skipRead = [];
foreach ($read as $index => $stream) {
if (isset($skipRead[$index])) {
// This stream was coupled with an already read one, it's been read already.
continue;
}
// GetProcessOutput
$proc = $this->getProcFromStream($stream);
$this->collectProcessOutput($proc);
// GetProcessStatus
$procStatus = proc_get_status($proc->procHandle);
if (!$procStatus['running']) {
$this->jobToExitStatusMap[$proc->job] = $procStatus['exitcode'];
// UpdateTrackedProcessPipes
// We'll empty the other stream now, skip that.
$otherStreamIndex = in_array($proc->stdoutStream, $this->readStdoutStreams, true) ?
array_search($proc->stderrStream, $readStreams, true)
: array_search($proc->stdoutStream, $readStreams, true);
$skipRead[$otherStreamIndex] = true;
--$this->runningCount;
$this->readStdoutStreams = array_diff($this->readStdoutStreams, [$proc->stdoutStream]);
$this->readStderrStreams = array_diff($this->readStderrStreams, [$proc->stderrStream]);
// GetProcessExitOutput
$this->collectProcessOutput($proc);
// MaybeStartOneMore
if ($this->startedCount < $this->jobsCount && $this->runningCount < $this->parallelism) {
$this->startWorker();
}
}
//CheckAllDone
if (count($this->jobToExitStatusMap) === $this->jobsCount) {
break 2;
}
}
}
return array_map(function ($job) {
return [
'job' => $job,
'stdout' => $this->jobToStdoutContents[$job],
'sterr' => $this->jobToStderrContents[$job],
'exitStatus' => $this->jobToExitStatusMap[$job]
];
}, $this->jobs);
}
}
function worker()
{
do {
$stdoutOrStderr = mt_rand(0, 1);
if ($stdoutOrStderr === 0) {
fwrite(STDOUT, "*", 1);
} else {
fwrite(STDERR, "*", 1);
}
$action = mt_rand(0, 3);
} while ($action > 0);
$exitStatus = mt_rand(0, 1);
exit($exitStatus);
}
if (!isset($argv[1])) {
// Start the loop.
$loop = new Loop(range(1, 5), 2);
echo json_encode($loop->run(), JSON_PRETTY_PRINT);
} else {
// Handle a worker request.
worker();
}
I've tried to port over, as comment blocks, the labels that would be in the specification.
Running the script will print output similar to this on the screen:
» php test-loop-02.php
[
{
"job": 1,
"stdout": "**",
"sterr": "******",
"exitStatus": 1
},
{
"job": 2,
"stdout": "",
"sterr": "*",
"exitStatus": 1
},
{
"job": 3,
"stdout": "***",
"sterr": "*****",
"exitStatus": 1
},
{
"job": 4,
"stdout": "**",
"sterr": "",
"exitStatus": 0
},
{
"job": 5,
"stdout": "*",
"sterr": "***",
"exitStatus": 0
}
Nothing to be too excited about, but enough to demonstrate the Loop component works as intended.
I've gone for clarity and verbosity over cleverness; it could be polished further; I'm not doing that now as there are more features I want to add to the specification that will need at least a further iteration over the code.
The most significant differences between the PHP code and the specification are:
- In PHP, `resource's cannot be used as keys to arrays, so I've added some indirect maps to deal with that.
- Again, in PHP, processes will emit output on the
STDOUT and the STDERR streams. To account for that, the Loop will read from both streams. I've taken some additional care to avoid calling proc_get_status twice on a terminated process, as the second call will return a -1 and not the actual exit code.
Besides the implementation differences, the structure is the same.
I've not gone as far as to use goto in the PHP code, though, and relied on regular loops.
The code lacks several security features like error handling and closing of the streams; it conveys the idea good enough as it is.
Next
In the next post, I will be adding fast-failure support to the Loop and start on a shared resources lock sharing system.
Previously
In the first post of the series, I've introduced the project I'm trying to complete using TLA+ and its specification verification power.
In the second post, I've started moving on with the specification first and the PHP implementation second.
This prior art contains insight into my thinking process and flow and provides a longer description of what I'm trying to build.
Pipes
My last iteration on the specification and implementation did nail the part where the main PHP thread, the Loop, dispatches the processing of each test block (whatever that will end up being) to separate PHP processes.
The issue with the last implementation was CPU-locking, all the possible mitigations of it (see the first post), and the lack of communication between the Loop thread and the worker threads.
Both issues can be solved using PHP stream_select function: the function, when provided a set of streams to observe, will block the execution of the PHP script until at least one of them is updated.
Why is this important? Because translating TLC's await keyword into PHP code that will actually wait, not poll on a CPU-locked cycle, cannot be achieved with many functions in PHP.
Furthermore, the stream_select function will deal with the problem of the Inter-Process Communication (IPC from now on), allowing the Loop to get "push updates" from the workers.
Pipes specification
I've modeled this new approach in the specification below:
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC, Integers, FiniteSets, Sequences
CONSTANTS Loop, JobSet, Parallelism, NULL
(*--algorithm loop
variables
jobsCount = Cardinality(JobSet),
_startedRegister = [x \in JobSet |-> FALSE],
_processStatusRegister = [x \in JobSet |-> NULL],
_processPipesRegister = [x \in JobSet |-> <<>>];
define
StartedCount == Cardinality({x \in DOMAIN _startedRegister: _startedRegister[x] = TRUE})
NextNotStarted == CHOOSE x \in DOMAIN _startedRegister: _startedRegister[x] = FALSE
CompletedCount == Cardinality({x \in JobSet: _processStatusRegister[x] /= NULL})
Running == StartedCount - CompletedCount
ParallelismRespected == Running <= Parallelism
StreamSelectUpdates == {x \in JobSet: _processPipesRegister[x] /= <<>>}
end define;
fair process loop = Loop
variables
streamSelectUpdates = {},
updatedProcess = NULL,
processStatus = NULL,
processToOutputMap = [x \in JobSet |-> <<>>],
processToExitStatusMap = [x \in JobSet |-> NULL];
begin
StartInitialBatch:
while StartedCount < Parallelism do
with p = NextNotStarted do
_startedRegister[p] := TRUE;
end with;
end while;
WaitForStreamUpdates:
\* Collect the live value into a copy.
streamSelectUpdates := StreamSelectUpdates;
await Cardinality(StreamSelectUpdates) > 0;
HandleStreamUpdates:
while Cardinality(streamSelectUpdates) > 0 do
\* A hack to pop the Jobs from the updates one by one.
updatedProcess := CHOOSE x \in streamSelectUpdates: 1 > 0;
streamSelectUpdates := streamSelectUpdates \ {updatedProcess};
GetProcessOutput:
with processOutput = _processPipesRegister[updatedProcess] do
processToOutputMap[updatedProcess] := Append(processToOutputMap[updatedProcess], processOutput);
end with;
_processPipesRegister[updatedProcess] := <<>>;
GetProcessStatus:
processStatus := _processStatusRegister[updatedProcess];
UpdateTrackedProcessPipes:
if processStatus /= NULL then
processToExitStatusMap[updatedProcess] := processStatus;
end if;
CheckLoopStatus:
if processToExitStatusMap[updatedProcess] /= NULL then
MaybeStartOneMore:
if StartedCount < jobsCount then
with p = NextNotStarted do
_startedRegister[p] := TRUE;
end with;
goto WaitForStreamUpdates;
end if;
end if;
CheckAllDone:
if Cardinality({x \in JobSet: processToExitStatusMap[x] /= NULL}) = jobsCount then
goto Done;
else
goto WaitForStreamUpdates;
end if;
end while;
end process
fair process worker \in JobSet
begin
WaitToStart:
await _startedRegister[self] = TRUE;
Work:
either
_processPipesRegister[self] := Append(_processPipesRegister[self], "*");
or
skip;
end either;
ExitStatus: \* Either exit 0 or 1, a process MUST produce output.
either
_processPipesRegister[self] := Append(_processPipesRegister[self], "*");
_processStatusRegister[self] := 0;
goto Done;
or
_processPipesRegister[self] := Append(_processPipesRegister[self], "*");
_processStatusRegister[self] := 1;
goto Done;
end either;
end process;
end algorithm;*)
\* BEGIN TRANSLATION
\* [...]
\* END TRANSLATION
OneWorkerPerJobStarted == <>[](StartedCount = Cardinality(JobSet))
AllWorkersCompleted == <>[](CompletedCount = Cardinality(JobSet))
=============================================================================
Before unpacking the code, a successful run with the following settings:
What to check? - Temporal formulaDeadlock - checkedProperties - something that should eventually be true.
* `Termination` - all processes will always eventually terminate.
* `OneWorkerPerJobStarted` - each job will, eventually, have a worker assigned to it.
* `AllWorkersCompleted` - all workers should eventually complete.
Invariants - citing the IDE "Formulas true in every reachable state".
* `ParallelismRespected` - There should never be a state with more workers than parallelism would allow.
The constants are set up as follows:
JobSet <- [ model value ] {J1, J2, J3}Parallelism <- 2
Registers
The registers, prefixed with a _, are artifacts of the specification that will not make it into code: they represent, utilizing a global variable, the inter-process communication between the Loop and the workers.
To "start" the processes, I'm using the same approach used in the previous post: a _startedRegister variable that maps Jobs to their having a Worker started for them.
This specification global variable is the equivalent of the proc_open PHP function.
Where the proc_open function returns a process resource handle of the resource type, the specification will set the flag indicating a worker started for a job in the _startedRegister variable to TRUE.
The _processPipesRegister variable keeps track of the output emitted by the workers on the write end of the IPC pipe assigned to them.
The proc_open function will create, contextually with starting the process in a separate thread, "pipes" for inter-process communication. Typically one to communicate with the process, the STDIN one, and two to get output from the process: the STDOUT and STDERR ones. For the sake of the specification, I conflated the two pipes into one, and the IPC is represented by the worker process writing the _processPipesRegister global variable, and the Loop process reading from it.
The last register is the _processStatusRegister and represents what a call to the proc_get_status function would return in PHP: the exit status of the worker processes.
More cases mean more models
The specification passes when I use 3 jobs and parallelism of 2, but what about other cases?
I found out a "data-provider-like" approach to TLA+ is not how I should use the tools. To test different cases, I should create a new model. Better: clone the one I used so far and change the value of the constants.
The first alternate model is to test if the Loop specification works in "serial" mode: jobs are executed one after the other, with a parallelism of 1.
Nothing changes from previous checks, if not the constants:
JobSet <- [ model value ] {J1, J2, J3}Parallelism <- 1
And it fails, the ParallelismRespected invariant violated at some point.
To debug the specification failure, I find it extremely useful getting, first, a view of the run, using the UI control to collapse all I can get an idea of where the violation happened (the MaybeStartOneMore phase of the Loop process) and what sequence led the run to that.
The first guiding light to find the cause of the model checking failure is that values changed by the state are highlighted in red; the second is that invariants and (temporal) properties will run after the state has changed the values.
After the MaybeStartOneMore state ran, the ParallelismRespected property is checked and will find 3 started processes in the _startedRegister variable, only 1 of which had its exit status collected and thus was deemed as completed. So, at this stage, there are 2 processes running which is more than the allowed parallelism of 1.
The fix is easy enough: in the MaybeStartOneMore phase, I have to check two conditions:
- There are still jobs that do not have workers assigned; the specification was already checking this.
- And the number of running workers is less than, or equal to, the parallelism; this check is the missing one.
Translated in PlusCal, the MaybeStartOneMore phase will change as follows:
MaybeStartOneMore:
- if StartedCount < jobsCount then
+ if StartedCount < jobsCount /\ Running < Parallelism then
with p = NextNotStarted do
_startedRegister[p] := TRUE;
end with;
goto WaitForStreamUpdates;
end if;
Now both models will pass.
More parallelism than jobs
The last model I want to check against the specification is the one where the parallelism is more than the number of jobs.
Clone another model and set:
JobSet <- [ model value ] {J1, J2}Parallelism <- 3
And this one fails again; this time, it's a TLC syntax violation.
The message:
TLC threw an unexpected exception.
This was probably caused by an error in the spec or model.
See the User Output or TLC Console for clues to what happened.
The exception was a java.lang.RuntimeException
: Attempted to compute the value of an expression of form
CHOOSE x \in S: P, but no element of S satisfied P.
line 102, col 19 to line 102, col 83 of module spec_loop
The error occurred when TLC was evaluating the nested
expressions at the following positions:
0. Line 131, column 22 to line 142, column 62 in spec_loop
1. Line 131, column 25 to line 131, column 54 in spec_loop
2. Line 132, column 25 to line 137, column 61 in spec_loop
3. Line 133, column 33 to line 135, column 82 in spec_loop
4. Line 133, column 36 to line 134, column 94 in spec_loop
5. Line 134, column 38 to line 134, column 94 in spec_loop
6. Line 134, column 58 to line 134, column 94 in spec_loop
7. Line 134, column 85 to line 134, column 85 in spec_loop
8. Line 133, column 45 to line 133, column 58 in spec_loop
9. Line 102, column 19 to line 102, column 83 in spec_loop
Three things to note:
- The line and columns number refer to the translation, not the PlusCal code. So, better learn to read that.
- The message is pretty clear about the failure but will use symbols.
- TLA+ does not handle the concept of returning anything.
When I say that the error report will use symbols, I mean that I did not write CHOOSE x \in S: P, but no element of S satisfied P anywhere in my PlusCal code, and that is nowhere to be found in the translation.
That error is about this line of the PlusCal code:
NextNotStarted == CHOOSE x \in DOMAIN _startedRegister: _startedRegister[x] = FALSE
And when I say that TLA+ does not handle returning anything, I mean that it does not work as PHP would.
If I had to write the same code in PHP I would translate the code above into the following:
$_startedRegister = ['J1' => true, 'J2' => true];
$nextNotStarted = array_filter($_startedRegister, function($started){ return !$started; });
Next
In the next post, I will make the specification more robust adding more invariants and properties to, finally, move into the PHP translation of it.
Previously
The previous post on the subject introduced the project I'm working on and why I've decided to use TLA+.
This post will take off from the end of that to complicate my specification to include something closer to what I need to implement.
Atomic labels
Closing the first post, I've said that a "step" in TLA+ is marked by a "label" and is "atomic".
In the context of a process block, a label is anything that looks like This: (alpha and then a :). A label has a body of statements that happen inside it that will occur atomically, in the order they appear.
The best way I can explain it is with an example:
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC, Integers
(*--algorithm loop
variables
accumulator = 0; \* Start with a value of 0.
define
AccumlatorLessThanEqualOne == accumulator <= 1 \* We'll use this as invariant.
end define;
fair process p \in {1,2} \* 2 processes will run at the same time.
begin
Increase_Accumulator:
accumulator := accumulator + 1;
Decrease_Accumulator:
accumulator := accumulator - 1;
end process
end algorithm;*)
=============================================================================
I've omitted the translation of the PlusCal code to TLC as it's machine-generated and not that interesting.
I've set up my model to this:
What to check? - Temporal formulaDeadlock - checkedProperties - something that should eventually be true.
* `Termination`, both processes should terminate.
Invariants - citing the IDE "Formulas true in every reachable state".
* `AccumlatorLessThanEqualOne` - the value of the `accumulator` variable should be <= 1
The last part, the one where I set the AccumlatorLessThanEqualOne invariant, is the one that will make the model fail.
The model will fail because there is at least one state where the value of accumulator is not <= 1.
The TLA+ will report on such state:
The UI requires highlights in red what changed between a step and the next:
- At the
Initial predicate, the start of the model checking, the accumulator value is 0, both processes will execute the Increase_Accumulator step next, the AccumlatorLessThanEqualOne invariant is not violated.
- The model checking picks process 2 to execute its
Increase_Accumulator step: that will increase accumulator by 1. All fine and dandy.
- The model checking will, then, pick process 1 to execute, and that too will complete the
Increase_Accumulator step, adding 1 to the accumulator and violating, thus, the AccumlatorLessThanEqualOne invariant.
The model checker output reads Invariant AccumlatorLessThanEqualOne is violated..
There is, indeed, a flow in which one process executes both steps, the Increase_Accumulator and Decrease_Accumulator ones, in sequence, and accumulator is never > 1.
But the model checker just verified that our logic will not hold under the pressure of concurrency.
There is a fix for this: before increasing the accumulator value, wait for it to be precisely 0:
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC, Integers
CONSTANTS Loop, NULL
(*--algorithm loop
variables
accumulator = 0;
define
AccumlatorLessThanEqualOne == accumulator <= 1
end define;
fair process p \in {1,2}
begin
Increase_Accumulator:
await accumulator = 0; \* If the accumulator value is not 0, wait.
accumulator := accumulator + 1;
Decrease_Accumulator:
accumulator := accumulator - 1;
end process
end algorithm;*)
=============================================================================
This solution passes, and it shows the use of a powerful feature of TLA+: the await keyword.
The keyword will stop executing that process step until the condition after await is met.
Atomic PHP
My purpose in using TLA+ is to develop good specifications for ideas that I will then have to translate into code.
That code happens to be PHP, and I need to understand what "atomic" means in the context of PHP.
I wrote a small script to test out what atomic execution means in PHP:
<?php
$isWorker = isset($argv[1]);
if($isWorker){
$id = $argv[1];
foreach(range(1,200) as $k){
echo $id; // Print the process name.
}
exit(0);
}
foreach(range(1,2) as $n){
$command = PHP_BINARY . ' ' . escapeshellarg(__FILE__) . ' ' . $n;
$specs = [STDIN, STDOUT, STDERR];
proc_open($command, $specs, $pipes);
}
sleep(1); // Dumb wait for all processes to be done.
echo "\n";
exit(0);
Running the script will yield a variation of this output:
22222222222222222111111111111111111112111212221212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121212
12121212121212121212121212121212121212121212121212121212121212121212121212121222
Note: I've used proc_open as it's a "fire and forget" function that will not block the execution of the main script waiting for the processes to finish. Using passthru, as an example would stop the main script to wait for the started script to complete.
The processes get their turn to print in a non-deterministic order.
A note: using smaller numbers will incur in some execution caching that will unwrap the execute the foreach a first time and will then store and just print a string; that is why there is a 200 max in there.
The processes will print when they are allocated CPU time; the allocation of that CPU time cannot be known before the execution, which means the atomicity of PHP is a single instruction.
The takeaway is: after each PHP instruction, the CPU might be allocated to another PHP thread.
There is no guarantee another PHP thread will not get the CPU and do something between two adjacent PHP instructions, like an echo following another in the example code.
This information will come in handy later when it is time to translate the specification to code.
A first Loop specification
After exploring some key concepts, it's time to get back to the Loop specification.
The first version of the specification leaves much to be desired but is helpful to illustrate more logic and TLA+ constructs I will use.
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC, Integers, FiniteSets
CONSTANTS Loop, NULL
(*--algorithm loop
variables
jobsCount = 5,
maxParallelism = 2,
_startedRegister = [x \in 1..jobsCount |-> FALSE];
define
StartedCount == Cardinality({x \in DOMAIN _startedRegister: _startedRegister[x] = TRUE})
NextNotStarted == CHOOSE x \in DOMAIN _startedRegister: _startedRegister[x] = FALSE
end define;
fair process loop = Loop
begin
StartInitialBatch:
while StartedCount < maxParallelism do
with p = NextNotStarted do
_startedRegister[p] := TRUE;
end with;
end while;
end process
fair process worker \in 1..5
begin
WaitToStart:
await _startedRegister[self] = TRUE;
Work:
goto Done;
end process;
end algorithm;*)
=============================================================================
I've imported the FiniteSets module into my specification. This module contains functions, like the Cardinality one, that allows me to operate on sets.
Sets are lists of elements of the same type: {1, 2, 3} is a set of integers with a cardinality of 3, 1..3 is a quick way of defining the set {1, 2, 3}.
This version of the specification defines two types of processes: one is the Loop, the other is the worker.
There will be one Loop and 0 or more workers at any given moment.
The first construct I'm introducing is the _startedRegister variable.
I'm sure someone with more experience would develop a better solution, but I'm stuck with my knowledge of the subject.
Any _variableName prefixed with _ indicates a "technical" variable that I've defined in the specification, but that will most likely not be translated to PHP code.
To understand the purpose of the _startedRegister variable, look at the worker process: the process worker will start in the WaitToStart phase and will move to the next phase, the Work one, only when it is started from the Loop.
The definition of the _startedRegister variable is _startedRegister = [x \in 1..jobsCount |-> FALSE] and it can be read like "a map from jobs to the boolean false value". It's better understood in PHP array terms:
$_startedRegister = [1 => false, 2 => false, 3 => false, 4 => false, 5 => false];
So, when the Loop sets the flag for process 2 to TRUE, the await condition of worker 2 will be satisfied, and it will start.
The StartedCount and NextNotStarted definitions are operators. They are like user-defined functions in PHP.
It's worth reiterating TLA+ is not an implementation language but a specification one; it expresses operations, for the most, using mathematical definitions and logic operations.
StartedCount == Cardinality({x \in DOMAIN _startedRegister: _startedRegister[x] = TRUE}) means "StartedCount is the number of keys of _startedRegister for which the value for that key is true".
Translate that to PHP:
global $_startedRegister;
function StartedCount(){
return array_count(array_filter($_startedRegister));
}
On the same note, NextNotStarted picks the key first element in _startedRegister the value of which is false; DOMAIN _startedRegister is like array_keys($_startedRegister.
To end the syntax tour, with is used to define a local variable.
Read more about TLA+ syntax through Leslie Lamport's video course, on the Learn TLA+ site and from Hillel Wayne's book.
Running the specification now, checking for Deadlock and Termination will fail due to a deadlock.
I've defined five workers to run, but only 2 will be started in the StartInitialBatch phase of the Loop; TLA+ is telling me 3 workers are deadlocked: they will await forever for something that will never happen.
Maybe start all the processes?
After some work, I came up with a second iteration of the specification that is starting all the processes:
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC, Integers, FiniteSets, Sequences
CONSTANTS Loop, JobSet, Parallelism, NULL
(*--algorithm loop
variables
jobsCount = Cardinality(JobSet),
_startedRegister = [x \in JobSet |-> FALSE],
_exitStatusRegister = <<>>,
exitStati = <<>>;
define
StartedCount == Cardinality({x \in DOMAIN _startedRegister: _startedRegister[x] = TRUE})
NextNotStarted == CHOOSE x \in DOMAIN _startedRegister: _startedRegister[x] = FALSE
Running == StartedCount - Len(exitStati)
ParallelismRespected == Running <= Parallelism
CompletedCount == Len(exitStati)
end define;
fair process loop = Loop
begin
StartInitialBatch:
while StartedCount < Parallelism do
with p = NextNotStarted do
_startedRegister[p] := TRUE;
end with;
end while;
CheckExits:
if Len(_exitStatusRegister) > 0 then
CollectExitStatus:
with status = Head(_exitStatusRegister) do
exitStati := Append(exitStati, status);
end with;
_exitStatusRegister := Tail(_exitStatusRegister);
CheckExitedCount:
if Len(exitStati) = jobsCount then
goto Done;
end if;
MaybeStartOneMore:
if StartedCount < jobsCount then
with p = NextNotStarted do
_startedRegister[p] := TRUE;
end with;
end if;
goto CheckExits;
else
goto CheckExits;
end if;
end process
fair process worker \in JobSet
begin
WaitToStart:
await _startedRegister[self] = TRUE;
Work:
skip;
Exit:
_exitStatusRegister := Append(_exitStatusRegister, 0);
end process;
end algorithm;*)
\* BEGIN TRANSLATION
\* [...]
\* END TRANSLATION
OneWorderPerJobStarted == <>[](StartedCount = Cardinality(JobSet))
AllWorkersCompleted == <>[](CompletedCount = Cardinality(JobSet))
=============================================================================
I will not go through the specification line by line and concentrate on important parts.
At the bottom, after the translation, I've defined a temporal formula: it looks like an operator between parentheses and temporal "symbols" before it.
Specifically:
<> means "Eventually"; there is at least one state where the condition between the parentheses is true.[] means "Always"; what is between the parentheses is always true.
"Eventually always" means "When it happens for the first time, then it should keep being true".
Eventually, a flower pot will fall and break apart; it will then always be broken. If someone came and put the pot back together, then it would violate the "always" part, and the model verification would fail.
The rest of the code is operators to make the code more readable.
I've added one more technical variable, the _exitStatusRegister one. This is a sequence, denoted with <<>>. In sequences, order matters; in sets, type matters.
That represents a worker process ending and returning its exit status to the Loop process.
This specification is not doing all I need it to do, but will pass verification with the following settings:
What to check? - Temporal formulaDeadlock - checkedProperties
* `Termination`
* `OneWorderPerJobStarted`
* `AllWorkersCompleted`
* `ParallelismRespected`
* `JobSet <- [ model value ] {J1, J2, J3, J4, J5}`
* `Parallelism <- 2`
Five jobs and a parallelism of two.
This is good enough to try and translate it into code.
Translating the specification
<?php
function worker()
{
exit(0); // Just exit, like skip.
}
function startWorker($key)
{
$command = sprintf("%s %s %s", PHP_BINARY, escapeshellarg(__FILE__), $key);
return proc_open($command, [STDIN, STDOUT, STDERR], $pipes);
}
function loop(array $jobSet, $parallelism)
{
$batchSize = min(count($jobSet), $parallelism);
$started = [];
$exitStati = [];
foreach (range(1, $batchSize) as $k) {
$started[] = startWorker($k);
}
while (true) {
$procKeysToRemove = [];
foreach ($started as $process) {
$procStatus = proc_get_status($process);
if ($procStatus['running']) {
continue;
}
$exitStati[] = $procStatus['exitcode'];
$procKeysToRemove[] = array_search($process, $started, true);
if (count($exitStati) === count($jobSet)) {
break 2;
}
if (count($started) < count($jobSet)) {
$started[] = startWorker(++$k);
}
}
// Reduce the checked set as they complete.
foreach ($procKeysToRemove as $key) {
unset($started[$key]);
$started = array_values($started);
}
}
return $exitStati;
}
if (!isset($argv[1])) {
$exitStati = loop(range(1, 5), 2);
echo "\n", print_r($exitStati, true);
} else {
worker();
}
The code should work on any machine, should one want to try it out.
Just one note: calling proc_get_status a second time on an already exited process will yield an exit value of -1. To cope with that and to get a smaller optimization to the loop logic, I remove the processes from the $started array when done.
The non-technical variables from the specification are all there, and the loop works, but with two significant issues:
- That
while( true ) is a CPU-locking loop. If the workers were running long enough, it would hoard CPU time running continuously, bringing the CPU load to 100% with wasted energy consumption: most of those cycles would not update a state.
- The workers do something, but what? There is no communication from them back to the loop. Did they fail? Did they explode?
As pipes for the workers, I'm setting the STD ones, but that is just to make the code shorter. Letting the workers print to the same output stream as the loop is a bad idea as it does not allow to control the print order, and thus formatting, and does not allow the Loop to get any information beyond an exit status.
Next
In my next post, I will add pipes and Inter-Process Communication (IPC) to the mix to see how the specification and the PHP code will evolve.
What is TLA+?
In my personal understanding of it: a specification language that helps me explore fallacies in my PHP software architecture and test that architecture under the pressure of concurring threads.
To better explain what TLA+ is, one should refer to the site of Leslie Lamport, the mind behind TLA+.
I've found Hillel Wayne's site and book to be the best way for me to learn the basics.
The pitch for this latest exploration of mine was Wayne's video "Tackling Concurrency Bugs with TLA+": it struck a chord. The concurrency part especially. Then I found out there was testing involved and could not let go of the thought.
What am I using TLA+ for?
Given enough hot metal under it, TLA+ could really be used to model any concurring process my mind could think of, from high-level distributed systems to low-level algorithms.
Yet I hate answers that sound like a variation of the "it depends on your situation and context", and I will avoid that by providing the particular context I'm applying TLA+ to.
I'm using TLA+ to model parts of a process-based PHP testing framework I'm writing.
I'm still pretty foggy about how this testing framework will fit in the grand scheme of things, but I've spent some time working with Jest and found it to be really delightful.
I'm writing something from scratch and, without the pressure of having to ship a product, I can take my time to do it right and in a way that satisfies me.
Let's start from an example file, a "spec files" in Jest jargon, this imaginary test framework would consume:
<?php
/**
* @env WordPress
*/
describe('get_foos_function', function () {
// Create the test users.
$admin = wp_insert_user(['user_login' => 'a', 'user_pass' => 'a', 'role' => 'administrator']);
$subscriber = wp_insert_user(['user_login' => 's', 'user_pass' => 's', 'role' => 'editor']);
// Create the test foos: one public, one draft.
foo_create(['status' => 'publish']);
foo_create(['status' => 'draft']);
describe('Admin context', function () use ($admin, $subscriber) {
// Simulate Admin context.
define('WP_ADMIN', true);
it('should return all foos to admin user', function () use ($admin) {
wp_set_current_user($admin);
$foos = get_foos();
expect(count($foos))->toBe(2);
});
it('should return only public foos to non-admin', function () use ($subscriber) {
wp_set_current_user($subscriber);
$foos = get_foos();
expect(count($foos))->toBe(1);
});
});
describe('Front-end context', function () use ($admin) {
it('should return public foos to admin user', function () use ($admin) {
wp_set_current_user($admin);
$foos = get_foos();
expect(count($foos))->toBe(1);
});
it('should return public foos to non-admin', function () {
wp_set_current_user(0); // Visitor.
$foos = get_foos();
expect(count($foos))->toBe(1);
});
});
describe('REST context', function () use ($admin) {
// Simulate a REST request.
define('REST_REQUEST', true);
it('should return all foos to REST admin user', function () use ($admin) {
wp_set_current_user($admin);
$foos = get_foos();
expect(count($foos))->toBe(2);
});
it('should show no foos in REST to visitors', function () {
wp_set_current_user(0); // Visitor.
$foos = get_foos();
expect(count($foos))->toBe(0);
});
});
});
To note in the above code example:
- The
@env WordPress annotation will tell the (again: still hypothetical) testing framework that WordPress should be loaded before the tests. This will allow the following code to call WordPress functions.
- The block at the top creating the users and the "foos" would be the equivalent of Jest
beforeAll function, or PHPUnit setUpBeforeClass method call: it will do something once, before any test runs.
- In the inner
describe blocks, I'm defining constants like WP_ADMIN and REST_REQUEST to simulate, respectively, the admin and the REST request contexts.
- Was this code to run sequentially, the second
describe block, the Front-end context one, would inherit a defined WP_ADMIN constant that would make it fail.
- Being "smart" about the order of the
describe blocks here, then, would not change much as at least one describe block would end up inheriting the constant defined by one of the describe blocks that ran before.
The "simple" solution is to run each describe block in isolation in a dedicated, separate PHP process.
Let's say I'm willing to do it: there will be a main tread in charge of starting, and managing the state of, multiple dedicated PHP processes running the describe blocks.
Loop concerns
A loop waiting on processes to be done is a pretty well-explored pattern.
Yet there are challenges deriving from the specific application and PHP limits.
A quick list:
- Processes might succeed or fail and will need to communicate that back to the main PHP process for it to be able to orderly print the output: this means Inter-Process Communication (IPC) is required.
- On Windows, one of the possible environments where this testing framework might run, IPC is not that easy given the many IPC pipes and streams inconsistencies.
- A loop should not be implemented with a logic that sounds like, "Every now and then check on the started processes, sleep a bit, repeat". Checking more often would make the Loop more responsive but hoard CPU cycles. Checking less often would reduce resource consumption, making the Loop less responsive.
Some prototypes later, I concluded I should use the stream_select function to "watch" the pool of running processes from the main PHP thread, get prompt updates, and avoid a CPU lock.
When used on blocking streams, the function will wait for some streams part of the pool (a stream could be opened over a file or an IPC pipe) to update to move on.
All of this long and winded introduction to get to the first TLA+ model.
CLI and TLA Toolbox
To verify my specification, I'm using the TLA+ toolbox; I'm not fond of the IDE part, though, and prefer using vim and Hillel Wayne's vim plugin to write my specifications.
With that out of the way, let's start with my model.
I would like to reiterate I'm not a TLA+ expert. I am sharing my discovery process and findings. Go back to the first section of this article to find the references and literature to provide expert insight into it.
I will be dealing, to model the Loop and ancillary PHP processes, with two types of process:
- The Loop, the PHP process the user starts that will, in turn, start and manage one PHP process per
describe block.
- The Worker, a PHP process started by the Loop to handle a
describe block.
I've created a new spec_loop specification in the TLA+ Toolbox application and wrote the first version of my specification, just to check if things work.
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC
CONSTANTS Parallelism, Jobs, Loop, NULL
(*--algorithm loop
process loop = Loop
begin
StartInitialBatch:
goto Done;
end process
end algorithm;*)
=============================================================================
The TLA+ Toolbox supports two languages:
- TLC - the somewhat difficult to read and write specification language initially used for TLA+.
- PlusCal - the higher-level language that will transpile to TLC.
PlusCal is what I put between (*--algorithm loop and end algorithm;*).
When I save the file and click "Translate PlusCal Algorithm", I get the following output:
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC
CONSTANTS Loop, NULL
(*--algorithm loop
process loop = Loop \* There is process called Loop
begin
StartInitialBatch: \* That process starts...
goto Done; \* ...and directly goes to Done, a state that means it's terminated.
end process
end algorithm;*)
\* BEGIN TRANSLATION (chksum(pcal) = "f8a984b0" /\ chksum(tla) = "83520898")
VARIABLE pc
vars == << pc >>
ProcSet == {Loop}
Init == /\ pc = [self \in ProcSet |-> "StartInitialBatch"]
StartInitialBatch == /\ pc[Loop] = "StartInitialBatch"
/\ pc' = [pc EXCEPT ![Loop] = "Done"]
loop == StartInitialBatch
(* Allow infinite stuttering to prevent deadlock on termination. *)
Terminating == /\ \A self \in ProcSet: pc[self] = "Done"
/\ UNCHANGED vars
Next == loop
\/ Terminating
Spec == Init /\ [][Next]_vars
Termination == <>(\A self \in ProcSet: pc[self] = "Done")
\* END TRANSLATION
=============================================================================
\* Modification History
\* Last modified Wed Mar 16 10:46:22 CET 2022 by lucatume
\* Created Mon Mar 14 08:40:43 CET 2022 by lucatume
The \* BEGIN TRANSLATION and \* END TRANSLATION comments fence the translation operated by the IDE.
In the following code examples, I will not report the translation as that is a direct output of the PlusCal algorithm and not that interesting to this post.
After \*, you see comments in PlusCal and TLC syntax.
Notes:
EXTENDS is like import in node: import one or more modules I will need.CONSTANTS defines the inputs my specification will have. They are the parametric part of my specification, the values Models will use to customize a check.
The specification is easy enough, I create a Model called "MC" in TLA+ Toolbox and set it up like this:
What is the behavior spec? -- Temporal formula.What to check?
* `Deadlock`
* `Properties`
* `Termination`
* `NULL <- [ model value ]`
* `Loop <- [ model value ]`
Here's a screenshot of the settings in the TLA+ toolbox:
What does all this mean?
Temporal formula means I want to check if my specification works over time. More on that later.
Deadlock means I want to check if there's any state, a combination of execution steps, that would put the model in a deadlock.
Termination means I want to check that, in any state, all the processes will get to the Done phase, the last one.
NULL and Loop are custom types I will use in my specification.
I run the model and... it fails: Temporal properties were violated., Stuttering.
That means the one temporal property I've set, the one where all processes should eventually terminate, is violated.
Here comes the first great power of TLA+: it will model states where some, or all, processes will not run.
If the Loop process never runs, it will not terminate. As such, the Termination temporal property is violated.
I'm not interested in modeling the case where the Loop does not run. That means the PHP code never got there, which implies some error happened before, the user is shown some exception or message: I do not need a specification for that.
Please, run the Loop process
How do I tell the model checker I'm not interested in a state where the Loop process does not run?
I tell it the process is fair. It means any state where the Loop process does not run should not be part of the model check.
I update the specification to make the Loop process a fair one:
----------------------------- MODULE spec_loop -----------------------------
EXTENDS TLC
CONSTANTS Loop, NULL
(*--algorithm loop
fair process loop = Loop
begin
StartInitialBatch:
goto Done;
end process
end algorithm;*)
=============================================================================
After re-compiling the code and rerunning it, the model check passes: the Loop process has only one step: the StartInitialBatch one, which will immediately go to the Done one.
A "step" in TLA+:
- Is indicated by a
Label:
- Is atomic.
This is enough for one post.
Check out the second post.
This post is the second in a series of posts where I will document my building of the WordPress testing framework I would have liked to find when I started working with WordPress.
The name of this framework in-fieri is ork, and you can read more about it in the first post dedicated to it.
Getting the whole testing setup up and running
In my previous post, I've set up some basic tests that got some of the job done. Still, I would like to come back to those to set them up once, while the involved logic is straightforward for me to follow, addressing Continuous Integration and IDE integration to start on the right foot.
For CI, I've picked GitHub Actions: it comes included in my plan and is fast enough and reliable enough for my current needs.
Being the lover or make I am, I've set up the project Makefile to allow me to build and run the tests by sticking to the trusted pattern of make build && make test:
name: CLI App CI
on: [push]
jobs:
bdd:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
ref: 'main'
- run: cd app && make build && make test
Not much to see here.
To avoid the back and forth of testing GitHub Actions through pushing and waiting for the worker to run, I'm using nektos/act. The one-file binary will run any workflow in the project .github directory using a container image that is the same as the one used by GitHub actions workers for most intents and purposes.
I have configured my machine ~/.actrc file to correctly alias the ubuntu machines.
-P ubuntu-latest=nektos/act-environments-ubuntu:18.04
-P ubuntu-18.04=nektos/act-environments-ubuntu:18.04
After confirming the tests are passing locally, I've been rewarded with a green light on GitHub Actions too.
Just a note: act will simulate the GitHub Actions environment almost perfectly, but the GITHUB_TOKEN environment variable that GitHub Actions will set up by default will be missing.
Since I'm working in a private repository, I've created a token dedicated to nektos/act in the ~/.github-token file that I export at run time to allow act to pull and test private dependencies the same way GitHub Actions would access them:
GITHUB_TOKEN="$(<${HOME}/.github-token)" act
This token definition is all it takes for act to work correctly.
Laying out the first steps
I made a habit of starting my development work from the end.
This approach helps me thinking of the "what" before thinking about the "how".
The project requirements will, roughly, be:
- Download
ork for your current operating system somewhere the terminal can execute it from your terminal emulator of choice.
- Use whatever local development webserver solution you want to host your development WordPress server.
- Navigate to the directory where your testing project will live.
- Run
ork and start testing.
Point 2 above, especially, is something I care about.
One of the things the wp-browser project taught me is I should not make assumptions about what operating system, or development stack, different developers will use to create, maintain and test WordPress projects.
There are many solutions available for WordPress developers, ranging from PHP built-in server to Mamp and other host-based solutions to complex Docker-based stacks or virtual machines.
Each one has its complications from the perspective of a testing framework: some are running "on the metal" (e.g., PHP built-in server, Mamp or valet), some are using containers (e.g., Devilbox or the previous version of Local by Flywheel) and some others are using virtual machines (e.g., the VVV project).
Understanding where the PHP server serving the development WordPress server is, in respect to the machine that is executing the tests, is where several users get lost and give up.
Say I have set up Codeception tests with wp-browser on my host machine. I've installed WordPress, locally, at /Users/lucatume/Sites/wp, but I am running my tests from within a Docker container. The path to the WordPress installation will probably be /var/www/html as that is the path to the WordPress installation in the default WordPress Docker image.
The point is: there are plenty configurations and local development server solutions out there that work for those using them.
I want ork to remove the friction of dealing with each and "just work" as long as one can visit the WordPress site using a browser.
Can you go to https://wordpress.local? Fine, ork should work.
Do you have your site served with PHP built-in server at http://localhost:8888? This setup, too, is fine, ork should work with it.
Have you set up WordPress with docker-compose where Docker runs into a Linux VM running, in turn, on your Windows machine? If you can visit the WordPress site from your browser, then ork should work.
All the different setups above have in common that the user can visit the development site URL. The ork binary will have to rely, at least initially, only on that piece of information to work.
The first step, then, assuming points 1 to 3 are taken care of by the user, is to model the following scenario:
» mkdir ork-tests
» cd ork-tests
» ork
What is the URL of the site you would like to test? ork.local
Checking site URL ... ok
Creating the first spec ... ok
Running specs ...
Homepage
√ it works (410ms)
1 passing (422ms)
Those familiar with JS testing should recognize mocha BDD output.
Since I'm currently using it to power my tests, I've just used the inspiration to draft this imaginary CLI output.
There's a bit to get done before getting to the actual running of tests, in any case, so I better get to it.
Testing a binary and the command line
The first test I would like to write, in Gherkin syntax, would read out like this:
Given I have changed the directory to one that does not contain an `ork` configuration file
When I run `ork`
Then I should be asked to provide the URL of the site I would like to run tests for
And I should see an `ork.config.json` file created containing the URL I provided
This seems a simple enough test to translate into a mocha test.
It turned out not to be that simple for me, though.
Being, in essence, a novice when it comes to NodeJs development, it took me some time to come to terms with how
the loop of execution is structured and how my code will execute.
Starting from the end here is the first test I wrote that is, currently, passing:
// test/FirstSetup.js
const { existsSync } = require('fs')
// I've created a test support library of functions to allow me to write tests the way I want.
const { invokeOrk, mkTmpDir, forFile, toReadFile } = require('./_support.js')
// I'm using `chai` to write assertions in spec/BDD format.
const expect = require('chai').expect
describe('First setup', function () {
it('should prompt the user to create config file if not found', async function () {
// As a first step, create a test directory in the test/_output directory. Yes, Codeception habit.
const tmpDir = mkTmpDir()
// This is where the configuration file should be, if everything goes according to plan.
const configFilePath = tmpDir + '/ork.json'
// To begin with, the file should not exist.
expect(existsSync(configFilePath)).to.be.false
// More on this later, I invoke the compiled binary in the test directory.
let event = await invokeOrk([], { cwd: tmpDir })
// The first event should be output coming from the binary, asking the site URL.
expect(event.type).to.equal('stdout')
expect(event.data).to.match(/url.*site/i)
// Reply to the request with `ork.local`.
event.proc.stdin.write('ork.local\n')
// Wait up to 5s for the configuration file to appear.
const exists = await forFile(configFilePath, 5000)
expect(exists).to.be.true
// Read the configuration file contents and make sure they are correct.
const configFileContents = await toReadFile(configFilePath, 'utf-8')
const config = JSON.parse(configFileContents)
expect(config.url).to.equal('http://ork.local')
})
})
There are some "gotchas" I collected along the way.
They might be obvious to more experienced Node developers, but this is all pretty new to me:
- Due to Node single-thread nature, blocking that only thread on one operation cannot be done. The single thread nature of Node is not that different from PHP, what makes it
very different is that thread is also the one in charge of handling the user input, and blocking it would mean freezing any UI; there is no UI in PHP. The issue is not that pressing in a CLI application, but most Node primitives are not synchronous, and they will yield control immediately. Appreciating and understanding this required some adjustment on my side.
- All this async code can be made sync-ish using
async\await; "ish" as what will happen is that Node will move to another task at the same scope level.
- Most existing primitives can be "promisified" to make them return promises when they would, instead, fire callbacks when something happens.
Grasping all the above did take me some time, but I'm pretty satisfied with two solutions in particular in thetest/_support.js file.
The first is the one that allows me to run the sub-process that is running the ork binary in "events".
Possible events are stdout, error, stderr and exit.
Each event will return a Promise to get the next event, and this is what allows me to write event-by-event
tests in my first spec and writing the test above in a way that closely resembles the interaction the user
would have with the CLI:
// Build a Promise for the process next event.
function processPromise (proc) {
return new Promise(function (resolve, reject) {
for (let eventName of ['error', 'data', 'exit']) {
for (listener of proc.listeners(eventName)) {
proc.off(eventName, listener)
}
}
proc.on('error', function (error) {
reject(error)
})
proc.stdout.on('data', function (chunk) {
resolve({ type: 'stdout', data: chunk, proc: proc, next: processPromise(proc) })
})
proc.stderr.on('data', function (chunk) {
resolve({ type: 'stderr', data: chunk, proc: proc, next: processPromise(proc) })
})
proc.on('exit', function (code) {
resolve({ type: 'exit', data: code, proc: proc, next: processPromise(proc) })
})
})
}
async function invokeOrk (args = [], options = {}) {
const mergedOptions = {
...{
env: { ...process.env, ...{ NODE_OPTIONS: '' } },
stdio: ['pipe', 'pipe', 'pipe'],
}, ...options
}
// Spawn the process, an async task, and set up some defaults.
const proc = spawn(locateBin(), args, mergedOptions)
proc.stdout.setEncoding('utf8')
proc.stderr.setEncoding('utf8')
// Return the Promise that will yield the process first event; probably output or an error.
return processPromise(proc)
}
The next piece of code I'm proud of is the one that allows me to wait for the file to come into existence.
In the tests, I use it to wait for the configuration file to appear, but I'm sure it will come in handy again:
// The name is to follow its most likely use of `await forFile`.
async function forFile (file, timeout = 5000) {
return new Promise(function (resolve, reject) {
const basename = file.replace(dirname(file) + '/', '')
if (fs.existsSync(file)) {
// If the file already exists when we start, resolve now.
resolve(true)
}
// Set a timeout: either the files appears before it runs out, or fail.
const timeoutRef = setTimeout(() => {
watcher.close()
resolve(false)
}, timeout)
// Start watching the directory that should contain the file.
const watcher = fs.watch(dirname(file), async (eventType, filename) => {
if (eventType === 'rename' && filename === basename) {
if (fs.existsSync(file)) {
// The file came into existence, block the timer and resolve.
watcher.close()
timeoutRef.unref()
resolve(true)
}
}
})
})
}
Finally, the test output at the end of all the tribulation:
» make test
First setup
✓ should prompt the user to create a config file if not found (354ms)
1 passing (360ms)
» tree -L 3 test/_output
test/_output
└── dirs
└── KJvSkAw4Nj
└── ork.json
2 directories, 1 file
» cat test/_output/dirs/KJvSkAw4Nj/ork.json
{
"url": "http://ork.local"
}
Next
Getting to this first test to pass took some effort, but I'm glad I did it.
Next time I will get the CLI output, right now pretty spartan, into shape and extend the functionalities using BDD.
The backstory or "The part you skip because you do not care about it"
What's a good introduction without a backstory?
When I started working with PHP and WordPress about eight years ago, my first line of PHP was a PHPUnit test.
I only wrote code using TDD in my previous career, and really thought one could not write PHP code without testing it.
I look back at that time smiling: there are, indeed, people out there somewhere that do write code without writing tests for it.
As I spent time working on more WordPress projects and with more teams, I appreciated some factors that have not
changed but softened my stance on testing.
First, writing automated tests for WordPress is not that widely documented, and the wider community is not that
"into it". I've seen some talks at WordCamps about testing, but not that many. Mind: there are developers that do that, just not so many to tip the scale of the typical WordPress conversation; if you're doing it, you've got all
my appreciation.
Second, there are tools and frameworks out there that will make testing WordPress easier; wp-browser objective is precisely that. Still, there are wide cracks in how they allow different types of testing to be covered,
and if a developer's need falls in one of those cracks, then that developer will have to do a lot of research and problem solving to get anything done. When one masters the instrument, then the payoff will be immense, but that's a steep price to pay to start.
Third: most testing tools will work well on new projects and either impose a "refactoring tax" on the existing
code to provide any meaningful return or outright not allow it.
With a touch of gatekeeping and elitism, all of the above pushes well-intentioned, but inexpert, people away from
adopting tests far too often by conflating adoption and maintenance price into, simply, too high a price to
pay to "just test code".
One test is better than none
When consulting with companies trying to adopt testing approaches, my first step is usually to demystify testing.
Sure, 100% coverage would be excellent; automated builds and flawless documentation of all the possible features and scenarios would be great to have, and blazing fast regression coverage would be even better.
But a developer, or a team of developers, has built something that does solve people problems; how can this be
so bad?
Would we be having this conversation if we could run a command like magic-test-all-my-code without any further input, with complete test coverage and checks? Probably not, in most cases.
Ease of setup, adoption, and use, I think, is fundamental to frame this kind of conversations; if I could write
magic testing frameworks, I would. I cannot, sadly.
So, in short, the ork project is my attempt at creating the testing solution I would have liked to find when I first
got into PHP and WordPress development.
The first PHP and WordPress code I wrote was a messy blob in the theme functions.php file; it took me some time
to be able to run tests for it. Like most, the first code they write will not be "new" code but will rely on differing
levels of legacy code.
And I've run into a lot of that in the WordPress universe.
What is ork then?
This post is not by chance titled "Building ork": I have not built it yet.
Not all of it, at least.
How would ork compare to wp-browser and Codeception?
That much is clear: ork is not a power tool.
Its objective is not to cover all the possible testing requirements but to allow someone that "just wants to start
testing" to do that with as little knowledge required as possible.
It's my take on the idea of getting more (testing) bang for the buck.
And then I'm building it from scratch. In Node. Cross-compiling binaries for different operating systems. And it
will support PHP and JS syntax to write tests.
What could go wrong?
Getting down to business
All this boring introduction to get to the first line of code.
The first decision I took, as anticipated above, is to ship ork as a cross-compiled executable for Windows, Linux, and Mac.
This decision comes from my experience in maintaining wp-browser and from the pain caused by keeping compatibility with a sizable matrix including different PHP, WordPress, Codeception, and Composer versions. I love the project, but I would like to avoid this particular pain.
After some research, I've settled on using Node and pkg to write once and compile something that should work on all operating systems.
I forgot what using something that is just one file feels like in years of package management with PHP and JS.
After some project scaffolding, here is the src/bin.ts file that will act as the entry point of my application; a
very humble "Hello World" type of message:
console.log('Hello ork!');
I've decided to use Typescript to develop the project to leverage its compile-time type-checking.
As it's been the norm for my development projects for some time now, I've set up a Docker image
to create a portable development system.
This is the Dockerfile of the node image I will be using over and over to run and compile the code:
## Use LTS version of node.
ARG NODE_VERSION="14.16.1"
FROM node:${NODE_VERSION}
## Make npm cache, the node modules and /usr/bin/lib accessible by all.
RUN mkdir /.npm \
&& mkdir -p /usr/local/lib/node_modules \
&& chown -R 501:0 /.npm \
&& chmod -R 0777 /usr/local/
## Change the user to my user to avoid file mode issues.
ARG UID=0
ARG GID=0
USER ${UID}:${GID}
## Install the required build dependencies.
ARG NODE_PKG_CACHE_PATH
RUN npm install -g typescript pkg dts-gen prettier
## pkg will require caching to not take forever.
ENV PKG_CACHE_PATH="${NODE_PKG_CACHE_PATH}"
## No default entrypoint or command, overwrite the default ones.
ENTRYPOINT [""]
CMD [""]
## Expose debugging port.
EXPOSE 9229
To streamline the build process as much as I can, I've set up a Makefile to be used with the make binary,
with a collection of solutions I've accumulated over time:
## Use bash as shell.
SHELL := /bin/bash
## If you see pwd_unknown showing up, this is why. Re-calibrate your system.
PWD ?= pwd_unknown
## PROJECT_NAME defaults to name of the current directory.
PROJECT_NAME = $(notdir $(PWD))
## Suppress `make` own output.
.SILENT:
## Make `build` the default target to make sure it will display when make is called without a target.
.DEFAULT_GOAL := build test
## Create a script to support command line arguments for targets.
## The specified targets will be callable like this `make target_w_args_1 -- foo bar 23`.
## In the target, use the `$(TARGET_ARGS)` var to get the arguments.
## To get the nth argument, use `export TARGET_ARG_2="$(word 2,$(TARGET_ARGS))";`.
SUPPORTED_COMMANDS := node_machine node npm
SUPPORTS_MAKE_ARGS := $(findstring $(firstword $(MAKECMDGOALS)), $(SUPPORTED_COMMANDS))
ifneq "$(SUPPORTS_MAKE_ARGS)" ""
TARGET_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
$(eval $(TARGET_ARGS):;@:)
endif
## Set up some defaults.
NODE_VERSION = 14.16.1
## Any target commented with `## <description>` will show on `make help`.
help: ## Show this help message.
@grep -E '^[a-zA-Z0-9\._-]+:.*?## .*$$' $(MAKEFILE_LIST) \
| sort \
| awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
.PHONY: help
build: ts pkg ## Builds the ork application.
pkg: ## Packages the application.
docker run --volume "$(PWD):$(PWD)" -w "$(PWD)" ork-app/node pkg package.json
_docker/node/uuid: _docker/node/Dockerfile
[ -d "$(PWD)/.cache/pkg" ] || mkdir -p "$(PWD)/.cache/pkg"
docker build \
--build-arg NODE_VERSION="$(NODE_VERSION)" \
--build-arg UID="$${UID}" \
--build-arg GID="$${GID}" \
--build-arg NODE_PKG_CACHE_PATH="$(PWD)/.cache/pkg" \
--tag ork-app/node:latest \
--iidfile _docker/node/uuid \
_docker/node
### Build the node image, if not built already.
ts: _docker/node/uuid
docker run --volume "$(PWD):$(PWD)" -w "$(PWD)" ork-app/node tsc
.PHONY: ts
node_machine: _docker/node/uuid ## Runs a generic command in the node container.
docker run --volume "$(PWD):$(PWD)" -w "$(PWD)" ork-app/node $(TARGET_ARGS)
node: _docker/node/uuid ## Runs a node command, e.g. `make node -- <command>`.
docker run --volume "$(PWD):$(PWD)" -w "$(PWD)" ork-app/node node $(TARGET_ARGS)
npm: _docker/node/uuid ## Runs a npm command, e.g. `make npm -- <command>`.
docker run --volume "$(PWD):$(PWD)" -w "$(PWD)" ork-app/node npm "$(TARGET_ARGS)"
Whit these files in place, building the cross-system application is as easy as running make build.
» make build
> pkg@5.1.0
Following the configuration in the package.json file, the executables for Windows, Linux, and macOS
have been created in the dist directory.
Running the command on macOS and Linux, I will get the following output:
» ./dist/ork-macos
Hello ork
The same will happen in the Windows terminal:
C:\Users\lucatume\Desktop>.\ork-win.exe
Hello ork
This might not look like much, but it lays the foundation for the flow I would like to adopt to develop the application.
Adding tests
There is almost nothing in the current CLI application that will warrant testing.
That is precisely why adding tests at this stage will be so cheap and convenient.
The tests I'm setting up for the CLI application, let's call it "acceptance" testing or
"end-to-end" testing for the sake of analogy of what I would do while developing a Web application, are
tests that will run the compiled ork binary in a sub-process and check its output and side effects.
For that, I've chosen to use the command-line-test package and the mocha testing framework.
To keep with the portability and reproducibility theme, I've set up two additional make targets to
execute the tests in the node container with, and without, remote Node debugger support:
test_debug: _docker/node/uuid ## Runs the tests in debug mode, port
docker run -i --volume "$(PWD):$(PWD)" -w "$(PWD)" -p "9229:9229" ork-app/node node --inspect-brk=0.0.0.0 \
node_modules/mocha/bin/mocha \ --timeout 0 --ui bdd
.PHONY: test_debug
test: _docker/node/uuid ## Runs the tests.
docker run -i --volume "$(PWD):$(PWD)" -w "$(PWD)" ork-app/node node \
node_modules/mocha/bin/mocha --timeout 0 --ui bdd
.PHONY: test
I wrote a first test just to confirm the setup works correctly:
const {orkBin} = require('./_support.js');
const CliTest = require('command-line-test');
const assert = require('assert');
describe('hello ork', function () {
it('should display hello ork', async function () {
const cliTest = new CliTest();
const {error, stdout, stderr} = await cliTest.spawn(orkBin);
assert(error === null, error);
assert(stdout !== null);
assert(stdout === 'Hello ork\n', stdout);
});
});
In case you're wondering, the _support.js file will only provide me with a portable solution to find the compiled
ork binary depending on the system the tests are running on:
const os = require('os');
const osName = require('os-name')(os.platform(), os.release()).substr(0, 3);
const locateBin = function () {
let binPostfix;
switch (osName) {
case 'mac':
binPostfix = 'macos';
break;
case 'win':
binPostfix = 'win.exe';
break;
default:
binPostfix = 'linux';
break;
}
return __dirname + `/../dist/ork-${binPostfix}`;
};
module.exports = {
orkBin: locateBin()
};
I'm closing this first post on the passing test output:
» make test
hello ork
✓ should display hello ork (408ms)
1 passing (414ms)
Next
In my next post, I will start working on the actual code laying out the pieces of my application one
by one as I develop the application concept.
Making two great solutions work together.
In my last post, I've ˙praised the uncelebrated power of the WP-CLI shell command.
WP-CLI maintainer Alain Schlesser provided a further tip to make the WP-CLI shell even better through the schlessera/wp-cli-psysh package.
I strongly suggest you try that out.
Another tool I use for my daily work is the Tinkerwell application.
Tintkerwell is a step up from an interactive PHP shell to an elementary code editor.
Tinkerwell UI does not replace a proper PHP IDE, but it provides a UI with some neat features (vim mode, for one) with a clean and straightforward approach.
I've found this article and videos by Ross Wintle enjoyable and easy to follow and suggest you give it a read and look.
I could set up Tinkerwell to work on my local WordPress site in little time, and you will be able to do the same if your WordPress application is served on localhost.
Should that not be the case, e.g., if you're using Local by Flywheel or some other "containerized" solution, then setting Tinkerwell up might require either its remote connection function to do some modification to the WordPress installation configuration file.
Setting up a Local site for Tinkerwell
Since the SSH-based approach seems overkill and too complicated for the simple needs of tinkering on a local site, I prefer the second approach and consistently update my Local by Flywheel websites wp-config.php file to correctly handle requests coming from the Tinkerwell application.
First of all, set the Tinkerwell working directory to the app/public directory of your Local by Flywheel WordPress site:
Second, modify the site wp-config.php file to detect an incoming request from the Tinkerwell application and connect to the WordPress database via the correct database host:
/** MySQL hostname */
if ( isset( $_SERVER['SCRIPT_NAME'] )
&& false !== stripos( $_SERVER['SCRIPT_NAME'], 'tinkerwell' ) ) {
// Host from the Local > Database > "Remote Host" field.
$db_host = '192.168.95.100';
// Port from the Local > Database > "Remote Port" field, omit if there's no port.
$db_port = ':4026';
define( 'DB_HOST', "{$db_host}{$db_port}" );
} else {
define( 'DB_HOST', 'localhost' );
}
I use the previous version of Local, the Docker-based one, but the snippet will work with the new, MAMP-like version.
If everything is correct, you should be able to start tinkering with your WordPress site using Tinkerwell.
If you're using more elaborate solutions, e.g, a Docker stack, then just set the $db_host and $db_port variables to the database container IP address and port.
Chances are, if you've got no idea what this means, then you're probably not using a container-based solution and the snippet and example above should be fine.
I do not see nearly enough mention, or use, of the wp shell command.
By now, anyone working on WordPress projects for a living should, at the very least, know what WP-CLI is. If you do not, then take a read here.
The short version is that WP-CLI is a tool that allows skipping the WordPress UI to perform mundane tasks or script complex ones. WP-CLI is, as the name suggests, a Command Line Interface that will not run outside of WordPress but in WordPress.
That means that when you type the wp command that, in most situations, is the aliased name of the tool binary, WP-CLI will load WordPress, and with it, its current theme, must-use, and other plugins. While this is not the case for some lower-level commands, it's accurate for most.
WordPress is a CMS (Content Management System); as such, its primary and intended use is the one to take an HTTP request as an input, say a GET request to the /2020-10-15/my-great-posts URI and go, roughly, through the following steps:
load the WordPress core components like user management, session management,
load the must-use plugins and active plugins
load the theme, and plugins, identify the request as one for the site front-end (as is the case for this request example)
identify the target area of the request (front-end as is the case for this example request, REST API, Admin UI et cetera)
finally, route the request to the components that should handle it
Produce output in response, the format depending on the nature of the request (e.g., HTML for a front-end request, JSON for a REST API request, and so on).
If only one could stop the loading process at step 4 from the list above, it would allow someone to tap into a fully up and running WordPress session, with all the core components, plugins, and theme loaded to "do things in the WordPress" installation.
That is precisely what the WP-CLI shell command allows.
From my terminal application, I can type wp shell and be greeted by a modest, although powerful, prompt:
Sites/wp-site » wp shell
wp>
That prompt is a PHP interactive shell prompt; you could get something similar by typing php -a and being greeted by the PHP interactive shell prompt. Since this is a PHP interactive shell and not your terminal application, only PHP code and functions will make sense.
The difference, though, is that the PHP interactive shell will not load WordPress for you as the wp shell command does:
Sites/wp-site » php -a
Interactive shell
php > get_option('siteurl');
Warning: Uncaught Error: Call to undefined function get_option() in php shell code:1
Stack trace:
##0 {main}
thrown in php shell code on line 1
php > exit
Sites/wp-site » wp shell
wp> get_option('siteurl');
=> string(19) "http://wordpress.test"
wp> exit
Sites/wp-site »
As is the case for PHP interactive shell, you can exit the WP-CLI interactive shell by issuing the exit command.
I use the wp shell command daily to debug the output of some functions deeply nested into the code and know their output or effects. Each PHP or WordPress function invoked will affect the underlying WordPress installation, requiring some care. Still, I prefer this method to test out ideas quickly or to avoid filling my code with echo, var_dump, print_r, die, and other debug methods that would entail long browser refresh trial and error approaches.
If you've never tried out WP-CLI, you definitely should, and if you've never used the wp shell command, I invite you to try it out now and get addicted.
Taking inspiration from this blog, I encourage you to make this reading worth it:
If you answer the questions below, you will create new neurons in your head through neurogenesis. But if you don't, you will forget about 80% of what you just read tomorrow.
- At which step of the WordPress loading process does
wp shell put you?
- What is the difference between
wp shell and your terminal emulator?
- Why should you use care to use the
wp shell?
Edit: some Twitter responses later, WP-CLI maintainer Alain Schlesser provided a further tip to make the WP-CLI shell even better with integration of the schlessera/wp-cli-psysh package.
Just run wp package install schlessera/wp-cli-psysh and marvel at the much better output you get out of the wp shell command.
Previously, in this series
In the first post in this series, I've covered the basic stuff: what Docker and docker-compose are and how they could enable a portable environment for development, and more interesting to me, testing WordPress projects.
In the second post, I've detailed the behavior of the same docker-compose.yml stack on macOS, Windows, and Linux machines.
I've concluded the post in a situation where files created by the WordPress container, on a Linux host, would not belong to my user, but another user.
In this third article, I will work to mitigate the "portability" issues arising from a stack that will behave differently depending on the host operating system (macOS, Windows or Linux) to move closer to the ideal of a portable testing environment for WordPress projects.
The docker-compose.yml file in question, the one I'm starting this post from, is this one:
version: '3.1'
volumes:
db:
services:
wordpress:
image: wordpress
ports:
- 8080:80
environment:
WORDPRESS_DB_HOST: db
WORDPRESS_DB_USER: exampleuser
WORDPRESS_DB_PASSWORD: examplepass
WORDPRESS_DB_NAME: exampledb
volumes:
# The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
- ./_wordpress:/var/www/html
db:
image: mysql:5.7
environment:
MYSQL_DATABASE: exampledb
MYSQL_USER: exampleuser
MYSQL_PASSWORD: examplepass
MYSQL_RANDOM_ROOT_PASSWORD: '1'
volumes:
- db:/var/lib/mysql
The difference between host users and container users in Linux
The main difference between how Docker works on macOS and Windows versus how works on Linux is, without delving too deep into technical aspects I'm not expert enough to discuss, there is an additional layer (allow me to call it a "virtual machine") between the macOS or Windows host and the containers that is not present on Linux.
Containers are a Linux technology ported over to macOS and Windows. The ports solve several compatibility problems across different operating systems and mitigate, leveraging the use of "virtual machine" of sorts, some of the key differences between the OSes. Containers ignore the underlying implementation; in a way, Docker (for Windows, for macOS and Linux) acts as Java: create an image, build, publish it, share it, and run wherever Docker runs.
The differences become something to deal with and put some care into when sharing files between the host machine and the container.
With some exceptions, most images are based on Linux OSes, where file permissions are serious business and not a small part of why Linux is the de-facto standard when it comes to servers.
When containers run on Linux, file ownership of shared files is maintained.
What this means is that:
If a file belongs to the user luca on the host machine, then that file will belong to luca in the container.
- Vice-versa, if a file belongs to
www-data in the container, it will belong to www-data on the host machine.
The second would explain why, in my previous post, I could not delete files created by the container from the host machine: my user on the host, luca is not the owner of the files created by the www-data user in the container.
When I tried to do that I would get the following output:
Repos/wp-docker » whoami
luca
Repos/wp-docker » ls -la _wordpress/wp-content/plugins/hello-dolly
total 16
drwxr-xr-x 2 www-data www-data 4096 Jun 6 16:58 .
drwxr-xr-x 4 www-data www-data 4096 Jun 6 16:58 ..
-rw-r--r-- 1 www-data www-data 2593 Jun 6 16:58 hello.php
-rw-r--r-- 1 www-data www-data 623 Jun 6 16:58 readme.txt
Repos/wp-docker » id -u
1002
Repos/wp-docker » id -u www-data
33
To delete the files from the host I am forced to use sudo:
Repos/wp-docker » sudo rm -rf _wordpress/wp-content/plugins/hello-dolly
[sudo] password for luca: ***********
Using sudo is a dangerous habit and a requirement I would like to avoid in this instance.
Fixing the file ownership issue in the WordPress container
The official WordPress image published on Dockerhub supports running Apache as a different user.
Looking at the docker-entrypoint.sh file that will run when the container starts, I see I could set the APACHE_RUN_USER and APACHE_RUN_GROUP environment variables to have the Apache web-server process belong to a specific user.
What user? If on my host Linux machine my user is luca, and I want my user to be able to modify and delete the files shared with the container without requiring the use of sudo, then Apache must be started by luca.
I will specify the user by its user ID (uid) and group ID (gid).
I've shown it before, but, as a refresher, I can fetch the current user ID and group ID from the terminal:
Repos/wp-docker » id -u
1002
Repos/wp-docker » id -g
1002
Usually, on Linux, those same values will be available on the UID and GID environment variables, but I prefer to rely on commands I run to make sure I'm getting the values I need.
I modify the docker-compose.yml file to define the APACHE_RUN_USER and APACHE_RUN_GROUP environment variables:
version: '3.1'
volumes:
db:
services:
wordpress:
image: wordpress
ports:
- 8080:80
environment:
WORDPRESS_DB_HOST: db
WORDPRESS_DB_USER: exampleuser
WORDPRESS_DB_PASSWORD: examplepass
WORDPRESS_DB_NAME: exampledb
APACHE_RUN_USER: ${APACHE_RUN_USER}
APACHE_RUN_GROUP: ${APACHE_RUN_GROUP}
volumes:
# The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
- ./_wordpress:/var/www/html
db:
image: mysql:5.7
environment:
MYSQL_DATABASE: exampledb
MYSQL_USER: exampleuser
MYSQL_PASSWORD: examplepass
MYSQL_RANDOM_ROOT_PASSWORD: '1'
volumes:
- db:/var/lib/mysql
The FOO: ${BAR} notation means "at runtime set the value of the FOO environment variable in the container to the current value of the BAR environment variable on the host". There is no requirement for the two variables to have the same name; it's just convenient.
I try to spin up the stack again, specifying the two environment variables:
APACHE_RUN_USER=$(id -u) \
APACHE_RUN_GROUP=$(id -g) \
docker-compose up
Recreating wp-docker_wordpress_1 ... done
Attaching to wp-docker_wordpress_1
wordpress_1 | AH00543: apache2: bad user name 1002
wp-docker_wordpress_1 exited with code 1
Ok, this makes sense. The user luca does not exist in the container; the Apache process cannot belong to a user that does not exist.
The docker-compose specification allows specifying a user, this is the equivalent of [using the -u|--user option when using the docker run command.
The documentation says:
When passing a numeric ID, the user does not have to exist in the container.
Which is what I need, I want to have a user somehow automagically created for me and I want to run Apache as that user.
I modify the docker-compose.yml file again:
version: '3.1'
volumes:
db:
services:
wordpress:
image: wordpress
ports:
- 8080:80
user: ${WORDPRESS_RUN_USER}:${WORDPRESS_RUN_GROUP}
environment:
WORDPRESS_DB_HOST: db
WORDPRESS_DB_USER: exampleuser
WORDPRESS_DB_PASSWORD: examplepass
WORDPRESS_DB_NAME: exampledb
APACHE_RUN_USER: ${APACHE_RUN_USER}
APACHE_RUN_GROUP: ${APACHE_RUN_GROUP}
volumes:
# The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
- ./_wordpress:/var/www/html
db:
image: mysql:5.7
environment:
MYSQL_DATABASE: exampledb
MYSQL_USER: exampleuser
MYSQL_PASSWORD: examplepass
MYSQL_RANDOM_ROOT_PASSWORD: '1'
volumes:
- db:/var/lib/mysql
I try again setting the two new variables as well:
APACHE_RUN_USER=$(id -u) \
APACHE_RUN_GROUP=$(id -g) \
WORDPRESS_RUN_USER=$(id -u) \
WORDPRESS_RUN_GROUP=$(id -g) \
docker-compose up
wp-docker_wordpress_1 is up-to-date
Creating wp-docker_db_1 ... done
Attaching to wp-docker_wordpress_1, wp-docker_db_1
db_1 | 2020-06-09 11:44:26+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.30-1debian10 started.
wordpress_1 | sed: couldn't open temporary file ./sedEJZmHq: Permission denied
wordpress_1 | sed: couldn't open temporary file ./sedLW2kUW: Permission denied
wordpress_1 | sed: couldn't open temporary file ./sedbvCEiP: Permission denied
wordpress_1 | sed: couldn't open temporary file ./sedEf5PRI: Permission denied
wordpress_1 | sed: couldn't open temporary file ./sed8AOfWS: Permission denied
wordpress_1 | sed: couldn't open temporary file ./sedo8VJU5: Permission denied
wordpress_1 | sed: couldn't open temporary file ./sedpTR0UV: Permission denied
wordpress_1 | sed: couldn't open temporary file ./sed0jXAY3: Permission denied
wordpress_1 | sed: couldn't open temporary file ./sedxBnvZX: Permission denied
wordpress_1 | sed: couldn't open temporary file ./sedgVhJmU: Permission denied
Not what I had expected.
When I run the docker-compose up command, I'm just starting a series of connected containers and running commands in each.
What does "running a command" means?
Containers are a "technology" to run commands in an isolated, self-contained environment.
As an example, the Nginx container will run the nginx -g daemon off command, and the WordPress container I'm using in the current stack will run the apache2-foreground command.
The commands, as is the case with the two I've just listed, might be commands to start a server, but any container image (from which containers are built) is essentially a way to prepare for, and run, a single command. The fact that a command might do something and return in a second or run for weeks does not change how containers work.
It's essential to understand this to appreciate why the user running the command is so important: if the user running the command, or any of the commands required to set up the environment for that command to run, is not authorized, then the container will exit.
By default, Docker containers will run using the root user (id=0).
As is the custom in Linux, the root user is the one who can do everything.
The change, in the docker-compose.yml file, that is causing the issue is the one where I specify a user other than the root one; that user is not authorized to write to /tmp.
In this case, writing to the /tmp directory is required while setting up the WordPress container.
I'm in a bit of a conundrum here: I want to run the final container command, apache2-foreground, to start the Apache web-server as my user (luca) but I cannot run the container as that user (what I just did), and I cannot specify, as APACHE_RUN_USER, a user that does not exist in the container.
Modifying the WordPress container command
What I got from the previous section, is that each container is a way to run a command.
To run the command, in the WordPress container, I can modify its command.
Note: specifying the command parameter in a service description in the docker-compose.yml file is the equivalent of specifying a CMD directive in the image Dockerfile.
On Linux, the user name, e.g. luca is just a local (to the machine) name, and what really has value is the user ID and the ID of the user group.
To make a parallel with WordPress, the user ID is the post ID, and the user name is the post_title: a post will keep being uniquely identifiable as long as the post ID stays the same, the title can change over time.
Knowing this, and keeping the objective of portability in mind, I call my user, the user that has the same user and group ID in the container that I have on my Linux machine (id=1002 gid=1002), docker. The user naming choice follows a common standard.
I modify the docker-compose.yml file section related to the wordpress container to:
- Remove the
user parameter and let the container run using root.
- Create a
docker user with the same uid and gid as my luca user on my Linux machine.
version: '3.1'
volumes:
db:
services:
wordpress:
image: wordpress
ports:
- "8080:80"
# The default entypoint is `/usr/local/bin/docker-entrypoint.sh`. Not overridden here.
# The default command is `apache2-foreground`, to start the Apache web-server.
# The command is overridden to create the `docker:docker` user and group mapped to the host machine
# user and group if they do not exist already.
# After creating the user and group, run the original command.
# The `$$` is to escape the `$` in YAML and avoid docker-compose trying to replace it.
command:
- /bin/sh
- -c
- |
test $$(getent group docker) || addgroup --gid ${APACHE_RUN_GROUP} docker
test $$(id -u docker) || adduser --uid ${APACHE_RUN_USER} --ingroup docker \
--home /home/docker --disabled-password --gecos '' docker
/usr/local/bin/docker-entrypoint.sh apache2-foreground
environment:
WORDPRESS_DB_HOST: db
WORDPRESS_DB_USER: exampleuser
WORDPRESS_DB_PASSWORD: examplepass
WORDPRESS_DB_NAME: exampledb
# Start the Apache server as the `docker` user.
# I've created the user in the `command` section, and this will work now.
APACHE_RUN_USER: "docker"
APACHE_RUN_GROUP: "docker"
volumes:
# The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
- ./_wordpress:/var/www/html
db:
image: mysql:5.7
environment:
MYSQL_DATABASE: exampledb
MYSQL_USER: exampleuser
MYSQL_PASSWORD: examplepass
MYSQL_RANDOM_ROOT_PASSWORD: '1'
volumes:
- db:/var/lib/mysql
When I run the docker-compose up command again, everything will work.
APACHE_RUN_USER=$(id -u) \
APACHE_RUN_GROUP=$(id -g) \
docker-compose up
The WordPress installation files belong to my user:
Repos/wp-docker » ls -la _wordpress
total 224
drwxr-xr-x 5 luca luca 4096 Jun 10 15:39 .
drwxr-xr-x 3 luca luca 4096 Jun 10 15:39 ..
-rw-r--r-- 1 luca luca 234 Jun 10 15:39 .htaccess
-rw-r--r-- 1 luca luca 420 Dec 1 2017 index.php
-rw-r--r-- 1 luca luca 19935 Jan 1 2019 license.txt
-rw-r--r-- 1 luca luca 7368 Sep 2 2019 readme.html
-rw-r--r-- 1 luca luca 6939 Sep 3 2019 wp-activate.php
drwxr-xr-x 9 luca luca 4096 Dec 18 23:16 wp-admin
-rw-r--r-- 1 luca luca 369 Dec 1 2017 wp-blog-header.php
-rw-r--r-- 1 luca luca 2283 Jan 21 2019 wp-comments-post.php
-rw-r--r-- 1 luca luca 3183 Jun 10 15:39 wp-config.php
-rw-r--r-- 1 luca luca 2808 Jun 10 15:39 wp-config-sample.php
drwxr-xr-x 4 luca luca 4096 Jun 10 15:39 wp-content
-rw-r--r-- 1 luca luca 3955 Oct 11 2019 wp-cron.php
drwxr-xr-x 20 luca luca 12288 Dec 18 23:16 wp-includes
-rw-r--r-- 1 luca luca 2504 Sep 3 2019 wp-links-opml.php
-rw-r--r-- 1 luca luca 3326 Sep 3 2019 wp-load.php
-rw-r--r-- 1 luca luca 47597 Dec 9 2019 wp-login.php
-rw-r--r-- 1 luca luca 8483 Sep 3 2019 wp-mail.php
-rw-r--r-- 1 luca luca 19120 Oct 15 2019 wp-settings.php
-rw-r--r-- 1 luca luca 31112 Sep 3 2019 wp-signup.php
-rw-r--r-- 1 luca luca 4764 Dec 1 2017 wp-trackback.php
-rw-r--r-- 1 luca luca 3150 Jul 1 2019 xmlrpc.php
I can now delete the files from my host machine without using sudo:
Repos/wp-docker » rm -rf _wordpress/wp-content/plugins/hello.php
Repos/wp-docker » ls -la _wordpress/wp-content/plugins
total 16
drwxr-xr-x 3 luca luca 4096 Jun 10 15:56 .
drwxr-xr-x 4 luca luca 4096 Jun 10 15:39 ..
drwxr-xr-x 4 luca luca 4096 Dec 18 23:16 akismet
-rw-r--r-- 1 luca luca 28 Jun 5 2014 index.php
And, after I've installed WordPress using its UI at http://localhost:8080, if I install a plugin from the WordPress installation, tat plugin files too will be accessible to my user.
The docker-compose.yml file, in its wordpress service section especially, could be reduced by building a dedicated image customized to my needs. Currently, I prefer the compact form of the single docker-compose.yml file and will leave that for later, if required.
Thinning the command to start the stack
It's a bit cumbersome to have to start the stack using this command:
APACHE_RUN_USER=$(id -u) \
APACHE_RUN_GROUP=$(id -g) \
docker-compose up
It's not impossible to remember, but it's error-prone, and, as the stack evolves, it will become more and more complex.
To get back the ease-of-use I've created a PHP script that will take care of running the command correctly every time with awareness of the current operating system:
##! /usr/bin/env php
<?php
// Default to `up` if no arguments are provided.
// $argv[0] is the script name itself, drop it.
$unescapedArgs = isset($argv[1]) ? (array)array_slice($argv,1) : ['up'];
// Escape the command arguments before using them.
$escapedArgs = array_map('escapeshellarg', $unescapedArgs);
// Get the current OS; `dar` is macOS, `lin` is Linux, `win` is Windows
// Other OSes (*nix, BSD...) require, probably, the same setup steps as for Linux.
$os = substr(strtolower(PHP_OS), 0, 3);
$binary = 'docker-compose';
switch ($os){
case 'win':
$binary = 'docker-compose.exe';
break;
case 'dar':
// There is nothing to do here.
break;
default:
// If we cannot get the current user and group ID use `0` (`root` in the container).
putenv('DOCKER_RUN_USER=' . (int)getmyuid());
putenv('DOCKER_RUN_GROUP=' . (int)getmygid());
break;
}
// Build the command line.
$commandLine = sprintf('%s %s', $binary, implode(' ', $escapedArgs));
// Debug the command line.
echo "\nCommand: {$commandLine}\n\n";
// Let's not apply the time limit to the processes launched by the script.
set_time_limit(0);
// Execute the command.
passthru($commandLine, $status);
// Print a blank line after the command output to clear the terminal a bit.
echo "\n";
// Exit the same status as the command.
exit($status);
I've called the script stack, so I can now run the equivalent of the previous command by running:
./stack up
I've modified the stack to work on Linux, will it be the same on Windows And macOS?
In my next post, I will test the current solution out on both to move a step closer to a portable WordPress testing environment and iterate on the CLI wrapper script.
Previously, in this series
In the first post in this series, I've covered the basic stuff: what Docker and docker-compose are and how they could enable a portable environment for development, and more interesting to me, testing WordPress projects.
In this second article, I will highlight a less covered issue that might seriously impede, or stop altogether, the adoption and use of Docker as a testing environment for WordPress projects.
I've concluded the previous article with this docker-compose.yml file:
version: '3.1'
volumes:
db:
services:
wordpress:
image: wordpress
restart: always
ports:
- 8080:80
environment:
WORDPRESS_DB_HOST: db
WORDPRESS_DB_USER: exampleuser
WORDPRESS_DB_PASSWORD: examplepass
WORDPRESS_DB_NAME: exampledb
volumes:
# Not using the volume defined in the volumes section anymore, so I've removed it.
# The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
- ./_wordpress:/var/www/html
db:
image: mysql:5.7
restart: always
environment:
MYSQL_DATABASE: exampledb
MYSQL_USER: exampleuser
MYSQL_PASSWORD: examplepass
MYSQL_RANDOM_ROOT_PASSWORD: '1'
volumes:
- db:/var/lib/mysql
The file allows me to run a fully working WordPress installation run locally on my macOS machine.
The next step, for this stack to be defined "portable", is to try and spin it up on Windows and Linux machines.
Running the stack on Windows
On Windows, docker and docker-compose can be installed using the Docker for Windows application; there is nothing else to do after installing it but start the Docker for Desktop application.
I can check the version of both commands in a vanilla Windows terminal emulator (Command Prompt):
The structure and content of the directory I'm using are similar to the use I used on macOS:
- The project lives in the
C:\Users\lucatume\Repos\wp-docker directory.
- The directory only contains, to start, the
docker-compose.yml file.
I run the docker-compose up command and wait for images to download and the containers based on the instances to start.
Following the initial set up, the command prompt will tail the logs of the the db and wordpress services, as it happened on macOS, docker-compose created a _wordpress directory where WordPress files are stored, and made WordPress available, locally, at http://localhost:8080:
After completing the WordPress installation using the UI, I want to make sure file modification will work both ways:
- Adding, modifying, or deleting a file on the Windows host will alter the content of the WordPress installation served at
http://localhost:8080.
- Adding, modifying, or deleting a file using the installation UI, say adding a plugin, will reflect on the contents of the
_wordpress directory.
First I runt this command from the C:\Users\lucatume\Repos\wp-docker directory:
del /f .\wp-content\plugins\hello.php
As expected the "Hello Dolly" plugin does not appear in the list of available plugins in the WordPress installation:
I, now, try to re-add the plugin using the WordPres installation plugin administration UI:
While I could be more thorough, I know this means that read/write is working on both sides.
Time to move to Linux.
Running the stack on Linux
Depending on the Linux distribution you're using, the Docker and docker-compose installation steps might differ.
In this example, I'm using a Ubuntu host machine; you can find the installation instructions here.
Whatever the distribution, I've followed the post-installation steps to make sure I can run the docker command without requiring sudo. It's not needed, but it's the reason I'm not using sudo in the examples, and it makes the use of docker and docker-compose easier.
I start by checking the version of the two commands:
I've reproduced the same starting directory structure as the macOS example:
- The project lives in the
/home/luca/Repos/wp-docker directory.
- The directory only contains, to start, the
docker-compose.yml file.
As I did on macOS and Windows before, I spin up the stack with docker-compose up and wait for the WordPress installation to be ready:
As has been the case on macOS and Windows, docker-compose created and filled the wp-docker/_wordpress directory with the WordPress installation contents:
Repos/wp-docker » tree -L 2
.
├── docker-compose.yml
└── _wordpress
├── index.php
├── license.txt
├── readme.html
├── wp-activate.php
├── wp-admin
├── wp-blog-header.php
├── wp-comments-post.php
├── wp-config.php
├── wp-config-sample.php
├── wp-content
├── wp-cron.php
├── wp-includes
├── wp-links-opml.php
├── wp-load.php
├── wp-login.php
├── wp-mail.php
├── wp-settings.php
├── wp-signup.php
├── wp-trackback.php
└── xmlrpc.php
4 directories, 18 files
Again, as I did on macOS and Windows, I check if the read/write operations work correctly between the host and the running WordPress container.
As a first step, I try to delete the "Hello Dolly" plugin, a single file, from the host to see if this has the expected effect on the running WordPress installation.
The expected effect is to not see the "Hello Dolly" plugin among the available plugins.
Repos/wp-docker » rm -rf _wordpress/wp-content/plugins/hello-dolly
rm: cannot remove '_wordpress/wp-content/plugins/hello-dolly/hello.php': Permission denied
rm: cannot remove '_wordpress/wp-content/plugins/hello-dolly/readme.txt': Permission denied
Repos/wp-docker »
Differently from what happened on macOS and Windows, I cannot delete a file created by the container from the host.
I check the file modes to understand why:
Repos/wp-docker » whoami
luca
Repos/wp-docker » ls -la _wordpress/wp-content/plugins/hello-dolly
total 16
drwxr-xr-x 2 www-data www-data 4096 Jun 6 16:58 .
drwxr-xr-x 4 www-data www-data 4096 Jun 6 16:58 ..
-rw-r--r-- 1 www-data www-data 2593 Jun 6 16:58 hello.php
-rw-r--r-- 1 www-data www-data 623 Jun 6 16:58 readme.txt
Repos/wp-docker » id -u
1002
Repos/wp-docker » id -u www-data
33
The id -u [<user>] will return the user ID of either the current user, luca in my example, or the ID of the specified user, www-data in the second example.
What the output above means is:
- the
_wordpress/wp-content/plugins/hello directory is owned by the www-data user, with user ID 33
- my user is
luca, with user ID 1002
- due to how user permission and file ownership work on Linux, a user that has no super-user (
su) rights cannot modify or delete other user files.
Ok, but where does the www-data user come from?!
In short: Apache web-server.
The official wordpress image available on Dockerhub is based on a Linux machine that serves the WordPress installation using Apache; the Apache user has ID 33.
So, in Linux, the owner of the files created by a running container is the user the container is currently using.
I execute a command on the running wordpress container to make sure of this:
Repos/wp-docker » docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a19f327780d2 wordpress "docker-entrypoint.s…" 20 minutes ago Up 20 minutes 0.0.0.0:8080->80/tcp wp-docker_wordpress_1
d21785e9fcca mysql:5.7 "docker-entrypoint.s…" 20 minutes ago Up 20 minutes 3306/tcp, 33060/tcp wp-docker_db_1
Repos/wp-docker » docker exec a19f327780d2 whoami
root
The user of the wordpress container is not www-data, as expected, but root.
And not my root, but the root user of the Linux installation. So: where does that www-data user Apache is running for, come from?
Finding where the www-data user comes from takes some searching, but looking at the official WordPress image files I can see the Apache user will be set to the www-data one if the APACHE_RUN_USER and APACHE_RUN_GROUP environment variables are not specified.
From the Ubuntu host I can still modify the files using sudo:
Repos/wp-docker » sudo rm -rf _wordpress/wp-content/plugins/hello-dolly
[sudo] password for luca:
Repos/wp-docker »
But having to use sudo to delete files or any other modifications to the files belonging to the WordPress installation is not a viable way of working.
In my next post, I will focus on the Linux version of the stack to solve this problem.
The Docker promise, in a WordPress developer context
Docker is almost everywhere now, and there are many good reasons for it.
I'm not going into all the reasons that make Docker amazing. Instead, I would focus on the possibilities it offers me, as a WordPress developer sharing production and test code across different teams and operating systems.
"It works on my machine" might be a meme, but it's still frequent feedback I run into when reviewing pull requests with broken tests that are, invariably, passing locally and failing in CI. Supposedly.
It would be great for a team, or even just for a developer using multiple operating systems, to have identical testing environments running on each developer machine, whatever the operating system.
The purpose of this series is to consolidate the knowledge I got along the way of learning "how to docker" into a progressive series of articles moving past base, copy-and-paste examples. Ideally, to set up to an OS-independent, reusable, flexible docker-compose based, WordPress local, and CI (Continuous Integration) testing and development environment.
Up and done
I'm using, to start, a container I use daily: the official WordPress container from Dockerhub.
This container is a constant in my builds and testing environments. I've spent quite a bit of time learning it's ins-and-outs and collecting several "gotchas along the path.
I've created a minimal docker-compose.yml configuration file copying the example from the dockerhub official WordPress image documentation:
cd ~/Repos
mkdir wp-docker
cd wp-docker
touch docker-compose.yml
I've just moved around some lines but changed nothing.
version: '3.1'
volumes:
wordpress:
db:
services:
wordpress:
image: wordpress
restart: always
ports:
# Expose port 80 of the container on port 8080 of localhost.
- 8080:80
environment:
WORDPRESS_DB_HOST: db
WORDPRESS_DB_USER: exampleuser
WORDPRESS_DB_PASSWORD: examplepass
WORDPRESS_DB_NAME: exampledb
volumes:
- wordpress:/var/www/html
db:
image: mysql:5.7
restart: always
environment:
MYSQL_DATABASE: exampledb
MYSQL_USER: exampleuser
MYSQL_PASSWORD: examplepass
MYSQL_RANDOM_ROOT_PASSWORD: '1'
volumes:
- db:/var/lib/mysql
I run the command to start the stack and wait for the setup to be ready:
docker-compose up
If the images are not already present on my machine, they will be downloaded and extracted first, and this will happen only once the first time I require new images.
After a bit of setup and build processes, I will be left with a terminal tab or window following (tailing) the two containers log files:
db_1 | 2020-05-16T11:24:01.540475Z 0 [Note] Event Scheduler: Loaded 0 events
db_1 | 2020-05-16T11:24:01.540824Z 0 [Note] mysqld: ready for connections.
db_1 | Version: '5.7.29' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
wordpress_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.80.3. Set the 'ServerName' directive globally to suppress this message
wordpress_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.80.3. Set the 'ServerName' directive globally to suppress this message
wordpress_1 | [Sat May 16 11:24:03.714651 2020] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.3.16 configured -- resuming normal operations
wordpress_1 | [Sat May 16 11:24:03.714774 2020] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
In the docker-compose.yml file I've specified, in services.wordpress.ports section, that I would like to expose port 80 of the container on port 8080 of localhost.
Up and down and up again
When I open the http://localhost:8080 address in my browser, though, WordPress will not be installed.
I go through the installation and will end up with a working WordPress installation:
If I stop the WordPress and database server closing the terminal tab running the logs or sending it the termination command (Ctrl+c), then the site will not be available anymore at http://localhost:8080, as expected.
How would I restart the WordPress installation to work on it, though? Using the same command I've used before:
docker-compose up
Do I need to install WordPress again? No.
In the docker-compose.yml file I've defined two volumes: wordpress and db.
The wordpress volume will store, in my host machine filesystem, the WordPress container files, the content of the WordPress installation.
The db volume will store the data produced by the database instance used by WordPress, again in my host machine filestystem.
The concept of volumes and "Docker volumes" was tricky for me to grasp until I've understood that it just means "store what files you create or update during your work on my machine".
So where are the files?
If they are on my machine, then I would expect the files to be somewhere I can see them; in the same directory containing the docker-compose.yml file, ideally.
Yet they are not there, the directory contains only, even while the wordpress and db containers are running, the docker-compose.yml file:
Repos/wp-docker » tree -L 1 .
.
└── docker-compose.yml
0 directories, 1 file
Getting the path to the real location where, in the host filesystem, the files live requires a bit of inspection into how Docker stores container information.
First, I list the currently defined WordPress containers with docker volume ls:
Repos/wp-docker » docker volume ls
DRIVER VOLUME NAME
local wp-docker_db
local wp-docker_wordpress
As expected, Docker created two volumes have named after the directory where the docker-compose.yml file lives:
wp-docker_wordpress for the wordpress volumewp-docker_db for the db volume
Since I want to see the WordPress container files, the container I'm interested in is the wp-docker_wordpress one.
The command docker volume inspect wp-docker_wordpress will provide me with useful information about the volume:
Repos/wp-docker » docker volume inspect wp-docker_wordpress
[
{
"CreatedAt": "2020-05-16T11:46:15Z",
"Driver": "local",
"Labels": {
"com.docker.compose.project": "wp-docker",
"com.docker.compose.version": "1.25.5",
"com.docker.compose.volume": "wordpress"
},
"Mountpoint": "/var/lib/docker/volumes/wp-docker_wordpress/_data",
"Name": "wp-docker_wordpress",
"Options": null,
"Scope": "local"
}
]
So, are the wordpress volume files, used by the wordpress container, from the wp-docker project, in the /var/lib/docker/volumes/wp-docker_wordpress/_data directory?
Turns out no, they are not:
Repos/wp-docker » ls -la /var/lib/docker/volumes/wp-docker_wordpress/_data
ls: /var/lib/docker/volumes/wp-docker_wordpress/_data: No such file or directory
I could go into the rabbit hole of trying and finding them, but Docker and docker-compose provide a structured and documented way to store the container data exactly where I need it.
Binding volumes
Binding volumes is container jargon to say:
When I put stuff in this directory on my machine, it should appear in this directory in the container. Possibly the other way too.
I do want the WordPress installation files to exist, on my host machine, somewhere practical.
Specifically in the ~/Repos/wp-docker/_wordpress directory.
I update the docker-compose.yml file to reflect that.
version: '3.1'
volumes:
db:
services:
wordpress:
image: wordpress
restart: always
ports:
- 8080:80
environment:
WORDPRESS_DB_HOST: db
WORDPRESS_DB_USER: exampleuser
WORDPRESS_DB_PASSWORD: examplepass
WORDPRESS_DB_NAME: exampledb
volumes:
# Not using the volume defined in the volumes section anymore, so I've removed it.
# The `.` means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
- ./_wordpress:/var/www/html
db:
image: mysql:5.7
restart: always
environment:
MYSQL_DATABASE: exampledb
MYSQL_USER: exampleuser
MYSQL_PASSWORD: examplepass
MYSQL_RANDOM_ROOT_PASSWORD: '1'
volumes:
- db:/var/lib/mysql
Two things to notice:
I've changed the services.wordpress.volumes entry to ./_wordpress:/var/www/html.
The . in the ./_wordpress first fragment of the wordpress service volume definition means "from the directory that contains this file", where "this file" is the docker-compose.yml file.
In docker-compose.yml files, as in the -v parameter of the docker CLI API, volume bindings are read on_the_host:on_the_container. The same applies to port bindings.
The first, the host part of a volume binding can either be a path relative to the docker-compose.yml directory, or an absolute path. I'm using the first option here to make the implementation portable.
Since I will not be using the wordpress volume anymore, I've removed it from the file.
If all goes well, firing up the stack again should create a _wordpress directory beside the docker-compose.yml file and put WordPress root directory in there.
I see WordPress
I fire up the stack again and wait, looking at the logs, for the message from the WordPress container to confirm it's ready.
Repos/wp-docker » docker-compose up -d
I open a new terminal tab and check what's in the project directory:
Repos/wp-docker » tree -L 1 .
.
└── docker-compose.yml
0 directories, 1 file
Nothing happened. Why?
Well: Docker is an efficient worker and, since I ran the stack before, will "recycle", reuse, the containers I was running the last time.
And the wordpress container I used before was never told to bind the /var/www/html directory to the _wordpress one in the project root directory. So it didn't.
How do I take control of this back? By tearing down the stack and firing it up again with the down (short for "tear down") command:
Repos/wp-docker » docker-compose down
Stopping wp-docker_wordpress_1 ... done
Stopping wp-docker_db_1 ... done
Removing wp-docker_wordpress_1 ... done
Removing wp-docker_db_1 ... done
Removing network wp-docker_default
Repos/wp-docker » docker-compose up
Creating network "wp-docker_default" with the default driver
Creating wp-docker_wordpress_1 ... done
Creating wp-docker_db_1 ... done
Attaching to wp-docker_db_1, wp-docker_wordpress_1
db_1 | 2020-05-17 11:16:48+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.29-1debian9 started.
db_1 | 2020-05-17 11:16:49+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
db_1 | 2020-05-17 11:16:49+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.29-1debian9 started.
db_1 | 2020-05-17T11:16:49.337125Z 0 [Warning
...
db_1 | Version: '5.7.29' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)
wordpress_1 | WordPress not found in /var/www/html - copying now...
wordpress_1 | Complete! WordPress has been successfully copied to /var/www/html
wordpress_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.96.3. Set the 'ServerName' directive globally to suppress this message
wordpress_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.96.3. Set the 'ServerName' directive globally to suppress this message
wordpress_1 | [Sun May 17 11:16:59.711493 2020] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.3.16 configured -- resuming normal operations
wordpress_1 | [Sun May 17 11:16:59.711569 2020] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
Can I see WordPress files in the _wordpress directory now?!
Repos/wp-docker » tree -L 2 .
.
├── _wordpress
│ ├── index.php
│ ├── license.txt
│ ├── readme.html
│ ├── wp-activate.php
│ ├── wp-admin
│ ├── wp-blog-header.php
│ ├── wp-comments-post.php
│ ├── wp-config-sample.php
│ ├── wp-config.php
│ ├── wp-content
│ ├── wp-cron.php
│ ├── wp-includes
│ ├── wp-links-opml.php
│ ├── wp-load.php
│ ├── wp-login.php
│ ├── wp-mail.php
│ ├── wp-settings.php
│ ├── wp-signup.php
│ ├── wp-trackback.php
│ └── xmlrpc.php
└── docker-compose.yml
4 directories, 18 files
I got myself a WordPress copy, ready to run, and in my host machine file system.
Trusting no one, I want to see if doing something on the host would modify the files on the container and vice-versa.
Two-way binding test
The first test will be to delete the "Hello Dolly" plugin from the host machine and see if the "Hello Dolly" plugin does disappear in the container:
rm ./_wordpress/wp-content/plugins/hello.php
A check-in the WordPress installation, at http://localhost:8080/wp-admin/plugins.php confirms my expectation:
To test that changing something in the container would, in turn, change it on the guest I will try to install the Hello Dolly plugin from the WordPress plugin repository again; if all goes as intended, then it should re-appear on the host.
Repos/wp-docker » tree -L 1 ./_wordpress/wp-content/plugins
./_wordpress/wp-content/plugins
├── akismet
├── hello-dolly
└── index.php
2 directories, 1 file
And it does.
This confirms volume binding works as intended.
On macOS. Will it be the same on Linux? And Windows?
Spoiler: no, it requires some leg work and understanding of file modes that I will plunge into in my next post.
This post is the third in a series; find the first one here and the second one here.
In this series, I'm documenting the steps to add parallel, container-based, builds to the wp-browser project.
To set expectations: as a result of this research and effort, I might add a container-dedicated command to wp-browser in the future, but this is not my primary goal now.
My main goal is being able to run wp-browser own CI builds (on Travis CI, at the moment) faster and more reliably.
Each step, reasoning, and assumption in these posts is part of my discovery process; as such, these might be confused, out of order, or outright wrong. Ye be warned.
You can find the final post shown in this post here.
Running make tasks in parallel
After the round-robin information and data collection I've done in the second post, before I spend any time developing further, I want to make sure the docker-compose based approach to parallel execution works.
I'm using make to run the scripts, and I do not know, at this stage, if this is the best way, but it's what I can immediately work with now.
The make utility comes with parallel execution support out of the box, I've used it a couple of times, but never with intent.
I set up a small test script and a new target in the project Makefile to test this out:
## Define the list of scripts to run
SCRIPTS = foo baz bar
## Generate the targets from the list of scripts.
targets = $(addprefix php_script_, $(SCRIPTS))
## Generate the targets.
$(targets): php_script_%:
@php -r '$$i=0; while( $$i<5 ){ echo "PHP Script $@, run " . $$i++ . PHP_EOL; sleep( 5 / random_int( 1,10 ) ); }'
## Define a target in charge of running the generated targets.
pll: $(targets)
There's a bit of make syntax and constructs here, but the general catch is that I'm generating a list of "targets" ("tasks to run" in make terminology) from a list and running them with the pll (short for "parallel" command).
Each PHP script runs 5 times and sleeps a random number of tenths of a second before resuming.
Please note the double $$ symbol: without going into too much detail into how make works, any single $ symbol would be interpreted, by make as a make variable; since PHP variable sign of choice is the same (the $ symbol) I've doubly escaped it. Each target is executing a bash command like this:
<?php
$i = 0;
while( $i<5 ){
echo 'PHP Script $@, run ' . $i++ . PHP_EOL;
sleep( 5 / random_int( 1,10 ) );
}
I execute the three recipes in parallel using:
make -j pll
To get the typical parallel processing output where the output from each command is mixed with all the others.
[
](https://theaveragedev.com/wp-content/uploads/2019/08/make-parallel-1.png)
To order the output a bit, I can use the -O flag (GNU version of make only, I use that on my Mac machine):
make -j -O pll
The target order might, in this case, be random, but the output is grouped by the target, depending on their completion order.
[
](https://theaveragedev.com/wp-content/uploads/2019/08/make-parallel-2.png)
Failing some parallels task
The next step is making sure the parallel execution does not break when one or more, of the tasks, fail.
With the objective of speeding up wp-browser builds by running each suite in parallel, really running any Codeception command in parallel, failing builds are a reality I have to cope with and react to.
I've added a second group of test targets running this randomly failing PHP script:
<?php
$i = 0;
while( $i<5 ){
echo 'PHP Script $@, run ' . $i++ . PHP_EOL;
$random_int = random_int(1,10);
if($random_int < 3){
// Fail if the value is less than 3.
exit( 1 );
}
sleep( 5 / $random_int );
}
echo 'PHP Script $@ completed.' . PHP_EOL;
Running the scripts with, and without, the -O option yields the following results:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/make-parallel-3.png)
[
](https://theaveragedev.com/wp-content/uploads/2019/08/make-parallel-4.png)
In the last screenshot not only I'm running the parallel "builds" using the -O flag, but I'm also printing the exit status code of the whole make run.
The exit code of 2 indicates an error in the Makefile; since all the build systems I know of take any non-zero value as an error, this is fine as an indication the build failed.
The problems of parallelism
As anything in development, parallelism comes with its own set of problems.
The first and most obvious one is that the order of termination is not guaranteed, while the second, and less obvious one, is that parallel processes accessing non-parallel resources generating race conditions and possible inconsistencies.
I can better explain this with an example.
I've added another target to the Makefile:
RUNS = 1 2 3
docker_runs = $(addprefix run_, $(RUNS))
$(docker_runs): run_%:
@docker-compose run --rm wpbrowser run test_suite -f -q --no-rebuild
pll_docker_builds: $(docker_runs)
The target is running the same test suite, the test_suite one I've created to test this behavior, three times.
The test_suite suite contains only the following test case:
<?php
class FileTest extends \Codeception\Test\Unit
{
protected static function getFiles()
{
$files = [ 'one', 'two', 'three', 'four', 'five', 'six', 'seven' ];
return $files;
}
protected static function cleanFiles()
{
foreach (static::getFiles() as $file) {
$filePath = codecept_output_dir($file);
if (file_exists($filePath)) {
unlink($filePath);
}
}
}
public function setUp()
{
static::cleanFiles();
}
public function tearDown()
{
static::cleanFiles();
}
public function filesDataSet()
{
foreach (static::getFiles() as $file) {
yield $file => [ $file ];
}
}
/**
* @dataProvider filesDataSet
*/
public function test_files_can_be_created_and_removed($file)
{
// Make sure no file initially exists.
$filePath = codecept_output_dir($file);
$this->assertFileNotExists($filePath);
touch($filePath);
$this->assertFileExists($filePath);
}
}
The test case only has one test method: that test method iterates on seven file names, make sure each file does not exist already, create each file in the Codeception output directory, and make sure that file exists.
All tests pass when running one after the other:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/parallelism-issues-1.png)
But fail when running in parallel, seemingly at random:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/parallelism-issues-2.png)
In its simplicity, it showcases the issue of parallelism: shared resources, the host machine shared filesystem in this specific case.
In the docker-compose.yml file I'm using I'm binding the host machine local filesystem, specifically the wp-browser folder, in the /project folder of each container:
version: '3.2'
services:
wpbrowser:
# Instead of using a pre-built image, let's build it from the file.
build:
context: ./docker/wpbrowser
dockerfile: Dockerfile
args:
# By default use PHP 7.3 but allow overriding the version.
- BUILD_PHP_VERSION=${BUILD_PHP_VERSION:-7.3}
- BUILD_PHP_TIMEZONE=${BUILD_PHP_TIMEZONE:-UTC}
- BUILD_PHP_IDEKEY=${BUILD_PHP_IDEKEY:-PHPSTORM}
- BUILD_XDEBUG_ENABLE=${BUILD_XDEBUG_ENABLE:-1}
volumes:
# Bind the project files into the /project directory.
- ./:/project
environment:
# As XDebug remote host use the host hostname on Docker for Mac or Windows, but allow overriding it.
# Build systems are usually Linux-based.
# Use default port, 9000, by default.
XDEBUG_CONFIG: "remote_host=${XDEBUG_REMOTE_HOST:-host.docker.internal} remote_port=${XDEBUG_REMOTE_PORT:-9000}"
# I use PHPStorm as IDE and this makes the handshake easier.
PHP_IDE_CONFIG: "serverName=wp.test"
That binding means each container has its own, independent, filesystem, but each container synchronizes, in real-time, its filesystem modifications to the host machine filesystem; this affects, implicitly, all other containers synchronizing their filesystem with the host machine filesystem. In even shorter terms: all containers are reading, and writing, from the same source.
Why would this be a problem for the wp-browser project? Or for any project using wp-browser to run WordPress tests?
Because wp-browser has extensions and modules using the filesystem.
The reason being that, at times, the only way to affect WordPress behavior is to scaffold themes or plugins (or must-use plugins) "on the fly" and remove them after the tests.
This need to modify the installation files could be involved in something simpler, like testing media attachments.
Long story short: the containers should not share their files; ideally each should work on its copy of them.
Can this filesystem isolation be achieved?
Resolving the container filesystem issues
The docker-compose file syntax allows limiting what containers can do with bound host directories by specifying that a bound directory is read-only.
While I do not want the containers affecting the host, I do need the containers to be able to write on their files, so the read-only approach is not the correct one.
The simple solution is modifying the test container Dockerfile to copy over the project files and rebuilding the container before each parallel run.
I've modified the docker-compose.yml file to remove the volume entry and update the wpbrowser.build.context to the project root folder:
version: '3.2'
services:
wpbrowser:
# Instead of using a pre-built image, let's build it from the file.
build:
# The context is the root folder any relative host machine path wil refer to in the Dockerfile.
context: .
dockerfile: docker/wpbrowser/Dockerfile
args:
# By default use PHP 7.3 but allow overriding the version.
- BUILD_PHP_VERSION=${BUILD_PHP_VERSION:-7.3}
- BUILD_PHP_TIMEZONE=${BUILD_PHP_TIMEZONE:-UTC}
- BUILD_PHP_IDEKEY=${BUILD_PHP_IDEKEY:-PHPSTORM}
- BUILD_XDEBUG_ENABLE=${BUILD_XDEBUG_ENABLE:-1}
environment:
# As XDebug remote host use the host hostname on Docker for Mac or Windows, but allow overriding it.
# Build systems are usually Linux-based.
# Use default port, 9000, by default.
XDEBUG_CONFIG: "remote_host=${XDEBUG_REMOTE_HOST:-host.docker.internal} remote_port=${XDEBUG_REMOTE_PORT:-9000}"
# I use PHPStorm as IDE and this makes the handshake easier.
PHP_IDE_CONFIG: "serverName=wp.test"
I've also modified the docker/wpbrowser/Dockerfile to COPY over the project files during build:
## Take the BUILD_PHP_VERSION argument into account, default to 7.3.
ARG BUILD_PHP_VERSION=7.3
## Allow for the customization of other PHP parameters.
ARG BUILD_PHP_TIMEZONE=UTC
ARG BUILD_PHP_IDEKEY=PHPSTORM
ARG BUILD_XDEBUG_ENABLE=1
## Build from the php-cli image based on Debian Buster.
FROM php:${BUILD_PHP_VERSION}-cli-buster
LABEL maintainer="luca@theaveragedev.com"
## Install XDebug extension.
RUN pecl install xdebug
## Create the /project directory and change to it.
WORKDIR /project
## Create an executable wrapper to execute the project vendor/bin/codecept binary appending command arguments to it.
RUN echo '#! /usr/bin/env bash\n\
## Run the command arguments on the project Codeception binary.\n\
vendor/bin/codecept $@\n' \
> /usr/local/bin/codecept \
&& chmod +x /usr/local/bin/codecept
## By default run the wrapper when the container runs.
ENTRYPOINT ["/usr/local/bin/codecept"]
## At the end to leverage Docker build cache.
COPY . /project
## Set some ini values for PHP.
## Among them configure the XDebug extension.
RUN echo "date.timezone = ${BUILD_PHP_TIMEZONE}\n\
\n\
[xdebug]\n\
zend_extension=xdebug.so\n\
xdebug.idekey=${BUILD_PHP_IDEKEY}\n\
xdebug.remote_enable=${BUILD_XDEBUG_ENABLE}\n\
" >> /usr/local/etc/php/conf.d/99-overrides.ini
I've finally modified the Makefile to force a build, that benefitting from Docker build cache, before running the tests:
RUNS = 1 2 3
docker_runs = $(addprefix run_, $(RUNS))
$(docker_runs): run_%:
@docker-compose run --rm wpbrowser run test_suite -f -q --no-rebuild
build_test_container:
@docker-compose build
pll_docker_builds: $(docker_runs)
Since parallel execution, in make applies to any specified target, I have to ensure the build_test_container target runs before the pll_docker_builds one.
It's as simple as concatenating two calls to make:
make build_test_container && make -j -O pll_docker_builds
[
](https://theaveragedev.com/wp-content/uploads/2019/08/good-parallel-1.png)
You can find the code I've shown in this post here
Next
I need to put all this together to try and run all the suites in parallel.
New challenges await me as not all suites have no infrastructure requirements as the unit suite does, and many require a database, a web-server, and a Chromedriver container to run acceptance tests.
This post is the second in a series, and you can find the first one here.
In my previous post I've written code that would work on my machine, a Mac OS machine.
To make this of any value, to myself and others, I want to spend some time trying to make the simple setup I have work with Windows and Linux too.
Since PHPStorm and similar IDEs are widely available on all platforms, the part I need to tweak and update is the one dealing with the stack setup.
Making it work on Linux
Looking at the current version of the docker-compose.yml file, making the current set up work on Linux requires the XDEBUG_REMOTE_HOST environment variable to be correctly set:
version: '3.2'
services:
wpbrowser:
# Instead of using a pre-built image, let's build it from the file.
build:
context: ./docker/wpbrowser
dockerfile: Dockerfile
args:
# By default use PHP 7.3 but allow overriding the version.
- BUILD_PHP_VERSION=${TEST_PHP_VERSION:-7.3}
volumes:
# Bind the project files into the /project directory.
- ./:/project
environment:
# As XDebug remote host use the host hostname on Docker for Mac or Windows, but allow overriding it.
# Build systems are usually Linux-based.
# Use default port, 9000, by default.
XDEBUG_CONFIG: "remote_host=${XDEBUG_REMOTE_HOST:-host.docker.internal} remote_port=${XDEBUG_REMOTE_PORT:-9000}"
# I use PHPStorm as IDE and this makes the handshake easier.
PHP_IDE_CONFIG: "serverName=wp.test"
From a terminal emulator, I can get the docker host IP address with this command:
ip -4 addr show docker0 | grep -Po 'inet \K[\d.]+'
It's not my finding or invention: I've found it around the web, precisely here.
Putting the pieces together, this one-line command will allow me to run unit tests, with XDebug support, on Linux:
XDEBUG_REMOTE_HOST="$(ip -4 addr show docker0 | grep -Po 'inet \K[\d.]+')" \
docker-compose run --rm -e XDEBUG_REMOTE_HOST="${XDEBUG_REMOTE_HOST}" wpbrowser run unit
It works, as shown in the screenshots below; it's not the most "accessible" and most effortless command to remember, but it works.
[
](https://theaveragedev.com/wp-content/uploads/2019/08/xdebug-on-linux-2.png)
[
](https://theaveragedev.com/wp-content/uploads/2019/08/xdebug-on-linux-3.png)
[
](https://theaveragedev.com/wp-content/uploads/2019/08/xdebug-on-linux-1.png)
Making it work on Windows
According to the Docker for Windows documentation Docker on Windows, I'm excluding WSL at this stage, should expose the container host at the host.docker.internal hostname.
After I've installed Docker on Windows, I launch cmder and make sure all works as expected with the following commands:
docker --version
docker run --rm -it alpine ping host.docker.internal
[
](https://theaveragedev.com/wp-content/uploads/2019/08/docker-on-windows-output-1.png)
I've already installed PHP, the extensions required by wp-browser , and Composer on the Windows machine using chocolatey.
Time to install Composer dependencies using composer install and wait for the dependencies installation to finish.
I configure PHPStorm the same way I've configured it on Mac:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/xdebug-on-windows-1.png)
[
](https://theaveragedev.com/wp-content/uploads/2019/08/xdebug-on-windows-2.png)
Running the same docker-compose (with some differences) command I've used to run the tests on Mac should yield the same results.
docker-compose.exe build
docker-compose.exe run --rm wpbrowser run unit
So it does.
[
](https://theaveragedev.com/wp-content/uploads/2019/08/xdebug-on-windows-3.png)
Windows is not my daily driver, but I have to admit the experience, so far, is a pleasing one.
Making it work on Windows Subsystem for Linux (WSL)
The round would not be complete without trying out WSL.
I've installed Bash on Windows, the required PHP packages and made sure I can connect to the Docker daemon from the bash session.
I've followed this guide and it worked... flawlessly.
[
](https://theaveragedev.com/wp-content/uploads/2019/08/docker-on-wsl-1.png)
Due to the number of layers involved, I'm taking baby steps to ensure all the moving parts are there are working as expected.
The first step is making sure I can get the IP address of the host machine (Windows) from WSL. I use the same command I would use on a Linux host:
ip -4 addr show docker0 | grep -Po 'inet \K[\d.]+'
This command, though, does not have the desired effect: the docker0 network does not exist in the context of a WSL session.
[
](https://theaveragedev.com/wp-content/uploads/2019/08/docker-on-wsl-2.png)
This result is not surprising: the docker client is connecting to the Docker host, see the blog post I've linked above, via the tco://localhost:2375 address.
On a real Linux machine, the Docker server would be available on a socket; it's not the case in WSL.
If I'm still using the Windows implementation of Docker when using WSL, then this means I should be able to ping the Docker host from the containers using host.docker.internal.
I'm using the alpine container to make sure of that:
docker run --rm -it alpine ping host.docker.internal
[
](https://theaveragedev.com/wp-content/uploads/2019/08/docker-on-wsl-3.png)
As expected, it works.
Since the Docker host is still the Windows machine, the PHPStorm configuration is not changed and the XDebug connection just works:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/xdebug-on-wsl-1.png)
As the last step I want, in the context of Windows and WSL, know what the PHP_OS_FAMILY constant value is.
I run the same command on a cmder and and WSL (Ubuntu 18.04) session:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/php-on-windows-1.png)
From PHP point of view, then, WSL is a Linux system.
For the sake of completeness running the same command on Mac and Linux yields the following results:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/php-on-mac-1.png)
[
](https://theaveragedev.com/wp-content/uploads/2019/08/php-on-linux-1.png)
Next
The next logical step is hiding the complexity involved in "looking around" and running the correct commands in a script.
Codeception is using the Symfony Console component to manage its own CLI, and wp-browser is doing the same with its custom commands; adding another command, maybe more, dedicated to running the tests in containers seems a reasonable choice.
The problem
In a perfect world, tests should "fail fast".
Failing "fast" means the tests should fail as soon as possible, providing me enough feedback to go back, fix the code, and try again.
As the volume of tests and test suites mounts, this proposition is far removed from the seconds-long run time of unit tests to move into the minutes-long run time of multiple acceptance suites.
Even worse: at times the testing load is so heavy for a local machine, or a locally-deployed infrastructure, that tests time out or otherwise fail for the wrong reasons.
I run into this issue while working on wp-browser and dealing with its test base; the solution is to close any other application and cross my fingers as my laptop churns along.
Yet this is not a sustainable approach.
In this series of posts I try and document the steps, and the thought process behind those steps, to set up a Docker-based parallel build that I should be able to run locally and on the build system of choice (currently Travis CI).
Without having to quit the world to run them from start to finish, possibly.
The files I'm showing in these posts are current to wp-browser version 2.2.18.
Disclaimer about my incompetence
I'm not a DevOps person. I know how to set up and use the systems I need to deliver code, but not much more. I'm laying out my reasoning and documenting my approaches, but those might be wrong at any step: please keep that in mind while reading through.
The current build system
Before I start to write code like a mad man it's worth illustrating what I have now as a working (although with some hiccups, at times) build system.
When I open a Pull-Request or push code in any way Travis CI runs the build from this configuration file (you can see the configuration file here):
sudo: required
language: php
notifications:
email: false
matrix:
include:
- php: '5.6'
env: CODECEPTION_VERSION="^2.5"
- php: '5.6'
env: CODECEPTION_VERSION="^3.0"
- php: '7.0'
env: CODECEPTION_VERSION="^2.5"
- php: '7.0'
env: CODECEPTION_VERSION="^3.0"
- php: '7.1'
env: CODECEPTION_VERSION="^2.5"
- php: '7.1'
env: CODECEPTION_VERSION="^3.0"
- php: '7.2'
env: CODECEPTION_VERSION="^2.5"
- php: '7.2'
env: CODECEPTION_VERSION="^3.0"
services:
- docker
cache:
apt: true
directories:
- $HOME/.composer/cache/files
addons:
hosts:
- wp.test
- test1.wp.test
- test2.wp.test
- blog0.wp.test
- blog1.wp.test
- blog2.wp.test
- mu-subdir.test
- mu-subdomain.test
env:
global:
- WP_FOLDER="vendor/johnpbloch/wordpress-core"
- WP_URL="http://wp.test"
- WP_DOMAIN="wp.test"
- DB_NAME="test_site"
- TEST_DB_NAME="test"
- WP_TABLE_PREFIX="wp_"
- WP_ADMIN_USERNAME="admin"
- WP_ADMIN_PASSWORD="admin"
- WP_SUBDOMAIN_1="test1"
- WP_SUBDOMAIN_1_TITLE="Test Subdomain 1"
- WP_SUBDOMAIN_2="test2"
- WP_SUBDOMAIN_2_TITLE="Test Subdomain 2"
matrix:
- WP_VERSION=latest
before_install:
- make ci_before_install
- make ensure_pingable_hosts
# Make Composer binaries available w/o the vendor/bin prefix.
- export PATH=vendor/bin:$PATH
install:
- make ci_install
before_script:
- make ci_before_script
script:
- make ci_script
- php docs/bin/sniff
I've configured the build to run a matrix of tests for PHP and Codeception versions, but There are some parts here that are worth pointing out in more detail.
I use make
You can see the Makefile here.
I find myself comfortable using make and Makefiles to automate my stuff.
I would not use make in code I share with others, but I'm fine using it here.
I've been using make in my previous life and enjoy its low-level approach to the tasks; if you've never used make before it's (I apologize to the experts for the over-simplification) an automation tool like composer run, npm run, gulp, grunt and similar would be.
But meaner and badder.
I run the tests on the host
I currently run the tests from the host machine, as one can see looking at the part of the Makefile in charge of running the tests, the make ci_script target. I use a docker-compose stack to spin up the containers I need, set them up, and run the tests from the host machine relying on the containers to serve the web requests and the database.
This detail is crucial as it's one of the limiting factors of the current build system: since each container with access to the code might modify the content of the host files (e.g., by adding a must-use plugin) having multiple Docker containers all accessing the same host files at the same time could lead to inconsistencies.
You can see the docker-compose file here.
I run the test suites sequentially
You can see the relevant make target here.
Due to limitations imposed by WordPress use of globals and constants, I'm running each suite with a dedicated command.
I want, and must, keep this behavior, but I would like the test runs to be non-blocking and parallelized, not dissimilar to what Codeception documentation suggests.
My initial idea is to leverage make built-in parallelism support to run each suite in parallel to the others; it might not be attainable, but I would like to keep the moving parts simple, if I can.
Objective 1: running unit tests in the container
I've forked latest master, version 2.2.18 of wp-browser, and started working on that.
The first objective is the ability to run the unit suite tests with one command, and the tests should run in the container.
Furthermore, I should be able to debug the tests using XDebug; this means I will bind files into the container in place of relying on the container-managed files.
I want to keep the ci_script target in the Makefile to "hide" more complex commands, issued to the docker-compose binary, in it, but am not against having something I could run in a Composer script too.
I feel the following command is an acceptable "API" to run the unit suite from the project root folder:
docker-compose run --rm wpbrowser run unit
Breaking down the command a bit: * I'm running the wpbrowser container and removing it afterward, which means I'm using the container as an executable * in that container I'm running the run unit command
I'll be implementing the build infrastructure with a test-driven development approach; with some adaptations, of course.
Running the command, right now, yields this error:
ERROR:
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?
Supported filenames: docker-compose.yml, docker-compose.yaml
Time to create a docker-compose.yml file in the root folder:
version: '3.2'
services:
wpbrowser:
# Instead of using a pre-built image, let's build it from the file, see later.
build:
context: ./docker/wpbrowser
dockerfile: Dockerfile
args:
# In the CI environment I need to test against different PHP versions.
# Default to 7.3 but allow for its override.
- BUILD_PHP_VERSION=${TEST_PHP_VERSION:-7.3}
volumes:
# Bind the project files into the /project directory.
- ./:/project
In the docker/wpbrowser folder I've created the following Dockerfile:
## Take the BUILD_PHP_VERSION argument into account, default to 7.3.
ARG BUILD_PHP_VERSION=7.3
## Build from the php-cli image based on Debian Buster.
FROM php:${BUILD_PHP_VERSION}-cli-buster
LABEL maintainer="luca@theaveragedev.com"
## Set PHP timezone to avoid warnings.
RUN echo "date.timezone = UTC" >> /usr/local/etc/php/php.ini
## Create the /project directory and change to it.
WORKDIR /project
## Create an executable wrapper to execute the project vendor/bin/codecept binary appending command arguments to it.
RUN echo '#! /usr/bin/env bash\n\
## Run the command arguments on the project Codeception binary.\n\
vendor/bin/codecept $@\n' \
> /usr/local/bin/codecept \
&& chmod +x /usr/local/bin/codecept
## By default run the wrapper when the container runs.
ENTRYPOINT ["/usr/local/bin/codecept"]
With these files in place, I can run the following command successfully and see the tests run:
docker-compose build
docker-compose run --rm wpbrowser run unit
Objective 2: debugging unit tests with XDebug
Whatever tests I'm running, I need to debug them using XDebug.
Code coverage generation requires XDebug so it's better addressing the issue now, while the stack is simple.
A rapid introduction to XDebug
Being able to debug one's code is considered a junior developer required skill; I very much agree with that.
XDebug, when used to run remote debug sessions, has two components to it: a client and a server.
Similarly to many client-server architectures, the client sends requests to the server, and the server processes them.
I've seen many developers failing to understand, though, that the machine running the PHP code is the client and the machine debugging the code is the server.
In this container stack, then, the client is the container, and the server is my debug tool of choice: PHPStorm running on my host machine (my laptop).
The second part of understanding is the one where the client needs to know-how, and when, to make requests to the server; this is accomplished by installing the XDebug PHP extension.
Installing and configuring XDebug in the container
Installing it is as simple as modifying the docker/wpbrowser/Dockerfile file:
## Take the BUILD_PHP_VERSION argument into account, default to 7.3.
ARG BUILD_PHP_VERSION=7.3
## Allow for the customization of other PHP parameters.
ARG BUILD_PHP_TIMEZONE=UTC
ARG BUILD_PHP_IDEKEY=PHPSTORM
## Build from the php-cli image based on Debian Buster.
FROM php:${BUILD_PHP_VERSION}-cli-buster
LABEL maintainer="luca@theaveragedev.com"
## Install XDebug extension.
RUN pecl install xdebug
## Set some ini values for PHP.
## Among them configure the XDebug extension.
RUN echo "date.timezone = ${BUILD_PHP_TIMEZONE}\n\
\n\
[xdebug]\n\
zend_extension=xdebug.so\n\
xdebug.idekey=${BUILD_PHP_IDEKEY}\n\
xdebug.remote_enable=1\n\
" >> /usr/local/etc/php/conf.d/99-overrides.ini
## Create the /project directory and change to it.
WORKDIR /project
## Create an executable wrapper to execute the project vendor/bin/codecept binary appending command arguments to it.
RUN echo '#! /usr/bin/env bash\n\
## Run the command arguments on the project Codeception binary.\n\
vendor/bin/codecept $@\n' \
> /usr/local/bin/codecept \
&& chmod +x /usr/local/bin/codecept
## By default run the wrapper when the container runs.
ENTRYPOINT ["/usr/local/bin/codecept"]
Note that, in the file, I'm also setting up Xdebug with the following configuration parameters:
zend_extension=xdebug.so - installing the extension is not enough; I need to tell PHP to use it explicitly.xdebug.idekey=PHPSTORM - the key by which the client request identifies itself. Sticking to the client-server architecture from before it's easy to understand how I could have many clients (machines running PHP code) running connecting to the server (my IDE, PHPStorm, running on my laptop): the server needs to know "who's calling"xdebug.remote_enable=1 - by default XDebug client and server would run on the same machine; it's not the case here as the machine running the code is the container (although virtual it's still a separate machine), while the server runs on my host machine (my laptop). The two machines communicate using an IP Address (sorta, see later) and not on localhost, so this is a remote connection.
I'm configuring the client and there is still one information missing: the server address, remote_host in XDebug terms.
Before any code example, it's important to understand remote_host means "what IP address or hostname should the client call to connect to the server?".
That IP address is **the IP address of my host machine, from the point of view of the container".
That IP address (hostname, really) is, in Docker for Mac and Windows, host.docker.internal.
Since I use Mac OS as my daily driver, it's reasonable, for me, to use host.docker.internal as a default, yet I can leave further scripting the possibility to set that value dynamically.
Below the docker-compose.yml file to accompany the update to the docker/wpbrowser/Dockerfile:
version: '3.2'
services:
wpbrowser:
# Instead of using a pre-built image, let's build it from the file.
build:
context: ./docker/wpbrowser
dockerfile: Dockerfile
args:
# By default use PHP 7.3 but allow overriding the version.
- BUILD_PHP_VERSION=${TEST_PHP_VERSION:-7.3}
volumes:
# Bind the project files into the /project directory.
- ./:/project
environment:
# As XDebug remote host use the host hostname on Docker for Mac or Windows, but allow overriding it.
# Build systems are usually Linux-based.
# Use default port, 9000, by default.
XDEBUG_CONFIG: "remote_host=${XDEBUG_REMOTE_HOST:-host.docker.internal} remote_port=${XDEBUG_REMOTE_PORT:-9000}"
# I use PHPStorm as IDE and this will make the handshake easier.
PHP_IDE_CONFIG: "serverName=wp.test"
I have, so far, configured only the XDebug client, it's time to configure the server: my IDE, PHPStorm.
Configuring PHPStorm to listen for XDebug connections
The debugging tool, my IDE in this example, is the server listening for XDebug connections from the client (the container) and "processing" them.
That processing is me scouring the code line by line or looking at variable values; it's slow and inefficient "processing", yet it's still a process.
When I've finished debugging, and allow the code to "move on", the client receives an "OK" response from the server and moves on. The cycle restarts for the next breakpoint, and so on.
From the values set in the section above I've configured PHPStorm to listen on port 9000 for XDebug connections:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/phpstorm-xdebug-config-1.png)
Also, set up the Servers accordingly:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/phpstorm-xdebug-config-2.png)
Running the tests with XDebug
Now that it's all set up, it's time to set a breakpoint in the code, run the tests, and see the code execution stop:
[
](https://theaveragedev.com/wp-content/uploads/2019/08/phpstorm-xdebug-set-breakpoint.png)
[
](https://theaveragedev.com/wp-content/uploads/2019/08/phpstorm-xdebug-execution-stopped.png)
Next
In my next post I will try to refine this configuration a bit further to make it more portable (e.g., easily usable on Linux) and more flexible: I've hard-coded XDebug usage in the code and having XDebug active all the times might not be the best choice in terms of speed.
