My \"how to name stuff when testing\" flow

April 5, 2017

Naming things is hard.

How should I call this suite? And where should I put this test?

When working with Codeception and wp-browser the concept or "suite" is recurring.
A suite is a group of test cases sharing a common methodology of testing and requirements; as an example all the tests exercising the UI of a WordPress site, plugin or theme, requiring the use of modules like WPBrowser or WPWebDriver, and asserting changes and expected behaviours on that UI (or API) only could be grouped in the "acceptance" suite.
On the other hand all the tests aimed at single classes using nothing but what methods PhpUnit provides could be grouped in the "unit" suite.
ometimes the borders between those testing methodologies and requirements are not clear and naming the suites becomes a challenge for clarity and communication.
After all "naming things" is one of the two difficult things in programming.

Testing is 80% jargon

When I got into testing my biggest challenge was understanding the jargon.
"Unit testing", "integration testing", "functional testing" and the like.
To approach the problem of definitions I'm making an example in the WordPress context:

<?php
/*
Plugin Name: REST Calculator
Description: Add numbers using WP REST API!
Version: 0.1.0
Author: Luca Tumedei
Author URI: http://theaveragedev.local
*/

add_action( 'rest_api_init', function () {register_rest_route( 'calc', 'add/(?P<o1>\d+)/(?P<o2>\d+)', [
        'methods'  => 'GET',
        'callback' => [ new Calculator, 'process' ]
    ] );
} );

class Calculator {
    function process( WP_REST_Request $req ) {
        try {
            $operand_1 = new Operand( $req->get_param( 'o1' ) );
            $operand_2 = new Operand( $req->get_param( 'o2' ) );
        } catch ( InvalidArgumentException $e ) {
            $response = new WP_REST_Response( 'Bad operands' );
            $response->set_status( 400 );

            return $response;
        }

        $operation = new Addition( $operand_1, $operand_2 );
        $value     = $operation->get_value();

        set_transient( 'last_operation', $operation . ' with result ' . $value );

        $response = new WP_REST_Response( $value );
        $response->set_status( 200 );

        return $response;
    }
}

class Operand {
    private $value;

    function __construct( $value ) {
        if ( ! filter_var( $value, FILTER_VALIDATE_INT ) ) {
            throw new InvalidArgumentException( 'Not an int' );
        }

        $this->value = (int) $value;
    }

    function get_value() {
        return $this->value;
    }
}

class Addition {
    private $o1;
    private $o2;

    function __construct( Operand $o1, Operand $o2 ) {
        $this->o1 = $o1;
        $this->o2 = $o2;
    }

    function __toString() {
        $v1     = $this->o1->get_value();
        $v2     = $this->o2->get_value();
        $line_1 = 'Add: ' . $v1 . '+' . $v2 . '=' . $this->get_value();
        $line_2 = trailingslashit( "Route: /wp-json/calc/add/{$v1}/{$v2}" );

        return $line_1 . "\n" . $line_2;
    }

    function get_value() {
        return $this->o1->get_value() + $this->o2->get_value();
    }
}

While not exceptional in its functionalities the plugin is fully working as intended.

Testing levels explained, TL;DR

This is my take on it and to use the example plugin above as a reference:

to make sure visiting /wp-json/calc/add/2/3 returns 5 I'd use an acceptance test
to make sure visiting /wp-json/calc/add/2/3 would return a code 200 and save the last operation result in a database I'd use a functional test
to make sure calling the Calculator class with different WP_REST_Request objects yields the expected result I'd use and integration test
to make sure the Addition class is behaving as intended I'd use a WordPress unit test
to make sure the Operand class is behaving as intended I'd use a unit test

The unit, wp-unit and integration flowchart

How did I get there? While I can quote different sources to define what is a "unit" test, what is an "integration" test I try to stick to a simple mental flowchart to name my suites and organize my WordPress test cases:

Are you testing the whole application via its entry points? If "yes" go to 2 else go to 5.
Are you testing the whole application as a user with a black-box approach? If "yes" go to 3, else go to 4.
This is an acceptance test: create the test in the acceptance suite using wpcept generate:cept acceptance <class>.
This is a functional test: create the test in the functional suite using wpcept generate:cest functional <class>.
Are you testing a single class? If "yes" go to 7 else go to 6.
This is an integration test: create the test in the integration suite using wpcept generate:wpunit integration <class>.
Does your class depends on WordPress globals, constants or functions? If "yes" go to 8 else go to 11.
Could you find yourself in need to check if your class called a function? If "yes" go to 9, if no go to 10.
Write an adapter then go to unit.
This is a WordPress unit test: create the test in the wpunit suite using wpcept generate:wpunit wpunit <class>.
This is a unit test: create the test in the unit suite using using codecept generate:test unit <class>.

I really use that and fits 80% of the times.

I will write at least one test for each type above and see what modules could be used and how.

How should I call this suite? And where should I put this test?

Testing is 80% jargon

Testing levels explained, TL;DR

The unit, wp-unit and integration flowchart

Next