QA Thing - 01

December 17, 2016 4-minute read

Scratching a new itch of mine.

The problem

I’m a developer and most of my work consists in writing code that’s supposed to fix bugs, issues, improve functionalities or implement features.
While my passion for test-driven development shows in all my posts (and talks, and meetups and chats…) I think the human-driven Quality Assurance (QA) process still remains fundamental.
Tests will cover code but those are written by a human with limited imagination and experience and are, in the most basic way, small AIs acting on pre-programmed patterns.
As any AI that’s not “learning” there are only so many cases a test will cover and humans keep, and will keep, “breaking stuff” well beyond the capability of a programmed machine to do so.
Now that I am at peace with the idea that “QA people” is a fundamental part of the “releasing stuff” process I’d like to solve a problem that emerged more than once working in a remote and distributed team: being on the same page.
Sometimes “being on the same page”, or setting pre-conditions for a test that has to be run by a human, might be trivial like

Create a post with a title that contains accented letters.

but things could be way more complex and, sometimes, beyond what the WordPress UI allows to do

Start with a scheduled post that has not been published at the scheduled time.

Recreating this is, simply put, impossible using WordPress UI only; latter instructions could be updated to read something like this:

Create a post and publish it; take note of the post ID. Now open the WordPress installation database with some SQL UI tool and find the row for the post ID in the posts table: set the post_status to future and both post_date and post_date_gmt to 2 days ago.

This kind of approach will not scale and lends itself to mistakes.

A solution

I will attempt to tackle the issue with a WordPress plugin aptly named “QA Thing” The idea is to allow developers to push different configurations for different issues in the plugin development repositories and have the “QA Thing” pick them up and allow easy and ignorant setup of the test pre-conditions.
As an example I could be developing a plugin called “T-wp-eet” that will sync a Twitter account with a WordPress site; the sync would happen on a cron job and, in some hedge cases where a connection is lost or something breaks, some of those tweets would not get synced.
Needing to test my disaster recovery routine I will tell QA

Activate the QA Thing and apply the configuration for issue 33445

In the plugin folder I would add a qa/qa-config.json file:

{
   "issues" : {
        "33445" : {
            "title" : "Issue 33445 - lost tweets",
            "target" : "qa/scripts/33445.php"
    }
   }
}

The fields are pretty self-explanatory except for “target”: this is the path, relative to the plugin root folder, where I’ve created a file that the “QA Thing” will run to set up the configuration; the file will be executed in the context of a WordPress AJAX request at wp_loaded time and, in this specific plugin and issue case, it might be this one:

// file qa/scripts/33445.php

$main = new t_wp_eet_Main;
$sync = $main->get_sync_engine();

qa_info('Syncing test account');

$start = strtotime( '-1 week' );
$end = time();
$ids = $sync->sync_with_account('lucatume', $start, $end );

if(empty($ids)){
    qa_error('There were no tweets to sync... something went wrong.');
    return;
}

if(count($ids) < 5){
    qa_error('There are less then 5 tweets to sync, not enough!');
    foreach($ids as $id){
        wp_delete_post($id, true);
    }
    return;
}

qa_info('Synced ' . count($ids) . ' tweets.');
qa_info('Removing 2 tweets');

$removed = array($ids[2], $ids[3]);

qa_info('Removing tweets: "'. get_post($ids[2])->post_title . '" and "' . get_post($ids[3])->post_title . '"');

foreach($removed as $id){
    wp_delete_post($id, true);
}

qa_info('Removed two tweets.')
qa_done('Done! Now run the recovery routine: can you see the deleted tweets?');

The qa_info, qa_error and qa_done functions are functions supplied by the plugin that will allow pretty output for the configuration process.
The “QA Thing” plugin will read the configuration and allow the QA person to apply it

[caption id=“attachment_3197” align=“aligncenter” width=“1436”] Applying a configuration in QA thing[/caption]

While this is still in the draft status the idea would solve an issue I’ve personally had more than once: being able to set up complex test pre conditions using the best language for the situation: code itself.

I will post about my progress and set up for the plugin and push a first version to GitHub.

The problem

A solution

Next