Skip to main content

Automated Fixturing

During this tutorial you have:

  • manually invoked a GitHub workflow to publish pre-canned results
  • implemented and executed new test specs from scratch
  • learned how to generate and manage GitHub issues in conjunction with a Project Board
  • formally defined a test session consisting of a set of specs for testing a build
  • leveraged Exploratory Testing for non-scripted verification
  • used variables and includes when implementing test specs
  • created a branch when developing new test specs

This section of the tutorial will be demonstrating the power of automation, integrated into a manual test, using a Spec Fixture based on a Github Action.

This example highlights the built-in automated fixturing for "manual" tests.

Setup#

To run a Github Action from Testspace a Github token is required that has "repo" scope access. This one time only step is required to be added to your Testspace account services.

Add GitHub token

warning

To run a GitHub automation in Testspace a one time only configuration needs to be done by your Testspace account owner. For details see this on how to enable automation.

Files Used#

The following files and their content is used in this section of the tutorial

  • Spec file - use.setup.fixture.md
  • GitHub workflow yaml file - fixture.yml
  • JavaScript file - handler.js

The repo file structure:

root
โ””โ”€ .github/workflows
โ””โ”€ fixture.yml
โ””โ”€ handler.js
โ””โ”€ specs
โ””โ”€ use.setup.fixture.md

Reference - https://github.com/testspace-com/testspace.getting-started

Spec#

To enable manual automation a front matter block is used at the beginning of the spec. The header section below defines a before fixture, which requires successful execution before test cases can be manually executed.

A fixture is used to execute serverless automation, which sets the state of the spec before manual test case execution begins.

The specs/use.setup.fixture.md file contains the following content:

---
testspace:
before:
name: github::fixture
description: example input
input:
a: one
b: two
---
# Use Setup Fixture
This is a manual test.
## Test Case One
Some description here.
* check this
* check that
## Test Case Two
Some description here.
* check this
* check that
important

The name field in the Spec is referencing the fixture.yml workflow file contained in the GitHub repo. The names are required to match exactly, excluding the "yml" extension.

Notes:

  • input parameters are optional
  • input can also contain files (i.e. json, binary, etc.)

Workflow#

In this example, we are using a GitHub Workflow to run handler.js, which implements the required setup.

The .github/workflows/fixture.yml file contains the following content:

name: Fixture
on:
repository_dispatch:
types: [fixture]
jobs:
fixture:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
with:
node-version: '12'
- name: Call Script implementing setup fixture
run: node ./.github/workflows/handler.js "$PAYLOAD"
env:
PAYLOAD: ${{ toJson(github.event.client_payload) }}
important

The types: [name] has to match the workflow file name, excluding the extension. (i.e. "fixture").

Note that the client_payload contains all of the input from the test spec.

Handler#

In our example, a generic handler will be implemented in JavaScript, but the concept applies to all languages supported by GitHub Actions.

The generic handler being used simply logs the input parameters.

The .github/workflows/handler.js file contains the following content:

function handler(payload) {
var input = JSON.parse(payload);
console.log("Input: ", input.a, input.b);
}
handler(process.argv[2]);

Notes:

  • The payload is converted to a JavaScript object
  • The payload is passed on the command line as the 2nd parameter

Execute#

The current test specs listing reflects the status of previous sessions and the new Use Setup Fixture spec (defined in specs/use.setup.fixture.md spec file).

To execute click on the Use Setup Fixture spec.

Click on the START button to trigger the automation. While running a blue โ—‹ spinning icon will appear, which should turn into a green โˆš icon to reflect success.

At this point, all test cases can be executed. Once the selected cases have been executed, the session can be completed and any teardown fixures would execute as well.

Recap#

Testspace supports integrating automation with manual testing. The following are some key benefits:

  • Reduce manual execution time. Fixturing can leverage automation for tedious and redundant setup/teardown requirements versus human execution.
  • Deploy "hybrid testing" leveraging automation and human observations.
  • Minimize IT setup for Human testers. All testing, including automation, is executed in the context of a web browser.
  • Maintain test specs and automation in the same repo.