Automation
Testspace support for serverless automation
using GitHub Actions and AWS Lambdas enables testing solutions that can leverage a hybrid of automation and manual verification.
See suite fixture section for details.
Implement more effective tests using a hybrid of automation and human observations.
For some of the values of integrating automation with manual execution:
Reduction in Manual Execution Time. Leverage automation for tedious and redundant setup/teardown requirements vs human execution.
More Effective Test Coverage. By leveraging a combination of automated fixturing and the power of human observations, more problems often can be discovered. Fully automating everything can have significant limits in verification and be time-consuming to implement and maintain.
Eliminated IT Setup for Human Tester. All testing, including automation, is executed in the context of a Browser. Automation is executed leveraging callable serverless functions and abstracted from the human tester. There are no dependencies on the computer being used by the Tester.
Test specifications and automation maintained together in a repo. All of the test artifacts are managed as code, leveraging the same development process as the application team.
Input
A Fixture's Input
is defined within a calling specification's header metadata block. Refer to the suite fixture section for details.
---
testspace:
before:
name: {name}
input:
p1: 1st parameter
p2: 2nd parameter
p3:
- one
- two
- three
p4:
this: one
that: other one
..
p10: "@file.json"
---
There are up to 10 top-level parameters supported. The value
of a parameter is set using one of the following types:
- string/number
- embedded YAML
- File
Each parameter value defined as embedded YAML is serialized
as a JSON-string and is required to be deserialized
by the fixture handler.
Parameter values defined as embedded YAML are required to be
deserialized
For parameter values defined with a file reference (i.e. @file.json
) are encoded as base64 and required to be unencoded
.
Example
The following JavaScript handler example is passed the input
parameters as a JSON String.
function handler(input, ..) {
var input = JSON.parse(input);
console.log("Input: ", input);
console.log("p1:", input.p1);
console.log("p2:", input.p2);
var p3 = JSON.parse(input.p3);
console.log("p3", p3);
var p4 = JSON.parse(input.p4);
console.log("p4", p4);
..
var p10 = Buffer.from(input.p10, 'base64').toString();
console.log("p10", JSON.parse(p10));
}
GitHub Workflows
GitHub has built-in CI/CD support. Testspace enables leveraging of this functionality in the context of executing a manual suite.
Workflow
A GitHub workflow is used to specify the runtime environment where the serverless automation handler is going to be executed. As Testspace uses a Repository Dispatch event to trigger the workflow, it has to be configured to start on an external event, i.e. respository_dispatch
event:
on:
repository_dispatch:
types:[workflow-name]
By default, all workflows within the repo will be triggered when configured for the respository_dispatch
event. The types
can be used for further filtering based on the identifier's name
as defined in the corresponding spec's fixture header block:
spec.md:
---
testspace:
before:
name: github::workflow-name
---
For the workflow file - myfixture.yml:
on:
repository_dispatch:
types: [myfixture]
Testspace sets the
respository_dispatch
event'stype
(aka event action) as the identifier's name defined in the Spec when executing a Suite's fixture. The workflow file name isrequired
to match.
A step
in the workflow specifies the way the serverless automation handler should be executed. For example, running a script (e.g. handler.js
):
jobs:
my-job:
runs-on: ubuntu-latest
steps:
- name: Call handler
run: node handler.js "$PAYLOAD"
timeout-minutes: 5
env:
PAYLOAD: ${{ toJson(github.event.client_payload) }}
The github.event
variable provides the Suite's fixture context being called:
github.event.action
represents the spec's "workflow-name". Notice, it is the same as therespository_dispatch
event'stypes
that the workflow is filtered on.github.event.client_payload
is the "input". Notice, it needs to be serialized as JSON string before being passed to a script handler.- the fixture's input is constrained to up-to 10 top-level input parameters.
Note, on a workflow failure do not re-run the failed job using the GitHub Actions UI. Testspace has no way of tracking that execution.
Handler
The serverless automation handler could be implemted in a language of your choice. Here is a JavaScript example (e.g. handler.js
, referenced above):
function handler(payload) {
var input = JSON.parse(payload);
console.log("Input: ", input.a, input.b);
}
handler(process.argv[2]);
AWS Lambdas
AWS has built-in serverless function execution support using AWS Lambdas. This functionality fits very well with Testspace automated fixturing. To enable Testspace invoking Lambdas, access keys are required to be set up for the account.
Handler
Lambda handlers can be written in any language supported by AWS. The following is a Lambda function for Node.js:
exports.handler = async (event, context) => {
console.log("Input:", event);
// TO DO
const response = {
statusCode: 200,
body: JSON.stringify('Hello from Lambda!'),
};
return response;
};
The ASW Lambda runtime automatically converts the JSON Object to the corresponding language object type.
Logging
Testspace captures the input parameters sent to the Lambda and the log stream generated by the function as an annotation for the Suite.