See spec fixture section for details.
Implement more effective tests using a hybrid of automation and human observations.
For some of the values of integrating automation with manual execution:
Reduction in Manual Execution Time. Leverage automation for tedious and redundant setup/teardown requirements vs human execution.
More Effective Test Coverage. By leveraging a combination of automated fixturing and the power of human observations, more problems often can be discovered. Fully automating everything can have significant limits in verification and be time-consuming to implement and maintain.
Eliminated IT Setup for Human Tester. All testing, including automation, is executed in the context of a Browser. Automation is executed leveraging callable serverless functions and abstracted from the human tester. There are no dependencies on the computer being used by the Tester.
Test specifications and automation maintained together in a repo. All of the test artifacts are managed as code, leveraging the same development process as the application team.
Input is defined within a calling specification's header metadata block. Refer to the spec fixture section for details.
There are up to 10 top-level parameters supported. The
value of a parameter is set using one of the following types:
- embedded YAML
Each parameter value defined as embedded YAML is
serialized as a JSON-string and is required to be
deserialized by the fixture handler.
Parameter values defined as embedded YAML are required to be
For parameter values defined with a file reference (i.e.
@file.json) are encoded as base64 and required to be
input parameters as a JSON String.
GitHub has built-in CI/CD support. Testspace enables leveraging of this functionality in the context of executing a manual spec.
A GitHub workflow is used to specify the runtime environment where the serverless automation handler is going to be executed. As Testspace uses a Repository Dispatch event to trigger the workflow, it has to be configured to start on an external event, i.e.
types is required to match the name of the workflow file name, excluding "yml" extension. The same
name is required to be defined in the corresponding spec's fixture header block:
For the workflow file - myfixture.yml:
Testspace sets the
type(aka event action) as the identifier's name defined in the Spec when executing a fixture. The workflow file name is
step in the workflow specifies the way the serverless automation handler should be executed. For example, running a script (e.g.
github.event.actionrepresents the spec's "workflow-name". Notice, it is the same as the
typesthat the workflow is filtered on.
github.event.client_payloadis the "input". Notice, it needs to be serialized as JSON string before being passed to a script handler.
- the fixture's input is constrained to up-to 10 top-level input parameters.
Note, on a workflow failure do not re-run the failed job using the GitHub Actions UI. Testspace has no way of tracking that execution.
handler.js, referenced above):
AWS has built-in serverless function execution support using AWS Lambdas. This functionality fits very well with Testspace automated fixturing. To enable Testspace invoking Lambdas, access keys are required to be set up for the account.
Lambda handlers can be written in any language supported by AWS. The following is a Lambda function for Node.js:
The ASW Lambda runtime automatically converts the JSON Object to the corresponding language object type.
Testspace captures the input parameters sent to the Lambda and the log stream generated by the function as an annotation for the corresponding Result Suite.