Skip to main content

Publish

This step in the tutorial demonstrates Testspace's integration with GitHub workflows for publishing test results. Testspace also works with many other popular CI systems. Refer to the Add to CI help article for details.

The example uses the Github repo created based on the sample in the Setup section of the tutorial.

Let's use the GitHub workflow

  • to manually invoked automation that will publish pre-canned test results

Run Automation​

GitHub Workflows supports many different types of events that will trigger a workflow, however, for this tutorial, we wanted to make things easy and interactive. For this reason, a workflow dispatch event is used versus a more common event such as push.

tip

Using the workflow dispatch event to manually execute the GitHub Workflow

To execute the GitHub workflow go to the Actions tab. Select the CI workflow under All workflows.

  • On the far right click the Run workflow button, use the Branch: main and click Run workflow
  • Watch the CI workflow execute using the GitHub UI
  • When the automation is complete, switch to your Testspace project
  • Explore Testspace, starting at the Current tab.

Published Results

Testspace automatically tracks the branch workflow. Go ahead and create a branch and watch Testspace track it with a matching space. Pull requests are also supported, along with merged and deleted branches.

Remember to execute the automation using the GitHub UI.

Files Used​

The following file content is used in this basic example for publishing automated testing results. Note that the test results are pre-canned (aka fake) and used for demonstration purposes only.

There are two files (ci.yml, results.xml) required for publishing results:

root
  └─ .github/workflows
              └─ ci.yml
  ..
  └── results.xml
  ..

Workflow File (ci.yml)​

The following content into the .github/workflows/ci.yml file.

name: CI
on:
  workflow_dispatch:
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
       - uses: actions/checkout@v3
       - uses: testspace-com/setup-testspace@v1
         with:
           domain: ${{ github.repository_owner }}
       - name: Push result to Testspace server
         run: |
           testspace results.xml
         if: always()
info

If using a private repo a Testspace token is required. Follow these additional steps.

  • The if: always() ensures test results are uploaded even in the event of a workflow failure.
  • Refer here for details on the setup-testspace action.

Results File (results.xml)​

For simplicity we are using pre-canned test results.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="example" tests="2" failures="0" time="2.0">
  <testsuite name="Hello Automation">
    <testcase classname="Hello Automation" name="Test Case One" time="1.00"/>
    <testcase classname="Hello Automation" name="Test Case Two" time="1.00">
      <failure type="Exception" message="Something unexpected happened"/>
    </testcase>
  </testsuite>
</testsuites>

Pro Tips​

The following tips section provides additional examples for common use-cases.

Using Folders​

Results can be organized into folders.

$ testspace "[myFolder]results*.xml"

sub-folders can also be used with a forward slash /.

$ testspace "[myFolder/mySubfolder]results.xml"

Using a Matrix​

When using a matrix it is recommended to use a folder to store the results specific to each matric entry.

name: CI
on: push
jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
    steps:
      ..
      - name: Publish Results to Testspace
        run: testspace "[ ${{ matrix.os }} ]results.xml"

Other details​

For more information on publishing features, refer to the Push Data section of the help documentation:

See Tools Support for language-specific information.

Recap​

This section covered the simplicity of publishing test results from automation based on GitHub workflows:

  • Single line command to publish results
  • Built-in branch workflow support for branching, pull requests, and forks
  • Additional useful information related to the health of the build can also be included, such as code coverage, custom metrics, etc.

Setup Action​

A Testspace setup action is provided:

Private Repo​

If the repo being used is private there are additional steps required:

  • Copy your Testspace Project push token
  • Create a secret, in your repo settings named TESTSPACE_TOKEN with value being your Testspace token.
  • Update the ci.yml file, adding the token input as shown below
- uses: testspace-com/setup-testspace@v1
  with:
    domain: ${{ github.repository_owner }}
    token:  ${{ secrets.TESTSPACE_TOKEN }}

Domain Name​

Note that the domain action parameter for setup-testspace is using a github context variable called github.repository_owner. This variable returns a string reflecting the name of the GitHub organization. When installing the Testspace app using the GitHub Marketplace the Testspace organization name matches the GitHub organization name. If the Testspace subdomain is changed the domain setup parameter needs to be updated as well.

Standalone​

If you are using GitHub Actions skip this section.

This section of the tutorial focuses on publishing using standalone projects.

A standalone project is required when running test automation - not directly connected to GitHub, GitLab, or Bitbucket repositories.

To begin login into your Testspace account.

New project​

You will now create a Testspace project to be used for this section of the tutorial. Select the New Project button(only availabe to admin accounts) at the top right of the page. You can select whether to make this project Public by the checkbox in the upper right corner, the default is private.

New Project

Provide a Project name and optional description and SUBMIT.

Note, if a list of repos is presented make sure to select the STANDALONE button on the left/bottom side.

New space​

Once a new Project has been created a space for publishing needs to be added as well. Select the New Space button to bring up the dialog shown below.

New Space

Provide a Space name and optional description and SUBMIT.

Install client​

As a prerequisite to pushing test content the Testspace client console utility is required to be installed on your host computer. Refer to the client installation instructions for Windows, Linux, and macOS computers.

Verify installation:

$ testspace --version

Push data​

For simplicity, create a file called results.xml with the following pre-canned test results:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="example" tests="2" failures="0" time="2.0">
  <testsuite name="Hello Automation">
    <testcase classname="Hello Automation" name="Test Case One" time="1.00"/>
    <testcase classname="Hello Automation" name="Test Case Two" time="1.00">
      <failure type="Exception" message="Something unexpected happened"/>
    </testcase>
  </testsuite>
</testsuites>

Using the newly created file go ahead and push data to the server using the following:

$ testspace results.xml YOUR_TOKEN:@YOUR_ORG.testspace.com/YOUR_PROJECT/YOUR_SPACE

After making the following changes:

  • Update YOUR_ORG to the name you picked when signing up to Testspace (e.g. 'myOrganization')
  • Update YOUR_PROJECT to the name used
  • Update YOUR_SPACE to the named used
  • Update YOUR_TOKEN to your Project push token or your User access-token

Config​

If appropriate you could do a one time config:

$ testspace config url YOUR_TOKEN:@YOUR_ORG.testspace.com/YOUR_PROJECT/YOUR_SPACE

Then pushing looks like:

$ testspace results.xml

For setting options see:

$ testspace --help config

Pro Tips​

For more information on publishing features, refer to the Push Data section of the help documentation:

Recap​

The section covered the setup steps and options for publishing test automation results using a standalone project:

  • Creating a Testspace project and one or more spaces
  • Installing the Testspace client
  • Option for configuring the Testspace client
  • Options for pushing data