Skip to main content

Testspace Client

The Testspace Client is a command-line tool used to push results to Testspace.

For installation information of the Testspace Client, please see download section below.

The Client executes on a host computer and communicates with Testspace via the Internet. It's a lightweight command-line tool with versions for Windows, Linux, and macOS. It's suitable for inclusion in automated build/test setups and can also be used interactively.

The Testspace Client command line supports five commands that specify the action the tool will perform. Each command has its own set of parameters.

$ testspace [--version] [--help] [<command>] [<args>] ​
CommandAction
configGet and set default global options.
pushPush existing result file(s) to a Space. (this is the default)

Any argument with NAME=VALUE syntax will be treated as an environment variable assignment available during the current execution. For example: testspace "mytests.xml" "my-space" NAME=VALUE

To prevent unexpected behavior due to the way the shell interprets the command line, any argument containing spaces and/or special shell character should be surrounded in double-quotes.

Config​

​The config command allows you to pre-configure all or part of the default Testspace URL used in subsequent testspace invocations. When running Testspace interactively, this simplifies the parameters you must enter.

​$ testspace config <option> [<value>]

If you run the command with no arguments, the current settings are displayed on the console.

Note: The configuration scope is the local user profile.

All base Testspace URLs have the following form:

credentials@domain-name/project-name/space-name

The URL decomposes into three segments delimited by the forward slash: domain (including credentials), project, and space.

With the config command, you can set defaults for any or all of these fields.

Tip: A common practice is to use the config command to set default values for some or all of the required URL segments. These defaults then don't have to be specified on the command line when you push - anything that is given for the URL parameter is appended to the current value previously set with config.

Note: If you are planning to work with more than one Space within a Project you can set a partial default URL and then specify the Space name on the command line when you push.

Note: If an argument contains one or more space characters, you must surround the argument in double quotes. This can be avoided by refraining from the use of the space character in Project and Space names.

Options​

The following options are supported:

project - sets/gets the project name segment of the URL; it is the name (non case sensitive) of an existing Project in your Organization

space - sets/gets the space name segment of the URL; it is the name (non case sensitive) of an existing Space in the Project

url - sets/gets the entire URL (all three fields) in a single operation. Notice, the current project (or space) configuration will be reset even if it is not specified in the url.

Note: In the context of well known CIs, the project and space segments of the URL could be omitted. The project and space will be automaticaliy determine based on the related repo and brnach names.

Credentials​

The specified login credentials (the part of the domain segment before @) must match an account with permission on the Project specified by the Testspace URL.

The credentials may be given either as a token1 or as user:pwd.

info

The credentials may be given either as a token1 or as user:pwd.

For example: mytoken:@myorg.testspace.com/project-name/space-name

Or, if using user/pwd: myuser:mypassword@myorg.testspace.com/project-name/space-name

Persisting your Credentials​

You may also persist your login credentials, which allows you to omit them in the URL.

Set TESTSPACE_TOKEN environment variable:

export TESTSPACE_TOKEN=mytoken

Or alternatively, in ~/.netrc file, simply add:

machine myorg.testspace.com
login mytoken
password

Notice the password field must be left blank. If using user/pwd:

machine myorg.testspace.com
login myuser
password mypassword

Note: This technique is insecure, as the password is stored in plain-text. Make sure you modify permissions to make the file readable only by yourself.

Proxy​

If you access the Internet via an HTTP proxy, you must set the environment variables HTTP_PROXY and HTTPS_PROXY to specify the name and port of the proxy server.

Symptoms of needing to specify a proxy is an error like:

Unable to proceed. Unable to access `My-Space` space in `My-Project` project at Testspace (myorg.testspace.com) with specified credentials (myuser).
Failed performing request: [7] Couldn't connect to server

Define the environment variables in format host:port, corresponding to your proxy server:

HTTP_PROXY=myproxy:8765
HTTPS_PROXY=myproxy:8765

would instruct the Testspace Client to use the proxy named myproxy on port 8765 to communicate via HTTP and HTTPS protocols.

If your proxy requires authentication use the format username:password@host:port where username and password are your credentials.

Examples​

Set entire URL as default

testspace config url "mycredentials@myorg.testspace.com/My-Project/My-Space"

Set partial URL as default; no space specified

testspace config url "mycredentials@myorg.testspace.com/My-Project"

View current config settings

testspace config

Note: To clear a setting use "". For example testspace config url project ""

Set project option; clear current setting for space

testspace config project "My-Project"
testspace config space ""

Push​

The push command uploads existing test results data in one of the supported formats to a Space.

$ testspace push [options] [<files>] [<url>]

Note that the push command is the default, thus is not required on the command line.

$ testspace [options] [<files>] [<url>]

Options​

OptionDescription
--repo argSpecifies the type of the repository to allow collection of commit details and generation of a code change report. Its value must be formatted in the following way:
type[:path]
Where type could be git, svn or p4 and path is an optional path to the local repository location, otherwise the current directory will be used. See Notes below.
Note: For git the optional path argument is equivalent to --git-dir git's option and can also be controlled by setting the GIT_DIR environment variable.
Note: For p4 there must be an active login ticket plus "P4PORT" and "P4CLIENT" (matching the specified optional path argument) settings configured in your environment.
--link argSpecifies an external service to link to, expressed in the following form:
[token@]type
Where type could be codecov or coveralls and token is an optional API access token.
--build-url argSpecifies the URL of the build that produced the content being pushed.
--build-status argSpecifies the status of the build that produced the content being pushed. Allowed values are success,failure, or error.
--message argSpecifies a textual description to associate with the pushed content.
--command [ -c ] argSpecifies a command to execute before anything else and propagate its exit code on completion.
--dryDry processing, just validate the input without uploading.
--verbosePrint detailed progress messages.

Files​

This specifies one or more files (space delimited) to be uploaded. Each entry can be full or relative to the current console directory ''file path'' (including wildcard pattern), formatted as follows:

[[folder]][+]/path/to/file[{metadata}]

where the optional prefix and suffix are:

OptionDescription
[folder]A result folder to place the content of the file in.
+When + prefix is present, instead of interpreting the test result data, the containing file is treated as opaque and is simply added as an annotation.
{metadata}File specific metadata that define special processing of the content:
- for Unit Test Formats must be the root file path to the test source code, base on file folder hierarchy the test results should be organized
- for Static Analysis "Test Log" Format must be set to lint
- when + prefix is present provides a textual description for the annotation
- for custom metrics must be a .csv file
- for log file must be exit code and/or textual description

Note: If a file argument is not a simple file-path or contains one or more space characters, you must surround it in double quotes. That would prevent unexpected behavior due to the way the shell expands wildcard patterns.

If multiple files are specified, the results of all files will be automatically aggregated into a single result set. Also a file containing a list of files to ''push'' can be specified.

@/path/to/content-list.txt

Where content-list.txt contains line separated list of files:

file1.xml
file2.xml

URL​

The form of the URL argument is as follows:

credentials@domain/project/space[?how][#[name][/path-to]]

By default, the root Space URL will use the one set with the ''config'' command, but you can optionally specify a URL on the command line that will override any existing config setting if present.

The other segments of the URL are described below.

how​

The optional ''how'' segment specifies the way results are to be pushed. Use this option when you want to aggregate several uploads into a single result set. If you invoke the Testspace Client more than once to push to a single result set, you use the ''how'' segment to specify the state that the result set should be in after the current invocation.

Recognized values are:

typedescription
fullCauses a new result set to be created and marked complete (this is the default).
startCreates a new result set and marks it incomplete.
addAdds more data to an existing incomplete result set and keeps it incomplete.
finishAdds more data to an existing incomplete result set and marks it complete.

When using the how argument, you must specify the identical result set name for each Testspace Client invocation.

Note: Use the how argument only when you invoke the Testspace Client multiple times and want to aggregate results from more than one publication.

name​

Use the optional name argument to specify a name for the result set created in Testspace. The name does not have to be unique to existing result sets.

A name is required when you specify anything but full in the how segment in order to match uploads from multiple invocations to a singe result set.

If not specified, the Testspace Client will supply the name Sequence_N, where N is a monotonically increasing integer.

path-to​

The optional path-to argument specifies a path under the Space root where the pushed results will be placed.

When uploading results, this feature is useful when you want to better organize a result set uploaded via multiple invocations of the Testspace Client; each upload can specify a different root folder. Any non-existent folders in the path are automatically created using the name given in the path.

Examples​

Note: Additional examples are shown in here.

Push local myResults.xml to a Space

testspace myResults.xml "mycredentials@myorg.testspace.com/My-Project/My-Space"

Push local myResults.xml to a Space; default Project Root URL has been set via config

testspace myResults.xml My-Space

Push multiple files to a single result set

testspace myResults1.xml myResults2.xml My-Space

Push multiple files to a single result set using wildcard

testspace myResults*.xml My-Space

Incrementally push files to specific folders in a single result set named my-resultset; default Project Root URL has been set via config

testspace myResults1.xml "My-Space?start#my-resultset/Folder1"
testspace myResults2.xml "My-Space?add#my-resultset/Folder2"
testspace "My-Space?finish#my-resultset/Folder3"

Download​

The Testspace Client is a command-line tool required to push results to Testspace. The client executes on a host computer and communicates with the Testspace web-application via the Internet. A prebuilt binary package is available for Linux, Windows and macOS hosts.

note

By downloading the Testspace Client your agree to the End User License Agreement.

64-bit glibc 2.17 or later

Download Testspace Client for Linux

Download Testspace Client for Linux ARM64 (aarch64)

Extract Archive​

Create a directory at $HOME/testspace and extract the tarball in there.

mkdir $HOME/testspace
tar -zxvf testspace-linux.tgz -C $HOME/testspace

Update PATH​

Finally, add $HOME/testspace to your PATH. If using the bash shell, add the following to ~/.bash_profile:

export PATH=$PATH:$HOME/testspace

Command Line Download and Extract​

curl -fsSL https://testspace-client.s3.amazonaws.com/testspace-linux.tgz | tar -zxvf- -C $HOME/testspace

Release Notes​

2.21 Release​

7/4/2024 v2.21.9799043624

  • Added support for macOS ARM64 (M series)
  • Fixed handling of JUnit relative attachment paths

2.20 Release​

3/8/2024 v2.20.8207774381

  • Support description of JUnit test-cases via 'property'
  • Fixed handling of CircleCI URL
  • Fixed handling of Bitbucket URL
  • Fixed parsing of JaCoCo XML

2.19 Release​

10/18/2023 v2.19.6566258886

  • Adjusted Semaphore CI integration

10/11/2023 v2.19.6515900629

  • Added support for Buildkite CI
  • In CI context, file-paths are now reported relative to the repo root
  • Static analysis details per file are now CSV formatted
  • Refined Rubocop output parsing

2.18 Release​

7/16/2023 v2.18.5513711687

  • Fixed handling of '**' glob expansion

2.17 Release​

4/6/2023 v2.17.4620860708

  • Adjusted handling of 'jest-junit' content

2.16 Release​

3/8/2023 v2.16.4368703307

  • Fixed reporting of JUnit test-suite attachments
  • Fixed source organization on Windows
  • Adjusted handling of 'mocha-junit-reporter' content

2.15 Release​

1/18/2023 v2.15.8141

  • Added support for Linux ARM64 (aarch64)
  • Fixed reporting of JUnit test-suite console output
  • Ignore system config in CI context

2.14 Release​

12/19/2022 v2.14.7904

  • Corrected explicit GitHub checkout of another branch

2.13 Release​

7/15/2022 v2.13.7358

  • Do CI auto-config only on "bare" domain
  • Fixed parsing of TRX inner-results
  • Allow gathering code changes from a remote P4 location

2.12 Release​

4/27/2022 v2.12.6954

  • Windows version is 64-bit now
  • Linux version requires glibc 2.17
  • Added support for custom "build-times" association with a result
  • Refined parsing of build logs to support unique Pyright format

2.11 Release​

12/15/2021 v2.11.6247

  • Fixed error on pushing GitHub/Azure tag build results

2.10 Release​

10/11/2021 v2.10.5840

  • In CI context, store configuration locally

2.9 Release​

6/30/2021 v2.9.5209

  • Fixed error on pushing to a Space with @ in the name
  • Refined JUnit failure message extraction
  • Dropped support for outdated CI variants (e.g. Shippable, CircleCI-1.0)

2.8 Release​

6/11/2021 v2.8.5061

  • Added support for aggregation of multiple GitHub workflows

2.7 Release​

5/13/2021 v2.7.4873

  • Fixed crash on parsing NUnit-v2 XML
  • Refined parsing of build logs

2.6 Release​

3/29/2021 v2.6.4630

  • Corrected Jenkins PR-builds handling
  • Fixed incorrect handling of projects with with ':' (colon) in their name
  • Adjusted handling of JUnit XML failure message

2.5 Release​

12/15/2020 v2.5.4061

  • Added support for custom "build-status" association with a result
  • Fixed incorrect handling of system locale
  • Fixed incorrect file truncation on Posix systems
  • Updated to support GoogleTest v1.10

2.4 Release​

6/30/2020 v2.4.3002

  • Added support for Codefresh CI
  • Adjusted GitHub Workflows integration
  • Adjusted GitLab Pipelines integration
  • Updated to exit with "non-0" code on failure in CI context
  • Refined parsing of build logs

2.3 Release​

2/24/2020 v2.3.2185

  • Added support for GitHub Workflows
  • Added support for post-report metrics assignment
  • Adjusted Drone integration
  • Adjusted Gauge "failure" type
  • Refined URL parsing
  • Refined parsing of build logs
  • Fixed incorrect handling of CTS "status" and "timing"
  • Fixed incorrect handling of 0/1 "link_file" XML attribute

2.2 Release​

8/26/2019 v2.2.1039

  • Separated handling of "failed" and "errored" tests
  • Adjusted Azure Pipelines integration

2.1 Release​

5/31/2019 v2.1.862

  • Updated to report JUnit "error" as "in progress"
  • Refined processing of JUnit results from pytest and Gauge
  • Adjusted Codeship integration
  • Fixed incorrect handling of CTest "stdout"

2.0 Release​

3/8/2019 v2.0.587

  • The Mac client binary now requires macOS 10.12 or later
  • Added support for Azure Pipelines
  • Added support for setting credentials via ''$TESTSPACE_TOKEN'' environment variable
  • Adjusted Travis CI integration
  • Adjusted Circle CI integration
  • Adjusted AppVeyor CI integration
  • Adjusted Bitbucket Pipelines integration
  • Adjusted GitLab CI integration
  • Adjusted Jenkins integration
  • Refined parsing of build logs
  • Refined P4 code change gathering
  • Fixed incorrect "folder" sorting for RSpec describe using symbols
  • Fixed failure on processing VS2017 binary coverage

1.15 Release​

8/13/2018 v1.15.600

  • Adjusted Circle CI integration
  • Adjusted GitLab CI integration
  • Fixed incorrect "folder" sorting for Java Surefire tests with "reportNameSuffix" attribute
  • Fixed incorrect "folder" sorting for PHPUnit results with no "classname" attribute
  • Fixed incorrect handling of of double '/' in folder path

1.14 Release​

6/08/2018 v1.14.591

  • Added support for linking to Codecov and Coveralls
  • Refined parsing of pytest "skipped" tests
  • Adjusted Travis CI integration
  • Fixed incorrect "folder" sorting for Java "public static class" tests
  • Fixed incorrect "folder" sorting for test classes with "/" (slash) in the name
  • Fixed incorrect handling of non-lowercase test case status

1.13 Release​

3/28/2018 v1.13.578

  • Fixed crash on parsing multi-level PHPUnit results
  • Optimized result parsing

1.12 Release​

2/06/2018 v1.12.567

  • Fixed incorrect "folder" sorting for some edge cases
  • Optimized "folder" sorting based on source location
  • Improved error handling
  • Added support for Bullseye code coverage grouping and filtering

1.11 Release​

11/06/2017 v1.11.540

  • Changed to always exit with "0" code in CI context
  • Added support for "command" execution before result processing
  • Added support for Codeship Pro
  • Added support for Bitbucket Pipelines
  • Added support for GitLab CI
  • Added support for CTest results
  • Fixed incorrect "folder" sorting of BDD tests with trailing spaces in the name
  • Fixed incorrect "folder" sorting of tests defined in PHP "namespace" or Ruby "module"
  • Refined P4 code change gathering
  • Refined processing of JUnit results from Ember.js, Karma, WDIO, Lab, and pytest

1.10 Release​

7/25/2017 v1.10.521

  • Removed the auxiliary "testrun" executable from the Client package
  • Added support for custom "build-url" and "message" association with a result
  • Added support for setting "pass" on Custom Suite
  • Fixed incorrect reporting of Bullseye coverage format version "4" and older
  • Fixed incorrect "folder" sorting of BDD tests with trailing spaces in the name
  • Fixed incorrect handling of "=" (equal) in file arguments
  • Fixed P4 code change gathering in Jenkins
  • Fixed incorrect handling of JSON with Unicode characters
  • Refined Visual Studio "log" lint patterns

1.9 Release​

5/10/2017 v1.9.501

  • Added support for anonymous results uploading to "public" Projects on Travis, AppVeyor, Circle, and Shippable CI
  • Fixed incorrect Travis "pro" build URL

1.8 Release​

4/24/2017 v1.8.490

  • Implemented "auto" configuration in CI context
  • Added support for JUnit and TRX attachments
  • Added support for Android CTS results
  • Added support for Custom Suite "failure" reporting
  • Changed code change collection to be optional
  • Lifted off Space name restrictions
  • Improved test source lookup
  • Improved Perforce code change reporting
  • Fixed incorrect XML transcoding

1.7 Release​

1/14/2017 v1.7.448

  • Implemented Git code change reporting
  • Implemented SVN code change reporting
  • Implemented Perforce code change reporting
  • Added support for Android lint results
  • Deprecated "publish" command in favor of "push"
  • Deprecated "issues" metadata for TXT static analysis in favor of "lint"
  • Fixed handling of "--verbose" option

1.6 Release​

12/08/2016 v1.6.437

  • Added support for associating "git" commit-id with results
  • Fixed static analysis issue's message truncated
  • Fixed empty Klocwork report not recognized

1.5 Release​

10/29/2016 v1.5.431

  • Added support for "**" directory traversal
  • Added support for "named" custom metrics
  • Optimized XML results loading
  • Optimized test source mapping
  • Refined "build log" analysis parsing
  • Fixed creation of "empty" incremental result set
  • Fixed handling of very long results listing

1.4 Release​

09/16/2016 v1.4.414

  • Made "publish" default command and deprecated "run"/"export"/"import"
  • Added support for "custom" report/metrics
  • Fixed handling of "empty" Static Analysis results
  • Fixed handling of "fatal error" issues in Static Analysis results
  • Fixed sorting of Mocha "xunit" results

1.3 Release​

09/03/2016 v1.3.407

  • Added support for static analysis results
  • Changed default code coverage suite name to "Code Coverage"
  • Optimized loading of huge Visual Studio code coverage results
  • Optimized loading of huge DotCover code coverage results

1.2 Release​

06/26/2016 v1.2.391

  • Added support for xUnit.net results
  • Added support for NUnit v2 results
  • Fixed handling of "chutzpah" JUnit results
  • Fixed organizing C# test code
  • Fixed loading XML with invalid DTD

1.1 Release​

05/28/2016 v1.1.388

  • Added support for many popular test frameworks and tools - Visual Studio Unit Test, NUnit, JaCoCo, OpenCover, dotCover...
  • Added support for publishing via an "options" file
  • Fixed handling of "flat" JUnit results
  • Fixed failure running in a read-only directory

First Official Release​

02/16/2016 v1.0.371

  1. You can use User Access token or use a specific Project Push token.↩