User Tools



Results

The most recent and historical test results are available on the Space's Results tab. In this view results are listed in date order with the newest results listed first. The Results View shows result summary data; to view pass/fail trends over time, visit the Space's Metrics tab.

Space's historical results

Descriptions of highlighted areas of the Results View:

  1. Organization View link
  2. Project View link
  3. Current Space Schema link
  4. Health indicator
  5. Hamburger menu for additional operations on corresponding result set
  6. Result Set name and link to results
  7. Commit id link and commit message (if available)
  8. CI system link (if available)
  9. Attached Note indicator - If one or more Notes is attached to this result set, an icon and count of Notes is shown here.
  10. Failures Column [ new, tracked, resolved ] totals for each result set

Results View Columns

Each result set is represented by a row that's divided into the following columns:

Name

The result set name is shown here as a clickable link.

To the left of each Result Set link (Label 6) is an icon and a number in square brackets. The number shows the Testspace-assigned upload sequence of the result set. This number is used to identify the unique result set.

The icon indicates the state of the result set as follows:

Icon
complete and current most current results, result set is complete
non-current, complete non-current results, result set is complete
incomplete partial (incomplete) results

Health

Healthy Healthy 0 nonexempt test failures and with all metric criteria met
Unhealthy Unhealthy 1 or more nonexempt test failures, or unmet metric criteria
Invalid Invalid See How-to: Manage Health Status for details about an invalid result.

Tests

The text here shows the number of passing and failing Test Suites and Test Cases. The numbers are shown in brackets and color coded:

  • Green → Pass Count
  • Red → Fail Count
  • Gray → No Data

Failures

A count of failed test cases in last five result sets [new, tracked, resolved]

  • new - A unique regression that was not previously in the tracked state.
  • tracked - A case that is not a new regression but has had at least one failure in the last 5 results
  • resolved - A case that has passed or has been missing from the past five successive results and is now being removed from the tracked state.

Duration

The duration of the entire test run represented by the result set is shown here if available.1)

Published

This column shows the date that the result set was published along with the username of the uploader.

If there are one or more comments on a Note created from this result set, the following icon is shown with the count of comments.

You can click on this icon to navigate to the referenced Note.


Hamburger Menu Items

At the far right end of each result row is a Hamburger Menu which becomes visible when you hover the mouse cursor over this area.

The menu offers the following options:

Comment Creates a Note that's pre-populated with high-level details of the result set. A comment is added to the note and you are dropped into the editor for this comment.

If the corresponding Note already exists, a new comment is added to it and you are dropped into the editor for the newly-added comment.
Map Displays a tree of the results.

This allows you to view the entire result hierarchy and quickly navigate to any node in the result set.
Rename Displays a dialog allowing you to rename incomplete result sets.

This changes the name shown in the Name column.
Add Allows you to manually upload additional results that are to be added to the result set.

(Active only for incomplete result sets.)
Complete Completes an inactive result set without adding any additional results.

(Active only for incomplete result sets.)
Export Downloads the result set (in Testspace XML format) to the host computer.
Delete Deletes the result set.

Results Details

To navigate into the result set details, click on the name link of the result set (Label 6). Results details are shown one Folder level at a time in the Results View, beginning with the root level.

As you navigate, at each level you will see the Folders and Suites that exist at that level.

Descriptions of the highlighted areas:

  1. Failure list button - Creates a flat list of suites that have at least one test case failure_
  2. Breadcrumb links - Since we are at the Root level of the results, only the Result Set name is visible
  3. Map button - Bring up the Map using this button. The current level is highlighted in the Map display
  4. Folder Name link - Clicking on a Folder name link will take you into its level in the hierarchy

Each Folder and Suite is displayed with columns that show roll up totals for their child items.2). Roll-up totals of the following are shown for each:

  • Suites - number of passing and failing Suites
  • Test Cases - number of passing and failing Test Cases
  • Failures - count of failed test cases in last five result sets [new, tracked, resolved]
  • Duration - total time (if available)

In the view below, we have clicked on the Tests Folder link in the previous view.

Descriptions of the highlighted areas:

  1. Breadcrumb links - Now shows current UnitTest level. Root link is clickable.
  2. Suite Name link - Clicking on a Suite name link will pop up a Suite window that shows details of the Suite's child Test Cases
  3. Roll-Up values are shown as before - Note that totals for the current level are summed at the bottom of the table

Suite Report

In the view below, we have clicked on the FormateTimeInMillisAsSecondsTest Suite link.

Descriptions of the highlighted areas:

  1. Parent Suite name - Parent Suite name is shown here
  2. Child Test Cases - Each child Test Case is listed along with its details
  3. Failure frequency - number of failures in the last five results if available. Clicking the number brings up a dialog with pass/fail information of the results. Historical information is only available if there were additional failures in the last five result sets. This allows for the determination of a whether a failure is new, recurring or other type.
  4. Roll-Up values - Totals for the Suite are summed at the bottom of the dialog


The Suite Report is displayed with columns that show roll up totals for their child items. Roll-up totals of the following are shown for each:

  • Status - number of passing and failing Suites
  • Test Cases - number of passing and failing Test Casesz
  • Annotations - count of Annotations ​(error and info annotations)
  • Duration - total time (if available)
  • Failures - count of failed test cases in last five result sets [new, tracked, resolved]
    • new - A unique regression that was not previously in the tracked state.
    • tracked - A case that is not a new regression but has had at least one failure in the last 5 results
    • resolved - A case that has passed or has been missing from the past five successive results and is now being removed from the tracked state.

Spaces

1) Some test frameworks don't include this information in their results.
2) This is the same information as seen in the top-level results listing

Page Tools