User Tools



Schema

The Schema tab presents a representation of your test hierarchy based on the most recent results. It's primary purpose is to help navigate the results to understand and manage failures.

Description of the numbered areas:

  1. Metric Badges - as shown on all Space tabs. See Metrics for more information
  2. Failures - provides a list of suites with failed test cases.
  3. Location - current location within the test hierarchy (root level viewed above).
  4. Map - provides quick navigation within the test hierarchy.
  5. Annotations - ancillary data and information for the current level.
  6. Folders/Suites - test items at the current level.
  7. Result Set - most recent results defining the hierarchy.

Each Folder and Suite is displayed with columns that show roll up totals for their child items.1). Roll-up totals of the following are shown for each:

  • Suites - number of passing and failing Suites
  • Test Cases - number of passing and failing Test Cases
  • Failures - count of failed test cases (new, tracked, resolved)
  • Duration - total time (if available)

Suite Report Dialog

Clicking on a Suite Name in the schema opens the Suite dialog as shown below. The dialog is used to view and manage status for each case in the suite.

Descriptions of the highlighted areas:

  1. Suite name - name of the selected suite show in the dialog
  2. Suite annotations - uploaded with the test results, see Collect Data for more details
  3. Exemption reason - used to exempt the test case
  4. Failure annotation - with link to content which opens in a new tab
  5. Exemption switches - currently unexempted
  6. Failure frequency - number of failures in the last five results if available Clicking the number brings up a dialog with pass/fail information of the results. Historical information is only available if there were additional failures in the last five result sets. This allows for the determination of a whether a failure is new, recurring or other type.

Click here for more information on managing health through exemptions.

The Suite Report is displayed with columns that show roll up totals for their child items. Roll-up totals of the following are shown for each:

  • Status - number of passing and failing Suites
  • Test Cases - number of passing and failing Test Cases
  • Annotations - count of Annotations ​(error and info annotations)
  • Duration - total time (if available)
  • Failures - count of failed test cases in last five result sets [new, tracked, resolved]
    • new - A unique regression that was not previously in the tracked state.
    • tracked - A case that is not a new regression but has had at least one failure in the last 5 results
    • resolved - A case that has passed or has been missing from the past five successive results and is now being removed from the tracked state.

Report Annotations

Report Annotations make it possible to include supplemental data to a result set you upload. The supplemental data becomes part of your results and can be downloaded as long as the result set exists. Annotations are well suited for binary data, such as Screen Shots and Media files. Textual files of type .txt, .log, .md and .html should be published to their own separate suite. See Collect Data for more information.

Each annotation gets attached to a Folder, Suite or Test Case in your Results. Annotations can then be accessed from the view showing the Results level where the Annotation resides. By default, Annotations are attached to the Root Folder of your results, but you can specify a different node if desired.


Using The Map

The Map dialog presents the test hierarchy in a collapsible tree. Exemptions and pass/fail counts are summarized for each suite

Double-click on the desired node for quick navigation. Your current location within the tree is shown in the banner at top.


Spaces

1) This is the same information as seen in the top-level results listing

Page Tools