Health is defined as the current condition or state of a software build associated with a particular Space. A healthy Space is without build issues, test failures (unless exempt) and failing indicators such as thresholds and criteria for code coverage, static analysis or other custom metrics.
Space health combines the results from your build, test, coverage and analysis into single indicator:
|Healthy||0 nonexempt test failures and with all metric criteria met|
|Unhealthy||1 or more nonexempt test failures, or unmet metric criteria|
|Invalid||A result set missing required data. See explanation below.|
A result set will be declared invalid based on the following conditions:
Test exemptions allow you to exclude failing cases from the calculation of health. Exemptions are persisted and are applied to each subsequent result set until removed. Removal can be done manually or will be done automatically after the 5th consecutive passing result. Exempting a test case does not hide or alter its
Exemptions are managed from the Space Schema tab in the test suite dialog as shown below.
Descriptions of numbered areas:
To exempt a failing test case, click on the exemption icon (circled exclamation) next to the
passed|failed status to open the exemption dialog. Enter a reason to enable an exemption. Use markdown syntax to add links to your defect and issue tracking systems. Clear the dialog text to remove it.
Exemptions are removed automatically after the 5th consecutive passing result.
Project Notes are used to track exemptions for all Spaces under the project. A new note is automatically created when an exemption is enabled. Each exemption enabled for that result set is tracked by a comment added to the note. An additional comment is added for each exemption when removed. The total number of exempt test cases is summarized in the results notification email.
Current exemption totals for each suite are shown in the Schema view.
Exemption history is maintained with each result set on the Results tab.
Code coverage thresholds allow you to set minimum levels of code execution for a Space to remain healthy. A default threshold of 50% is assigned to all coverage Metrics at the time of Metric creation, which occurs the first time you upload a coverage file to a Space.
Min threshold value in the edit Metric dialog as show below and
SUBMIT your change.
Static analysis, in addition to more traditional dynamic testing, can help to identify defects or bugs (issues) early and throughout the development process. Although informational by default, setting
Pass/Fail criteria for Analysis Metrics can prevent new issues from going unnoticed. The level of issue severity (high, medium, low) for reporting or failing Health can be configured as well.
To setup analysis criteria, select the Metrics tab in the Space of interest and click on the options icon (hamburger) to the left of the Metric graph name, then select edit.2)
Update the Health and reporting criteria in the edit Metric dialog as show below and
SUBMIT your changes.
Results sets are marked invalid (1) for the following reasons:
Results due to drop in the number of Test Suites will be marked invalid until either the number of Test Suites recovers or after five results a new baseline will be set.