There are two distinct types of Projects: Connected and Standalone.
Connected project is "connected" to a single Git repository through an integrated service:
- Each project is connected to a single Git repository.
- Each space under the project maps to a branch in the repository.
- Spaces are automatically created with each new branch and are hidden when the branch is deleted.
Implementing and running manual tests requires
Connected projects. As of this writing, the manual functionallity is available only using
Manual tests require
- Results for each git push tested are collected and analyzed for the life of the branch.
- Metrics and trends detail the history of the branch’s health – regression rates, test and coverage growth, code churn, and more – indispensable data when reviewing pull requests before merging.
- Metrics for all branches, maintained after deletion, are used to calculate Insights for the project.
- Connected projects work with common CI services, such as GitHub Actions, Azure, Travis, Circle CI, etc.
Publishing automated results using
Connectedprojects works with common CI services.
To create a new project click the
New Project button at the top right of your organization's view.
Creating a new project or editing the properties of an existing project requires Administrator or Owner privileges. Also, note that a project's type (connected vs standalone) cannot be changed.
Select one of the available repositories and click OK. All currently connected repositories will be
grayed out and unselectable. A
public repo is represented by an open lock vs a closed lock for
|Name||Read-only. Derived from the connected repository (|
|Description||Read-only. Derived from repository description.|
|Public||Read-only. Controlled by the access setting for the repository: |
|Users||A selection of users that have access to the project with their current email notification settings.|
Spaces under the project represent the repository's branches that have associated test results. Only the default branch (i.e. master) will be listed even without having results.
When a new branch in the repository is created a new corresponding space is automatically created too:
- It will be named after the branch name
- Is pre-populated with exemptions and metrics from the most recent result in the space corresponding to the base branch (if such data exists). The prepopulated "base" content is given an auto-generated result name:
There are also spaces automatically create:
- For each Fork Pull Request to the repository, named after the PR
PR-123) and pre-populated with exemptions and metrics from the most recent result in the space corresponding to the PR target branch.
- Single, named :TAGS:, for results representing all Tags in the repository.
|Name||Read-only. Derived from the branch name.|
|Description||Textual description of the space. The description text supports a subset of Markdown.|
|Sandbox||Read-only. Branches that are "non" default branches are automatically set as a sandbox.|
|Health Indicators||Allows you to specify a Publication deadline for results in days. Exceeding the deadline fails the space's health. A value of ''0'' disables the deadline (default).|
To edit the settings of an existing project, in the organization's
Projects tab, hover your mouse over a row in the listing, on the very left a "hamburger" icon will be shown. If you left-click on the icon, a menu is displayed with options to
Testspace can be customized to manage "non" default branches as release branches, preventing the sandbox attribute from being enabled. To control that behaviour a
.testspace.yml configuration file is required to be placed at the root of your
repo, for example:
.testspace.yml content that specifies any branch with
_release name ending as a release branch:
Sandbox spaces provide a fully-functional space that can be used for non-mainline tasks. They are used primarily for staging new tests, and for providing a sandbox when bug fixing or experimentation.
connected Projects, unless explicitly suppressed via configuration, the
sandbox state of a space is automatically set based on the existance of a non-draft Pull Request with the corresponding branch (or fork) set as a base.
Sandbox spaces have the following attributes:
- Email notifications are never sent regardless of the space's settings
- These spaces are not shown in the
- These spaces are shown only on their parent's
- They are excluded from project
A Space in a
connected Project follow the life time of its corresponding branch (or fork-PR) - it is automatically hidden when the branch is deleted (or fork-PR closed).
All metric data is maintained to provide statistics for the project.
The lifetime of individual Results is bound to a separate retention policy.
A repository deletion does not result in the connected project being deleted. A project can only be deleted manually through its options.
Deleted projects may be restored up to thirty days after they are deleted. Refer to the Standalone Project Deletion for restoration policy.