Running Tests

The Manual tab lists a Space's specs. Click a spec to open it, then select START to run it.

A run is either part of a Test Session (a set of specs run together for one purpose, see Session) or Ad hoc, outside any session. Both are saved as Results in the Space (see Results).

Run Spec

The spec's dialog lists its cases on the left; each case gets its own status. The dialog can be moved and resized.

Start

Select START to begin; the timer starts with it. Each case starts as Untested. STATUS is a row of icons; click one to select it:

STATUS	Meaning
Passed	Working as expected
Failed	Not working as expected
Not Applicable	Doesn't apply in this run
Blocked	Couldn't be run
Untested	Not tested yet

Each case also has:

• ISSUES: the issues linked to the case, and what to report to the issue provider (see Issues)
• a comment editor (Markdown, drag-and-drop images, issue references), with a History toggle that shows the case's comments from earlier runs

When you START from the spec's dialog, Testspace asks which context to run in: As part of an existing test session (offered when a session includes the spec, and then selected), Create a new test session, or Ad hoc, outside any formal test session (selected when no session includes the spec). Only open sessions are offered: a session you've completed doesn't take more runs. It doesn't ask when no session includes the spec and its last run was ad hoc: START continues ad hoc. With a session selected on the Manual tab, a spec runs in that session.

:::tip A spec with an automated `before`/`after` fixture has extra execution behavior; see [Automated Fixtures](automated-fixtures#execution). :::

Checklist Steps

In a Standalone Project, a case's checklist steps can be checked off while you test the case. The checks belong to this run: they don't change the spec, and each run has its own. In a Connected Project, checklist steps are shown but can't be checked.

Keyboard Support

The run dialog, and the spec's dialog before START, put the focus on the current case when they open.

• Case list: Up/Down move one case; Home/End jump to the first or last case; Page Up/Page Down move by the number of cases visible. The list doesn't wrap.
• STATUS (while running): Left/Right move between the statuses and wrap at the ends; moving selects the status. Home/End jump to the first or last status. Enter/Space select the focused status.
• START/STOP: reach them with Tab and press Enter or Space.

Stop

Select STOP to save the run. When cases are still untested, Testspace asks first:

With Keep the previous status for the N non-tested cases checked (the default), those cases keep the status they had before this run.

A spec can be run more than once; the last run counts.

Session

A Test Session is a set of specs run for one purpose, such as testing a build. Any number of sessions can be open at the same time. Each session is a Result in the Space.

New

Select the specs on the Manual tab: tick each spec's checkbox, or open the menu next to the checkbox above the list (Select specs) and choose:

Option	Selects
All	every spec
None	nothing (clears the selection)
Failing	specs whose last run failed
Untested	specs with one or more untested cases

A disabled spec can't be selected (Disabled - running is disallowed.), and neither can a spec that fails to parse.

Click New Test Session. In the New Test Session dialog:

• Name: left blank, it defaults to Sequence_n
• Exploratory (Connected only): adds an Exploratory spec to the session (see Session Observations)
• Description: optional
• Specs: the selected specs, each with a checkbox; the Specs checkbox selects or clears them all

Select SUBMIT.

The session appears as a card at the top of the Manual tab and stays there until it is completed. A card shows the session's status ring, a summary (not started, N failing or N/N passed), when it was created or last tested, and its open issues. Click a card to select the session: the list then shows only its specs, and a spec you open runs in it. Click the selected card again to show all specs.

Complete

Complete a session when its testing is done, whatever has been run. Select the session and click the blue check on its card (Complete this test session), or open the card's menu and select Complete. The same menu has Edit and Delete. Organization admins and the session's creator can edit and complete it.

• A completed session leaves the Manual tab.
• Its specs can't be run in it any more; it stays as a Result.

Folders

Specs can be organized in folders: in a Standalone Project a spec's folder is set when it is created (Creating & Editing Specs); in a Connected Project folders come from the repository (Manual Spaces).

The Manual tab shows the folders in a column on the left:

• ·Overall· covers every spec: its coverage (tested cases), its pass rate, its open and closed issues, and the number of cases and specs
• one tab per folder (when there is more than one), with its number of cases and specs; specs at the root are under No folder

Each has a status ring; hover it for the counts. Click a folder to list only its specs. With a session selected, the folders show that session's status.

Assignee

A spec can be assigned to a tester in the Assignee column: click it and choose a user, or Clear assignee. New sessions start with that assignee. Running a spec makes you its assignee in that session.

Spec Changes

In a Connected Project, specs change in the repository. When a spec has changed since its last run and a case that was run (not left untested) is missing from it under that name, Testspace warns that the status shown may not match the spec:

• on the Manual tab, an orange warning icon next to the spec (Presented status may be mismatched due to newer source code changes.), linked to the file's history in the repository
• in the spec's dialog, a banner: Presented status and content may be mismatched due to newer source code changes.

An open session keeps the spec as it was when the session was created; sessions created later use the new version.

caution

A case is identified by its name (its ## heading). Renaming a case starts it over: it loses its history, and its tracked failures are retired (see Results and Failure Tracking).

Issues

Cases can be linked to issues in the Project's Issues provider, the same provider that automated Results use. You can push new issues or reference existing ones.

Push New Issue

The select next to a case's ISSUES says what to report when you select STOP:

Option	Does
Do not report	nothing
Push new issue	creates an issue; edit its title in the field next to the select (by default [Failed] <spec> - <case>). The issue includes the case's comment, and the new issue is added to the case's ISSUES.
Add Comment to #N	adds the case's comment to an issue already linked to the case

Setting a status picks a default:

Status	Default
Failed or Blocked	Add Comment to the case's open linked issue, if it has one; otherwise Push new issue
Passed or Not Applicable	Add Comment to the case's open linked issue, if it has one; otherwise Do not report
Untested	Do not report

The select is disabled, and reads Do not report, unless the Space isn't a sandbox and the Project has an enabled, connected issue tracker (GitHub, GitLab, Bitbucket or Jira); the generic provider can't create issues or comments. Its tooltip gives the reason.

tip

Issues that Testspace creates can carry a label: issues.label in .testspace.yml for a Connected Project (Issues), the Label field of a Standalone Project (Label). See Issue Providers for which providers support labels.

Reference Existing Issue

To link an existing issue, click the pencil in the case's ISSUES row (Edit issues) and paste the issue's URL. With a provider other than generic, a short reference also works: #123 or PROJ-123. Issue URLs and references in a case's comment are picked up the same way when you leave the comment box.

Linked issues show on the Manual tab's Issues column and on the Result's Issues tab. The Space's Issues metric graphs open and closed issues per Result (see Issues).

Exploratory

Exploratory testing is unscripted testing based on observation: it adds coverage and assesses usability beyond what the specs describe. Testspace supports two kinds of observations, and both can push issues.

Spec Observations

Every spec has an Observations entry at the end of its case list, for anything noticed while running the spec. It has no status, and its ISSUES select defaults to Do not report; a pushed issue is titled [Observations] <spec>.

Session Observations

info

Session Observations are available in Connected Projects only.

Check Exploratory in the New Test Session dialog to add an Exploratory spec to the session (on the Manual tab it is the last row, Do exploratory informal testing and provide observations). Several testers can run it and add their observations. Testspace creates one issue for the session, [Exploratory] <session name>, and adds each tester's observations to it as comments.

Results

A session is a Result: it is listed on the Space's Results tab and opens on the same Result page as published Results, with its specs as Suites. An Ad hoc run is saved as a Result too, named Run '<spec>' spec.

Spec Listing

The Manual tab's columns show each spec's last run:

Column	Shows
Status	a bar and N passed · N failed · N blocked · N untested; hover for not applicable and exempt cases. Not run yet when the spec has never run. The failed and blocked counts open those cases in the Result of the run.
Issues	N open · N closed, the issues linked to the run's cases; opens that Result's Issues tab
Tested	when the spec last ran, and below it the session (Ad hoc for an Ad hoc run), the duration and who ran it
Assignee	see Assignee
Updated	Standalone only: when the spec last changed, and its version and author (vN · name)

Automated and Manual Results Together

A Space with Manual testing combines everything tested there into each Result: your spec runs, and the automated results published to it. Its active Result always shows the latest status of every spec next to the latest automated run, with one Health and one Failures list for both. What a Result carries from the one before depends on its kind:

• A session's Result holds the specs run in it plus everything else from the previous Result: every other spec's last run, and its automated Suites (in a Connected Project, those whose file is still in the repository).
• A published automated Result holds what it ran plus the Manual specs; earlier automated Suites aren't carried.

On the Result page, Show switches between All and Ran in this result, and carried Suites are marked (see Show). A Result's Duration counts only what ran in it.

Open Spec Run

On the Result page, click a spec in All tests, or Open spec run on a case in Failures or Issues, to see the run: its cases with their statuses, issues and comments. The dialog only shows the run, with no START; run specs from the Manual tab. When the spec has added or removed cases since that run, a line says so (The spec has changed since this run: 2 cases added, 1 removed); hover it for the names.

A failing case in the Space's active Result has a Triage pencil in this dialog, and in the spec's dialog on the Manual tab. A Manual case can't be exempted.

Results and Failure Tracking

Manual Results feed Failure Tracking like automated ones, with Manual's rules:

• a tester's pass resolves a failure at once, so a Manual failure is New or Consistent, never Passing or Flaky
• deleting a spec retires its failures at the next session; removing or renaming a case retires its failures at the next session that includes the spec (renaming also loses the case's history)
• turning Manual testing off retires every spec's failures at the next Result

See In Manual Testing.

Sandbox Spaces

In a sandbox Space you can START a spec to try it out, but the run is a trial and isn't saved: the run dialog says In "sandbox" mode, this run isn't saved. The specs themselves are kept. Sessions can't be created (Functionality is not available in "sandbox" mode.), and no issues are pushed.