Test Run Overview

Test Run is a single occurrence of performing a set of Test Cases required to establish successful testing. A Test Run can consist of a standalone Test Case or multiple Test Cases, an entire Test Suite - or a Test Plan can be used that contains various Test Cases from different domains.

There are two ways to perform a Test Run: an Express Test Run and a regular Test Run.

Express Test Run

It is set up by selecting multiple Test Cases in the repository and hitting the "Run" button; optionally, you can choose which Environment testing should be performed in, and which Milestone a test run will be related to:

Regular Test Run

To create a regular test run, navigate to "Test Runs" section, and hit the "Start New Test Run" button:

In the configuration form, there are several fields you can fill out:

  • Run title: the name of your test run, this field is pre-filled in with the current date, but you can change it to whichever nomenclature you use.

  • Description: additional details about the test run.

  • Plan: choose which Test Plan should be performed in the Test Run.

  • Environment: define in which environment should the test run be performed (i.e., testing, staging, production).

  • Milestone: select which Milestone the Test Run is tied to.

  • Default Assignee: choose whether all Test Cases within the Test Run should be automatically assigned to a specific teammate; leaving this field undefined will leave the Test Cases unassigned in the Test Run (you will then need to assign them manually).

  • Tags: you can assign any of the tags that you have used previously. Or you can create a new tag as well.

  • Custom Fields: you can set up custom field values for a run, if you have configured any such custom fields previously.

  • Test Cases: if you selected a Test Plan in the Plan field earlier, all Test Cases contained in this Test Plan would be automatically included in the Test Run. However, if the Test Run does not encompass the entire Test Plan, you can choose which Test Cases from it should be performed, and which shouldn't.

  • Integrations: You can link any of the integrated apps’ issues with your new test run as well.


Keep in mind that any settings established for Test Runs in the Project Settings will be applied to your Test Run:

With the new Test Run configured, you will now see it in the Test Runs view with a condensed summary of it:

  • Time will show how much time has been spent so far on performing Test Cases in this run.

  • Status will show a summary of test cases per status - how many have passed, how many have been failed, skipped, blocked, invalidated, or untested.

  • Environment will show the environment of the run, which was selected when the run was created.

You can filter by Status, Assigned to, Environment, Milestone, Tags, Automation status, and Creation date.

From the "..." menu, you can do several things:

  • access Test Run Dashboard

  • Open Wizard: this option will take you to the list of Test Cases that need to be performed, and you can go through them one by one:

  • Clone Run: quickly create an identical instance of a Test Run, if you want to perform it again from scratch

  • Edit run: change the parameters of the Test Run

  • Delete Test Run

Test Run Dashboard

The Dashboard is a detailed overview of a Test Run and various cases it contains. You can access it either from the "..." menu or by clicking the Test Run name in the Test Runs section:

From the Dashboard, you can:

  • Open Wizard: the wizard will guide you through the Test Cases contained in the run, step-by-step.

  • Share Report: provide access to a report to anyone who needs to track testing progress - even if they are not your Qase account members - with a public link.

  • Export: download a CSV or PDF export of your Test Run.

  • Edit: change the parameters of your Test Run - title, description, environment, milestone, included test cases, and app integrations.

  • Abort: cancel the Test Run if it no longer needs to be performed.

There are two tabs in the Test Run Dashboard: Test Cases, and Team Stats. Once you create at least one defect during the test run, there will be another tab “Defects” to track all defects. In the Test Cases tab, you can search for Cases in a Test Run by title, and filter by specific parameters:

For a Test Case that has not been performed yet, the "..." menu button allows you to Edit Case, Run Wizard, Assign it to a specific user, Assign it to yourself, View Case details or Delete it.

For a Test Case, that has already been performed, the options available are to Edit Case, Re-test case, View its details, or Delete.

If a Test Case has been sent to Re-test, it is only possible to go to Run Wizard, Assign the case, Unassign the case, View Case details, or Delete it.

Selecting multiple cases in a Run, you can perform bulk actions on those Test Cases. It is possible to send Cases to be Re-Tested, Assign them to a specific user, Unassign them, or Remove these Cases from the Run:

Team Stats tab will give you details on all of your teammates that have been participating in the Test Run, how much time they spent working, what were the results and how they have been performing over time:

On the right side of the Test Run view, there is a Test Run Details sidebar. It provides you with the vital details of your Test Run:

  • Donut Graph shows the breakdown of Test Cases in the Run by status. Hovering over a section of the graph you will see the actual number of Cases in a respective Status. If you click on a section of the graph, a quick filter will be applied to show you only the Cases in a specific status.

  • Completion Rate represents the percentage of Cases in a run that have been performed so far.

  • Started by and Start time provide details on who started this Test Run, and when.

  • Estimated: based on how much time has been spent in the previous Test Runs on Test Cases included in your current Test Run, Qase provides you with a prediction on how long your current Test Run is likely to take.

  • Time Spent: the summary of time that has been spent on Test Cases in the current Test Run so far.

  • Environment, Tags and Custom Fields: Details about the specific environment of the test run and the associated tags.

Test Run Wizard

Test Run Wizard is a simple tool to perform test cases in a test run.

Wizard can be accessed from several locations:

  • Click "Run Wizard" in the "..." menu of a Test Case:

  • Hit "Open Wizard" in the Dashboard:

  • Click "Open Wizard" in the Test Runs menu:

  • You can also click on the title of the test case in the run to open the wizard:

Each of these actions will take you to the Wizard, where details of every Test Case that is to be performed are presented.

You can progress from one Test Case to the next, closing each of them with a specific status and providing a result for each step of the Test Case - and ultimately for the entire Test Case:

It is possible to find specific cases in the wizard using Search box and Filters:

When changing the status of a Test Case step, or an entire Test Case, you can also provide additional details in the form of attachments:

From the Wizard, a Test Case can be edited:

By clicking on “View”, you will be redirected to the Test Case in the repository:

If a Test Step or a Test Case is failed, you can file a Defect upon failing a test (a corresponding Defect will automatically be filed in the "Defects" section):

NB: if you have a "Fail case on step fail" option enabled in Project settings, failing any Test Case step will automatically lead to failing an entire Test Case. If you want to avoid that from happening, make sure to adjust this setting:

Editing a Test Run result

If you would like to make changes to the results in an open run, you can do so by navigating to the Test Runs page and once you've opened the run, click on "Open Wizard". In the wizard, click on the edit icon (paper and pen icon). You can then add a comment or an attachment and update the duration of the run. Once done, click on the green checkmark icon, or if you would like to discard it, click on the red cross icon.

Also, once you've completed a Test Run, you can choose to add results to the run. To do so, you will need to enable the setting "Allow to add results for cases in closed runs. Once you've enabled this setting, you can navigate back to the Test Runs page and click on the "Open Wizard" option. In the wizard, click on the edit icon (page and pen icon). Once the Run is open for edits, you can make changes to the duration of the run, add a comment, and add attachments. Once you've made the changes, you can click on the green checkmark icon to save the changes. If you would like to discard the changes, click on the red cross icon to discard changes.

Test Run Settings

If you navigate to Project settings, you will find a section dedicated to the options that define how the test run behaves:

Let's take a look at those settings and how they change the behavior of the run.

  • Fast Pass: if the option is switched on, then upon marking a test case as "passed", you won't be prompted to add extra details like comments or attachments. Vice versa, if you disable this option upon marking a case as "passed", you will see a modal window that will allow you to submit extra details with the result:

  • Default create/attach defect checkbox: with this option enabled, the state of the checkbox for creating/attaching a defect will be checked, with the option disabled, the checkbox state will be unchecked by default when you fail or invalidate a case or mark it as "blocked":

  • Auto complete: with this option enabled, as soon as you submit results for all cases of a run (regardless of which results those are, but Skipped result does not count here) the run itself will get marked completed automatically. With this option disabled, you'll need to mark the run completed manually, even if all the results have been submitted.

  • Auto passed: when this option is enabled, once you mark all steps of a case "passed", the entire case will also get marked "passed" automatically. If you have this option disabled, you will need to mark the case "passed" separately, even if all of its steps have been marked as "passed".

  • Auto assignee: enabling this option will result in any unassigned test cases getting automatically assigned to the first person opening them in the Wizard. Conversely, if you have this setting switched off, the unassigned cases will remain unassigned even if opened by someone, until they are explicitly assigned to someone specific.

  • Auto create test cases: this option relates specifically to automated executions. With this option switched on, if you are sending an automated execution result for a test case that is not yet represented in your Qase repository, such a case will be created automatically (later on, if something changes in that test case upon the next result submission, related test case in the repository will get updated accordingly with this option enabled). With the option switched off, there will be no new cases created, if an automated execution result refers to a non-existent test case in the repository.

  • Fail case on step fail: with this option enabled, failing any step of a test case will result in the immediate termination of the entire case and marking it "failed" as well. If you disable the option, even after one of the steps has failed, you will be able to proceed further with the case.

  • Allow to add results for cases in closed runs: when this option is enabled, it allows you to continue submitting results for cases, or retest cases in those runs that have been marked complete or aborted. If you switch that option off, any runs that have been completed or aborted would not allow any further result submissions.

  • Assignee result lock: when this option is enabled, it restricts other users but the assignee from submitting the results - for someone to submit a result in such a case, they would need to assign themselves to a case explicitly. Conversely, if this option is disabled, any user - even if not assigned to a case - will be able to submit a result for it (however, even though the case would stay assigned to the original assignee, the result itself will be associated with the exact person that submitted the result).

  • Allow submitting results in bulk: when this option is enabled, it allows you to submit a result for multiple test cases - keep in mind that for all cases you can submit the same status, the "time spent" you insert is distributed evenly between the cases you submit a result for, and the comments and attachments added will get attached to every case you submit a result for in bulk:

    Conversely, if the option is switched off, there will be no way to submit results for cases in bulk.

  • Redirect after adding result: this option establishes the policy for where you will be taken after the result has been submitted, there are three options:

    • No redirect - you will stay on the same case for which you have submitted a result;

    • First case in a run without result - you will be redirected to the first case that has no result yet;

    • Next case in suite - you will be taken directly to the next following case in the suite that does not have any result yet.

Did this answer your question?