Test Runs

Test Run Overview

Test Run is a single occurrence of performing a set of Test Cases required to establish successful testing. A Test Run can consist of standalone Test Case or Test Cases, an entire Test Suite - or a Test Plan that contains various Test Cases from different domains.

There are two ways to perform a Test Run: an Express Test Run and a regular Test Run.

 

Express Test Run is set up by selecting multiple Test Cases in the repository and hitting the "Run" button; optionally, you can choose which Environment testing should be performed in, and which Milestone a test run will be related to:

 

To create a new regular Test Run, navigate to "Test Runs" section, and hit the "Start New Test Run" button:

 

In the configuration form, there are several fields you can fill out:

• Run title: the name of your test run, this field is pre-filled in with the current date, but you can change it to whichever nomenclature you use.
• Description: additional details about the test run.
• Plan: choose which Test Plan should be performed in the Test Run.
• Environment: define in which environment should the test run be performed (i.e., testing, staging, production).
• Milestone: select which Milestone the Test Run is tied to.
• Default Assignee: choose whether all Test Cases within the Test Run should be automatically assigned to a specific teammate; leaving this field undefined will leave the Test Cases unassigned in the Test Run (you will then need to assign them manually).
• Custom Fields: if you have created custom fields for Test Run entities, you will fill them out at this stage.
• Test Cases: if you selected a Test Plan in the Plan field earlier, all Test Cases contained in this Test Plan would be automatically included in the Test Run. However, if the Test Run does not encompass the entire Test Plan, you can choose which Test Cases from it should be performed, and which shouldn't.
• Configurations
  
  

Keep in mind that any settings established for Test Runs in the Project Settings will be applied to your Test Run:

NB: keep in mind that some settings of the Test Runs are dependent on one another.

• For example: the setting "Allow submitting results in bulk" allows you to submit multiple results for cases. However, if the option "Auto assignee" is off, and you attempt to submit bulk result for cases some of which are unassigned or assigned not to you, then the bulk action will only be applied to cases that were assigned to you.

 

 

With the new Test Run configured, you will now see it in the Test Runs view with some condensed summary of it:

• Time will show how much time has been spent so far on performing Test Cases in this run.
• Status will show a summary of test cases per status - how many have passed, how many have been failed, skipped, blocked, invalidated or untested.

 

Test Runs section is filtered to show only Active Runs by default, but you can switch it with the help of this button (and/or apply additional filters):

 

 

 

From the "..." menu, you can do several things:

• access Test Run Dashboard
• Open Wizard: this option will take you to the list of Test Cases that need to be performed, and you can go through them one by one:
  
  
  
• Clone Run: quickly create an identical instance of a Test Run, if you want to perform it again from scratch
• Edit run: change the parameters of the Test Run
• Public Report: access the publicly available Test Run Report (option appears only if public link access has been enabled for a Test Run)
• Delete Test Run

 

Test Run Dashboard

The Dashboard is a detailed overview of a Test Run and various cases it contains. You can access it either from the "..." menu or by clicking the Test Run name in the Test Runs section:

From the Dashboard, you can:

• Open Wizard: the wizard will guide you through the Test Cases contained in the run, step-by-step.
  
• Share Report: provide access to a report to anyone who needs to track testing progress - even if they are not your Qase account members - with a public link.
  
  
  
• Export: download a CSV or a PDF export of your Test Run.
  
• Edit: change the parameters of your Test Run - title, description, environment, milestone, included test cases.
  
• Abort: cancel the Test Run if it no longer needs to be performed.
  

 

There are three Tabs in the Test Run Dashboard: Test Cases, Defects, and Team Stats.

In the Test Cases tab, you can search for Cases in a Test Run by title, and filter by specific parameters:

 

For a Test Case that has not been performed yet, the "..." menu button allows you to Edit Case, Assign it to a specific user, Assign it to yourself, Run Wizard, View Case details or Delete it.

 

For a Test Case, that has already been performed, the options available are to Edit Case, Assign it to someone else, or Unassign it, send it to be Re-tested, View its details, or Delete.

 

If a Test Case has been sent to Re-test, it is only possible to go to Run Wizard, View Case details, or Delete it.

 

If you click the Result of a Test Case performed, you will see the short summary of how a Case was tested, how much time was spent on it, whether it had any defect filed as a result:

 

Selecting multiple cases in a Run, you can perform bulk actions on those Test Cases. It is possible to send Cases to be Re-Tested, Assign them to a specific user, Unassign them, or Remove these Cases from the Run, as well as submit a bulk result (depends on your Test Run settings for a project):

 

 

Defects tab is the place where you'll find every defect that has been filed during the Test Run; if you opted to create a Defect after failing a Test Case during a Run, all such defects filed during the Run will be found in the Defects tab, with additional details and actions available:

 

Team Stats tab will give you details on all of your teammates that have been participating in the Test Run, how much time they spent working, what were the results and how they have been performing over time:

 

 

 

On the right side of the Test Run view, there is a Test Run Details sidebar. It provides you with the vital details of your Test Run:

• Donut Graph shows the breakdown of Test Cases in the Run by status. Hovering over a section of the graph you will see the actual number of Cases in a respective Status. If you click on a section of the graph, a quick filter will be applied to show you only the Cases in a specific status.
• Completion Rate represents the percentage of Cases in a run that have been performed so far.
• Started by and Start time provide details on who started this Test Run, and when.
• Estimated: based on how much time has been spent in the previous Test Runs on Test Cases included in your current Test Run, Qase provides you with a prediction on how long your current Test Run is likely to take.
• Time Spent: the summary of time that has been spent on Test Cases in the current Test Run so far.

 

Test Run Wizard

Test Run Wizard is a simple tool to perform test cases in a test run.

Wizard can be accessed from several locations:

• Click "Run Wizard" in the "..." menu of a Test Case:
  
• Hit "Open Wizard" in the Dashboard:
  
• Click "Open Wizard" in the Test Runs menu:
  

 

Each of these actions will take you to the Wizard, where details of every Test Case that is to be performed are presented.

You can progress from one Test Case to the next, closing each of them with a specific status and providing a result for each step of the Test Case - and ultimately for the entire Test Case:

 

 

It is possible to find specific cases in the wizard using Search box and Filters:

 

 

When changing the status of a Test Case step, or an entire Test Case, you can also provide additional details in the form of attachments:

 

From the Wizard, a Test Case can be edited:

You can View the full details of a Test Case, including which Test Runs it was performed in - and what were the results of these runs:

 

If a Test Step or a Test Case is failed, you can file a Defect upon failing a test (a corresponding Defect will automatically be filed in the "Defects" section):

NB: if you have a "Fail case on step fail" option enabled in Project settings, failing any Test Case step will automatically lead to failing an entire Test Case. If you want to avoid that from happening, make sure to adjust this setting: