We’re celebrating TestRail Server customers everywhere by dropping a massive update! This release is packed with upgrades and finally brings TestRail Server into near-parity with TestRail Cloud.
TestRail Server 7.4 is packed with upgrades including new interactive charts and huge improvements to how you attach media files to test cases and test results. We also fixed a bunch of issues along the way and made some performance improvements under the hood.
Tired of finding inaccurate test cases in your library? It’s time to get proactive! TestRail Server Enterprise teams are getting new workflows for reviewing and approving test case designs. We’re also introducing test case versioning, making it easy to analyze and compare changes made to test cases over time.
Join us on Tuesday, April 12 at 11 pm EDT for a live webinar to learn more!
Overview
Attachment management improvements
We know how important it is to be able to supplement the design and results of your testing with screenshots, gifs, documents, and other artifacts. Attaching that information to your test cases and inserting it into fields so it can be viewed alongside instructions or testing results are important capabilities for any tester. TestRail Server 7.4 contains a range of enhancements intended to make life easier for attachment power users – and everybody else!
This release contains improvements to the attachments dialogue throughout TestRail, geared towards more flexible and intuitive use of attachments when creating test cases, runs, plans, and milestones, and when adding results or pushing defects to your integrated tools.
To make life easier for administrators, we also added a new attachment management area in the administration console and included a filter to help you identify exactly which attachments may be ripe for pruning.
New interactive charts
We’ve implemented a new charting library developed by our sister company FusionCharts. New interactive charts on virtually every view in TestRail provide you with greater visibility and faster analysis of all of your testing efforts. Basically, anywhere you see a chart in TestRail, you should be able to click on a data point and get a view of the tests that make up the item of interest.
In addition to the chart replacements, we’ve made it easier to grab images of the charts should you wish to use them in bespoke reports, emails, and the like. So the next time someone asks you for an update, share it in a picture – worth more than a thousand-word email or an even longer meeting.
Test case reviews
TestRail Server Enterprise teams get test case reviews and approval workflows, allowing quality leaders to collaborate with test case designers to deploy only accurate test cases to a library. TestRail Server 7.4 introduces test case statuses, assignees, and comment abilities to manage test case changes and ensure test cases written meet organizational standards.
Test case versioning
Also new for TestRail Server Enterprise customers, we’ve made some major improvements to test case histories. Now you can analyze changes made to test cases over time. We even made it easy to compare any two test case versions side-by-side by highlighting for you any differences between the two versions. And to get you to a faster resolution, you can on the spot choose to restore an entire test case version or individual field values as needed.
Attachment management improvements
Improved UX when adding/removing attachments to/from entities inside of TestRail
One of the main things you’ll notice in TestRail Server 7.4 is the new attachment dialogue when it comes to uploading attachments to your TestRail entities (cases, runs, plans, milestones, test results, and defect pushes where supported).
The dialogue introduces the idea of a media library. In addition to simply uploading artifacts and attaching them to your test cases in an ad-hoc fashion (‘scuse the testing pun!), your attachments now become part of a library from which they can be re-used in the future.
As such, the dialogue includes a number of other features intended to support the use and re-use of attachments across TestRail entities, including across projects if needed. You can upload attachments to the attachment library by browsing your local filesystem or drag-and-drop.
As you build up a library of attachments, you can sort through them to find the attachments you need by Date, Name, or Size. You can also search for a specific attachment(s) by name—and to make that easier, we’ve added the capability to change the names of your attachments by clicking on the attachment and viewing its details:
The Attachment Details view opens a wealth of further information and adjustment opportunities. You can:
- Change the attachment name
- View the file type, size, and upload timestamp
- View and copy the unique URL for the attachment
- Change the projects the attachment is visible in
- Delete the attachment
- Replace the attachment with a new one and keep the same name and URL
- View the attachment at full resolution
We think you’ll agree: that’s all pretty swish.
In addition, you can also refer to an attachment throughout TestRail and update the referenced attachment in all places using the Replace function.
New attachment management console for administrators
In TestRail Server 7.4, we continue to improve the administrators’ lot by introducing a new attachment management console in the Admin area. Administrators will be able to carry out all of the actions available in the improved attachments dialogue, with the benefit of being able to see all attachments across all projects.
This means that administrators can also take advantage of the search and filtering mechanisms to prune attachments from their instances if they’re no longer needed, helping to reduce the storage footprint of the instance.
Improved attachment performance and management
While adding these features, we’ve made some significant changes under the hood of TestRail, implementing a whole new database structure for the management of attachments to improve performance and help make sure that attachments can be searched and filtered effectively.
For server customers, some additional installation requirements need to be followed to enable the configuration of a Cassandra database. TestRail Server administrators can refer to our detailed guide on the TestRail documentation site.
Attachment deletion
Whilst carrying out these improvements, we fixed or tweaked behavior in a number of other places too:
- By default, attachments will no longer be deleted when the entity to which an attachment is associated is removed (e.g. a test case, or run), to prevent problems where attachments are utilized in other entities in TestRail. Attachments must now be deleted explicitly from within the attachment library dialogue.
- This rule applies to the removal of attachments from fields and sidebars also. The attachment will remain in the attachment library unless it is explicitly deleted.
- We were aware of similar issues whereby attachments could be orphaned in the event they were uploaded to a dialogue (e.g. a test result) and the dialogue was subsequently abandoned. The use of an attachments library means all attachments are uploaded to the library and must be explicitly deleted from it.
- We have added a filter to the Attachment management console in the administration area which will allow administrators to easily identify and remove attachments that are no longer linked to a valid entity (e.g. attachments that used to be linked to a test case or run which have now been deleted). Administrators should use the “Orphaned” filter for this purpose.
- Problems in other areas (such as attachments added to plan entries) are resolved by the library mechanism as well.
New improved charts
In TestRail Server 7.4, we’ve implemented a new charting library developed by our sister company FusionCharts. You’ll see the improved charts on virtually every view in TestRail, including the Dashboard, the Project, Milestone, Plan, Run, and Todos views, as well as the activity, history, and defects views.
You’ll also find the new charts have been implemented in the out-of-the-box reports.
Although you may not notice a huge amount of difference in the charts themselves (they’ve been replaced on a like-for-like basis), the implementation of the new charting library paves the way for further improvements we’ve got planned for the future, so stay tuned for those!
In addition to the chart replacements, we’ve made it easier to grab images of the charts should you wish to use them in bespoke reports, emails, and the like. When you’re viewing a chart in the context of the TestRail application (i.e. not in a Report), you can now download the chart image directly by clicking on the image button.
Similarily, you can now also download the high-level metrics for the chart, by clicking on the CSV button.
But that’s not the most exciting part…
Drill down from chart directly to test data capability from most charts
In TestRail Server 7.4, you can now get to the information that’s important to you much more quickly and easily, by simply clicking on the data point of interest to you on the chart itself. So, from the Dashboard, you can get to your Project activity:
Or, in the context of a Project Milestone, you can get to your failing tests simply by clicking on the relevant pie chart status segment:
Or you can go to your Progress tab, and quickly load up the tests that still need to be run to stay on track with your execution timeline:
Basically, anywhere you see a chart in TestRail, you should be able to click on a data point and get a view of the tests that make up the item of interest. Except in the todos area where the ability to click through to runs and see what tests are assigned to the tester of interest was already present.
When you’ve opened up a drill-down view of your tests, you can expand the test and see more detail by clicking on the plus (+) button on the left of the test, and collapse it by clicking on the minus (-) button.
You can also click through (or drill down) from the status bars in TestRail. As an example, if you’re looking at your milestones and want to see which tests need to be re-tested, you can click on the re-test segment, and a view of all the tests in a status of Retest will be displayed in a new browser tab (in the Activity view).
Print report configuration and filtering capabilities
In TestRail Server 7.4, we improved the print report capability (accessed via the printer button in milestone/plan/run/cases views) so that it can now be configured and filtered to remove or include required information before printing.
The print report capability is available in the Milestone, Plan, Run, and Cases views, and can be accessed by clicking the print button in the toolbar at the top of those views.
Prior to this release, carrying out the print report action would have generated a report containing all of the tests and results in the context of that view, with the capability to select between a detailed or an outline perspective.
In this release, we’ve introduced some new sort and filter capabilities to the print report configuration, so that you’re able to further configure the report — selecting which data is actually included within it, and where.
Report administration improvements
We’ve made some report administration improvements too! Administrators now have better access to reports so they can make changes if necessary, reports can now be deleted in bulk (assuming you have the relevant permissions), and a “run until” date can now be added to scheduled reports.
Test runs, plans, & milestones can now be deleted more easily and in bulk
And by popular demand, we implemented the same bulk deletion capability throughout the Milestone, Plans, and Runs views, so that your teams can more easily remove data that are no longer required.
Test Case Review
Enterprise teams can now implement a test case review and approval process for testers. When a tester makes a change to a test case, its status will be reverted to default status, such as ‘Design’, indicating changes have been made, but not approved. The tester can then assign the test case to a reviewer who can approve the changes, deny changes, and add comments to the test case version. Let’s take a look at how the review process can be configured and enabled for projects.
First, a TestRail administrator should review the default test case statuses and make edits or add new statuses as needed. These statuses are available under ‘Administration > Customizations > Case Statuses. By default, we’ve added a few statuses for test cases: Ready, Design, and Review. However, administrators can modify these statuses and add new ones to meet business needs. Please keep in mind that there must always be one Default status that applies when a test case is edited or created (unless the status is changed while editing), and there must also be at least one status which is an ‘Approved’ status (more on this later).
Administrators should also grant permission to Approve test cases to the user roles responsible for reviewing test case changes. Users without approval permissions will be able to change statuses of test cases to any status which isn’t ‘Approved’ for testing.
Finally, test case versions and reviews are an optional setting for projects. For any project where a review process is required, Administrators can edit the project settings and enable Test Case Approvals. This will enable statuses for test cases. Also, any Test Run configured with ‘All Test Cases’ will only include test cases that are in an approved status. Test cases that are not approved will not be included in these runs. It will still be possible to create test runs using specific filters for cases with any status, but runs using ‘All’ cases will exclude tests that are not approved.
Test case assignment and status
When test cases need to be reviewed, edited, and/or approved, testers and test managers will be able to assign test cases to various users, add a comment, and change the case status all in a single dialog. The following screenshot shows the dialog for updating a test case’s assignee and/or status:
NOTE: If test case reviews are not enabled for a project, testers will still be able to assign and comment on test cases, however, the status field will not be available.
It’s also possible to assign and comment on test cases in bulk:
The status of multiple test cases can be updated using the bulk edit screen for test cases (by selecting multiple test cases within a test suite, then pressing the ‘Edit’ button. Please keep in mind that if the approval setting is disabled for the project, the status field will not appear (the assignment and comment options will still be available).
Click here to learn more about Test Case Review Processes >
Test Case Versioning
When viewing the change history for a test case, Enterprise users will now see some new buttons for comparing, restoring, and commenting on test case versions:
When a tester compares two test case versions, they will be presented with a new interface that shows both versions of the test case side-by-side. Any differences between the two versions will be highlighted in Red. The reviewer can choose to restore an entire previous test case version, or select individual field values and revert only specific changes.
Each test case version also has comments associated with it so reviewers can add comments to the test edits. It’s also possible to view all comments associated with a test case:
Click here to learn more about Test Case Versioning >
Shared test step histories
In addition to the changes to test case histories, shared test steps now have histories associated with them, as well, so users can see who made changes to the steps over time. The change history is available for both Professional and Enterprise users.
Similar to test cases, users can compare two versions to each other and restore a previous version. At this time, the restore option for shared steps only allows restoring the entire set of steps, not data from individual steps. Unlike test cases, shared steps will not have statuses, comments, or assignees.
Click here to learn more about Shared Test Steps >
TODO area improvements
Now that test cases can be assigned to users, the TODO area for a project received an update as well. We’ve added a tab to track test cases in non-ready states so users can see which test cases need attention, along with a quick view into recent comments on the test case.
Another change to this page is the ability to view the workloads of other testers. If you review the permissions screenshot from earlier in this blog post, you can see an additional permission for viewing workloads. This setting allows administrators to choose which users are allowed to view the dashboard of test cases and tests assigned to other users. While this doesn’t seem like a big change, this can help with TestRail’s overall performance by reducing the amount of data being pulled when users view the TODO area.
API Improvements and Changes
Attachment management endpoints
We’ve made a few adjustments to the attachment API endpoints as well. By popular request, we’ve added run_id, plan_id and plan_entry_id’s to the associated get_attachments_for_run, get_attachments_for_plan and get_attachments_for_plan_entry endpoints.
We’ve improved the behavior of the various add and get attachment endpoints so that they align better with what we’ve come to understand is the (still somewhat complex) expected behavior:
- get_attachments_for_run returns plan entry attachments if no run-specific attachments have been added.
- get_attachments_for_run returns run attachments if plan entry runs specific attachments have been added.
- attachments added to runs in plans via the add_attachment_to_run endpoint are visible when viewing the runs in the UI.
- ID’s of attachments added to a run in a plan via add_attachment_to_run endpoint is returned when calling get_attachments_for_run endpoint.
- get_attachments_for_plan_entry only returns attachment IDs for the requested plan entry.
Other things to mention here are that the ID of the attachments returned via get_attachment endpoints has changed from a simple integer to a reference:
{
“attachment_id”: “2054f652-fe2f-4542-acb0-422910ed87e3"
}You’ll also see some additional attachment metadata fields in the attachment response bodies. Please check the attachments API page for further details.
Test case review and versioning endpoints
Updates to existing endpoints:
- Test case endpoints will return assignees, statuses, and comments
- get_cases supports filtering by assignee and status
- Endpoints for adding and editing test cases will support setting test case assignees
- get_case_history will return test case comments
- Project endpoints will return a field indicating whether or not test case approvals is enabled
New endpoints:
- get_case_statuses will return the available test case statuses
- get_shared_step_history will return the change history of a set of shared test steps.
Example Responses:
get_case_statuses
{
“offset”: 0,
“limit”: 250,
“size”: 3,
“_links”: {
“next”: null,
“prev”: null,
}
“case_statuses”: [
{
“id”: 12,
“name”: “Approved”,
“abbreviation”: “Approved”,
“is_default”: false,
“is_approved”: true
},…
]
}
get_shared_step_history
{
“offset”: 0,
“limit”: 250,
“size”: 7,
“_links”: {
“next”: null,
“prev”: null,
}
“step_history”:
[
{
“id”: 7
“user_id”: 74,
“timestamp”: 1632774015,
“custom_steps_separated”:
[ {
“content”: “Navigate to https://192.168.147.2/login”,
“additional_info”: null,
“refs”: null,
“expected”: “Login page loads”
},…
]
},…
]
}Getting TestRail Server 7.4
Try out the new features
You can start a 14-day free trial of TestRail here (cloud or server): http://www.gurock.com/testrail/trial/
Updating to TestRail Server 7.4
Registered customers can download the full version from our customer portal: http://www.gurock.com/portal/
Get Started with TestRail
To subscribe to TestRail Cloud, you can do so from within TestRail via Administration > Subscription. To order TestRail Server licenses, you can do so from our website here: https://secure.gurock.com/customers/shop/annual/purchase/
TestRail Enterprise
If you’d like to learn more about SSO, Project Level Administration, Test Case Reviews, or any of the other features on TestRail Enterprise, please email us for a trial or quote via [email protected], or you can use the contact form here: https://secure.gurock.com/customers/support/
Not sure which TestRail plan you’re on? Reach out and we’d be happy to help.
