We are excited to announce the availability of TestRail 3.0, a new version of our test management tool. When we started to design and plan this release, we reviewed tons of feedback we received for TestRail over the previous months, went through internal notes and our feedback system as well as discussed future roadmap options with various teams (both new and existing TestRail customers). It quickly became clear that there were a few things many of our customers would benefit from, namely better reporting capabilities, more native defect integrations as well as a redesigned API.
So for the past 6 months we have been working on the new version to implement all this and more. TestRail 3.0 features an awesome reporting engine to generate new built-in reports, to make it easy to schedule and share statistics as well as allowing teams to develop their own custom reports. The new release also features a redesigned and improved API to automate various things in TestRail and to integrate your automated tests and tools. TestRail 3.0 also comes with improved integrations for tools such as Rally, Assembla, Bitbucket, GitHub, JIRA and OnTime. We also included various other improvements and new features in this new version and you can review all changes below.
New reporting section
TestRail’s built-in reports have always helped teams track the status and progress of their software tests. But it became clear that teams would benefit from more comprehensive reports to generate snapshots of project statistics, to trace the coverage of references, test results and defects, as well as having reports to compare test results across runs or to compare the workload of testers. TestRail’s new reporting section makes all this possible and provides some of the most flexible reporting capabilities available in test management tools today.
Besides shipping various new built-in reports and reporting options as explained below, it’s now also possible to develop your own custom reports for the new reporting engine. We specifically designed the new system in a way that you can easily extend it with custom code. Instead of bundling a limited reporting system with TestRail (which wouldn’t allow us to generate many of the advanced reports we now include), TestRail’s code-based reporting engine is much more flexible and works similar to how you can already customize TestRail’s defect plugins.
New built-in reports
One of the major new features of TestRail 3.0 is the addition of new built-in reports. Based on customer feedback we added more than a dozen new reports to generate useful statistics of your test results, to track the coverage of test cases, defects and references, as well as reports to compare test results and runs over time and various other key metrics. All reports are highly customizable so you can choose the scope, time frame and level of detail you want to include.
Summary reports
The summary reports allow you to generate detailed statistics of your testing efforts for test runs, test plans, milestones and entire projects. The summary reports are especially useful to share your progress with all team members, management or customers. They can also be used to generate snapshots of your test status and activity at various points of your projects for archival and auditing purposes.
Coverage reports
TestRail allows you to link test results to bug reports in your defect tracking tool and to link test cases to external requirements. TestRail 3.0 now comes with useful coverage and traceability reports to easily get an overview of your requirement coverage, see all bug reports for your test cases at a glance and get a detailed matrix of the relationships between requirements, test cases and bug reports.
Comparison reports
Have you ever wanted to see how your tests perform and change between releases, configurations and runs? The new comparison reports for test cases and references make this extremely easy and allow you to see results of multiple test results side-by-side. In addition to this, the new comparison reports also allow you to track the latest status of your tests across all selected runs.
Additional reports
TestRail also comes with additional useful reports to e.g. easily track the workload of your team and to see the distribution of your test cases for various attributes such as test types or priority.
TestRail’s new reports also make it easy to identify test cases that had a high number of failures over time or to identify problematic functional areas of projects based on other test statuses.
Schedule and share reports
Adding more useful built-in reports that help our customers answer important questions about their testing efforts and help teams make informed decisions during development was a major design goal of TestRail 3.0. Making it as easy and flexible as possible to run, schedule and share said reports is equally important, so we added a bunch of useful options for this. TestRail now allows you to generate new reports on a pre-defined schedule so you can, for example, automatically generate new reports once a day or once a week so your team or project stakeholders get fresh reports automatically.
TestRail’s new version also makes it extremely easy to share reports with both TestRail users as well as external stakeholders such as customers. Besides hosting and displaying reports inside TestRail, you can now also forward reports as HTML attachments via email or download reports to archive them. Combined with the above mentioned scheduling feature you can keep your entire company updated on your latest testing efforts automatically and run your reporting on autopilot, awesome!
New REST-like API
TestRail’s existing Mini API has always been useful to integrate automated tests and to automate various tasks such as starting test runs, submitting test results and creating new test cases programmatically from your scripts. However, because of a few design choices we made with our earlier API it was difficult to add more advanced API capabilities, so we designed and built a new REST-like API that we are now shipping with TestRail 3.0.
The new API addresses various issues and will also allow us to add more API methods and capabilities in future updates:
- API calls now use JSON universally for both requests and responses
- The API is now part of TestRail (no separate download/activation needed anymore)
- You can now authenticate against the API with your standard TestRail user login credentials. There’s no need to link the API to a single user account anymore and you don’t need to configure any API keys
- The new API supports various new API methods (such as methods to manage projects, milestones, test plans, change test case selections and retrieve information about custom fields)
The new API is part of all TestRail 3.0 instances and can be enabled under Administration > Site Settings > API. You can learn more about the new capabilities in the API documentation and also refer to our migration guide if you’ve already been using our previous API. We also updated our previous Mini API to add various new API methods and we will continue to support the old API for now. Please use our new API for new projects as we will deprecate the old API in future TestRail updates.
Redesigned charts
We included various real-time charts in TestRail from the very first version and making your test statistics and activities easy to understand and to follow has always been a major priority for us. While the charts in TestRail have always been useful, there were some limitations that the new TestRail 3.0 release addresses. For the new version we migrated all charts to a new system so that TestRail’s charts are easier to print, can be accessed from mobile devices (such as your phone or iPad) and don’t require Flash anymore.
We use the new charting system for both the real-time charts on the status, activity and progress pages, as well as in all new reports we added to TestRail. Besides making it easier to view, print and share charts with the new version, you can now also change charting options such as the time frame of any built-in activity charts.
New defect integration
Integrating with existing defect and issue tracking tools has always been one of TestRail’s major features and almost all of our customers integrate TestRail with additional tools. We are constantly working on adding more defect plugins and we are happy to announce new and updated defect plugins as part of our TestRail 3.0 release.
The new defect plugins allow you to directly push bug reports to tools, look up the status of issues from TestRail and link test results to bug reports. We now include the following new defect plugins in TestRail:
TestRail 3.0 also includes updated defect plugins for OnTime as well as JIRA. For OnTime we now include a defect plugin for their new REST API, which makes it much easier to configure and use the integration. We also included an updated version of our recently introduced JIRA REST defect plugin that addresses some previous limitations:
Other improvements
TestRail 3.0 also comes with various other improvements and new features. For example, we improved various parts of TestRail’s user experience and navigation and made it easier to add images and screenshots to test steps, added section names to breadcrumbs, added pagination support for lists such as completed test runs, activities and milestones, as well as included improved milestone grouping.
TestRail’s new version now also includes support for the latest versions of Microsoft’s web stack, such as SQL Server 2012, Internet Explorer 10 as well as Windows Server 2012 (you can of course also install TestRail on a Linux server or use our SaaS/cloud edition). We also improved TestRail’s background task for faster processing of queued tasks such as email notifications & report generation and optimized various built-in reports to use less resources and scale better for large projects.
We also made TestRail easier to customize with the new version. For example, our unique UI Script feature supports additional context information and JavaScript hooks now. Based on popular demand we now also support changing the names of the built-in statuses under Administration > Customizations.
Getting the new version
Want to upgrade to and benefit from TestRail’s latest version? Upgrading TestRail is very easy and we’ve included all the necessarily details below, depending on the edition you use:
- TestRail Hosted: your account has already been updated!
- TestRail Download (licensed): you can download the latest version or renew your support plan from your customer portal account. If you have been using TestRail’s existing API, it needs to be updated as well.
- TestRail Download (trial): please contact us to download the latest version.
- New user: want to try TestRail? Get a free trial.
You can read the full change log to learn more about all new features, improvements and bug fixes included in TestRail 3.0. If you have any questions or feedback about the new version, we look forward to hear from you. We hope you enjoy the new version as much as we enjoyed building it!
