TestRail reviewed by TestLab²

Alex of TestLab² just published their review of our test management software TestRail. As TestLab² is a provider of software testing services, we were especially curious to find out how TestRail performed in an environment with regularly changing clients, requirements and test projects:

The system proved to be almost perfect for us, both for its’ features and everyday usability. We’re involved in outsourced testing and run lots of small projects, and because of that the vendors’ license policy really fits us: the limit is applied to active system users only, so we deactivate client accounts at the end of the project.

Not bad for our initial release, thanks Alex! You can read the full review on TestLab’s website or you can check out the original review in Russian on habrahabr.ru (a popular Russian IT site, I was told).

TestRail review by TestLab² »

Posted in Gurock Software, TestRail | Leave a comment

Announcing TestRail 1.0

We are really excited to announce the release of our new test case management tool TestRail. After working on TestRail for more than 18 months and running an extensive beta program, it’s great to finally release the product and get it out the door.

We also want to thank all testers who participated in TestRail’s beta test. Without your feature suggestions, ideas and bug reports, the 1.0 certainly wouldn’t be what it is today. We’ve got more than 350 beta signups during the beta and are really grateful for all the feedback and emails we’ve received. Thank you! If you participated in the beta test, please check your email inbox for a small TestRail launch present.

When we released the first beta versions of TestRail last August, we didn’t plan for such a long beta phase. We intended to release a few beta versions and release it in November 2009 at the latest. Based on the feedback we received during the beta, however, it became apparent that TestRail would benefit from a few critical enhancement that we wanted to add before the release (such as test plans and configurations, nested sections including drag&drop, print support and importing/exporting).

The feedback during the beta made us confident that a modern and fresh approach to test management is needed and it was great to see that it was so well received by beta testers. There are still a lot of things that didn’t make it into the 1.0 that we want to see in the future of course, but I believe we’ve created a strong foundation for future enhancements and improvements.

We’ve updated TestRail’s website with new screenshots and a product tour as well as pricing details and licensing information. To celebrate the release of TestRail we are offering special introductory prices, so make sure to evaluate TestRail soon if you are interested in the tool. You can download a free and fully-functional trial copy of TestRail from our website.

As usual, the change log and update instructions for this release can be found in our forum. If you have any question about TestRail or this release, just let us know!

Posted in DelphiFeeds.com, Gurock Software, TestRail | 1 Comment

New Shop and Customer Portal

In preparation of the upcoming launch of our new product TestRail, we have just updated our online shop and customer portal. All existing customer accounts and SmartInspect licenses have been moved to our new system and you can use your existing login details to access your new account.

In case you have any problems accessing your customer account or if there are any issues with placing an order, please let us know.

Please note: Due to technical reasons we unfortunately won’t be able to accept US dollar transactions for up to two weeks. We are sorry for this inconvenience and are working on resolving this issue as quickly as possible. Please either place your order in Euros (your credit card company will automatically convert this to your local currency) or email us to request an US dollar invoice and pay by check/wire transfer.

Posted in Gurock Software | Leave a comment

Happy Holidays and New Year!

We wish all customers, friends and partners happy holidays and a successful new year 2010! We had a great year here at Gurock Software and reached some important milestones such as completing a critical customer project, launching our new website, improving SmartInspect, relaunching DelphiFeeds.com and releasing the first TestRail beta.

Next year will definitely be an interesting (and busy!) one for us and we will start with releasing TestRail in January, moving to a new office (again) and relaunching our customer portal and online shop.

We are looking forward to working with you again next year!

Posted in DelphiFeeds.com, Gurock Software, SmartInspect, TestRail | Tagged | 1 Comment

TestRail Beta 1.0.4 released

We’ve just released beta 1.0.4 of our upcoming test case management tool TestRail. This new version introduces support for improved printing, importing/exporting test cases, and also adds a feature to copy and move cases between suites and projects. This beta version will probably be the last beta drop before we release TestRail 1.0 in early-mid January.

Print support

As mentioned previously on this blog, we wanted to provide a better way to print test suites, test runs and entire test plans to make it easier to generate reports for customers and management. TestRail 1.0.4 now allows you to easily print those reports.


Print support in TestRail

Importing/exporting test cases

The latest TestRail beta also introduces support for importing and exporting test cases. If you are just getting started with TestRail, you might already have existing test cases stored in other documents (such as Excel) or test management tools. Chances are that you want to import those test cases into TestRail and this is now possible with TestRail’s new import XML file format (please take a look at TestRail’s built-in online help to learn more about the exact file format).

You can now also export test cases and results to XML, CSV or Excel. You can use this, for example, to build additional custom reports with Excel or other tools.

Copy/move test cases

Various beta testers asked for a feature to copy or move test cases between suites and projects, so we have just added this capability to the latest beta release. You can either copy entire sections or just specific cases from another suite to the one you are currently working on:


Copy/move test cases in TestRail

We also improved and fixed a bunch of other things in this release and you can view the full change log of the new version in our forum. Let us know if you have any questions about this release or if you notice anything not working correctly.

Posted in Gurock Software, TestRail | Leave a comment

TestRail Roadmap to 1.0

Now that the latest beta of TestRail with support for nested test sections is out, I wanted to take a few minutes to share our plans for version 1.0 and when it’s going to be released. We usually don’t publish release dates or which exact features we will include in new versions of our products (it usually does more harm than good in our experience), but we feel an exception to this rule makes sense for TestRail 1.0 (especially as quite a few users are waiting for the final release to go live).

Before we release TestRail 1.0, we plan to add two more critical features to TestRail (and fix a bunch of smaller issues), namely better print support and the ability to import/export test cases and suites. While printing is theoretically already supported (TestRail comes with a print stylesheet so you can basically print any page with your browser), a better way to print reports of test suites and test runs for customers and management is needed. We still plan to make use of the browsers’ print support for this, but the resulting reports will be more useful and also somewhat customizable as we currently plan it.

Upcoming print support in TestRail
Upcoming print support in TestRail

Adding a feature to import and export test cases and suites has been often requested and we understand that this is a critical feature for all users who already have a large collection of test cases (either managed in a classic test case management tool or with tools such as Excel). We plan to add this capability to TestRail and use a simple XML format for importing and exporting test cases, so that converting existing test cases to our file format will be as easy as possible.

We currently plan to release TestRail 1.0 in early-mid January. However, because the TestRail release date also depends on some other internal efforts at Gurock Software, this is still subject to change.

Posted in Gurock Software, TestRail | 3 Comments

TestRail Beta 1.0.3 has been released

We are happy to announce the release of the latest TestRail Beta 1.0.3 with support for nested sections and section hierarchies, among other improvements and fixes. You can download the latest beta version by visiting your personalized beta link you received when your beta application was approved (just email us in case you lost that email). If you haven’t yet applied for the TestRail beta, you can still participate in the beta test and we are happy about all feedback.


Test suites now support nested subsections

When we originally planned and designed TestRail, we wanted to keep the structure of test plans as simple as possible for various reasons and decided that a flat list of sections and test cases would be ideal. After gathering beta feedback from users over the past months and using TestRail internally ourselves for various projects, we have come to the conclusion that nested sections and section hierarchies are needed to make test suites more scalable. When I say more scalable, I specifically mean that the ability to nest sections makes it possible to organize more test cases in the same suite. It also makes it a lot easier to adjust the suite structure to the needs of individual projects.

One of the reasons we originally implemented flat lists was that tree-based structures are usual hard to use, especially with web applications. So when we decided that we would allow nested sections within test suites, we wanted to make sure that it’s as easy to use as the rest of the application. As you can see in the screenshot below, we accomplished this by adding support for drag&drop, making it really easy to rearrange and copy sections and cases within test suites.


Test cases and sections can be moved and copied via drag&drop

We also improved and fixed a bunch of other things in this release and you can view the full change log of the new version in our forum. Let us know if you have any questions about this release or if you notice anything not working correctly.

Posted in Gurock Software, TestRail | 2 Comments

EurekaLog exception reporting tool for .NET

Our friends at EurekaLab recently released a new .NET edition of their exception reporting tool EurekaLog. While EurekaLog is a popular and well-known tool with Delphi developers, .NET developers haven’t been able to benefit from its powerful exception and bug reporting capabilities until recently. Now that the new version of EurekaLog for .NET is available (which I understand is a managed port of the Delphi version), I’m sure it will be as well received as the original Delphi tool. Please also note that you can easily integrate EurekaLog with SmartInspect, allowing you to benefit from both tools.

Posted in DelphiFeeds.com, Programming | Leave a comment

Aspect-oriented logging for .NET with PostSharp and SmartInspect

This article is part of a series about aspect-oriented logging with SmartInspect, our logging tool for .NET, Java and Delphi. Please see the first article of this series for a complete overview over the available posts.

There are various AOP frameworks available for each of the three platforms SmartInspect supports, so why did we decide to start with PostSharp for the first part of this blog series about apsect-oriented logging? PostSharp has become the de-facto standard AOP framework for .NET and there are many things to like about PostSharp:

  • PostSharp is really easy to use. It integrates nicely with MSBuild (and therefore Visual Studio) and you just need to complete the PostSharp setup and are good to go.
  • It provides different APIs depending on what you are trying to accomplish. You can use a low-level API for injecting snippets of MSIL code for complexer tasks (PostSharp.Core) or a very simple high-level API (PostSharp.Laos) to develop aspects with custom .NET attributes.
  • It’s very well documented. The APIs are fully documented and there are getting started tutorials, videos, a plugin repository and lots more. Can’t get a lot better than that.
  • It’s a compile-time AOP framework. Unlike other tools which inject the AOP code at runtime, it post-processes the assemblies directly after compliation which is more efficient and allows for more flexibility.
  • Last but not least, some SmartInspect customers were already successfully using PostSharp in their applications and told us that they liked the combination of both tools (that’s how we became aware of PostSharp in the first place).

We will be using the simpler Laos API for our SmartInspect PostSharp aspects, as we found it sufficient for all our needs. The resulting library is able to automatically trace method execution, log exceptions and watch changes of field variables. The library comes with three aspects for the different tasks, each implemented as a custom .NET attribute.

Using the SmartInspect aspects

The general procedure of using a PostSharp aspect is to install the PostSharp framework and to add the PostSharp.Public and PostSharp.Laos assembly references to your application. PostSharp injects itself into the MSBuild build process and automatically processes all assemblies that reference PostSharp.

For our logging aspects, you further need to add references to the Gurock.SmartInspect and Gurock.SmartInspect.PostSharp assemblies (the logging library and PostSharp aspects, respectively). The logging library is included in the SmartInspect trial download and the SmartInspect AOP plugin can be downloaded from the SmartInspect resources page.

Tracing method execution

The first aspect I am going to demonstrate is for tracing method execution. This aspect makes it really easy to generate a complete call trace of your application, showing you the entire application flow and letting you inspect stack traces at each execution point. It automatically generates two log messages for each method call: one for entering a method and one when a method exits. You can also log the passed method arguments (when entering a method) and the method result (when exiting a method), respectively.

To use this aspect, simple add the SiTrace custom attribute to your application as follows (note how you can continue to use SmartInspect for complementary log messages, of course):

using System;
using Gurock.SmartInspect;
using Gurock.SmartInspect.PostSharp;

[assembly:SiTrace(SessionName="Main")]

namespace Gurock.SmartInspect.PostSharp.Samples
{
	class Program
	{
		static void Main(string[] args)
		{
			SiAuto.Si.Enabled = true; // Enable logging
			Hello("World");
			Goodbye("World");
		}

		static void Hello(string name)
		{
			SiAuto.Main.LogMessage("Hello, {0}.", name);
		}

		static string Goodbye(string name)
		{
			string msg = String.Format("Goodbye, {0}.", name);
			SiAuto.Main.LogMessage(msg);
			return msg;
		}
	}
}

The attribute can be specified on a per-class/per-method basis or, like in our example, for the entire assembly. You can further specify filters for including/excluding certain modules/classes (called attribute targets in the PostSharp terminology), set aspect priorities (useful with nested aspects) and much more.

Our aspects have a few additional SmartInspect-related options. First, as you can see in the example above, you can set the session name used for sending log messages. If you do not want a hard-coded session name, you can tell the aspect to make use of the logging context and automatically adjust the name based on the current namespace or class. You can do so by setting a so called session policy. For example, to automatically use a session name that equals the name of the current class context, you can use a session policy of SessionPolicy.TypeName and omit the session name:

...
[assembly:SiTrace(SessionPolicy=SessionPolicy.TypeName)]
...

There are additional options for including the method arguments (IncludeArguments) and method return value (IncludeReturnValue). With both options enabled, the resulting log looks as follows in the SmartInspect Console (SmartInspect’s viewer application). Note how the automatic enter/exit messages are mixed with the manual log messages:


The SiTrace sample log shown in the SmartInspect Console

The SiTrace sample log shown in the SmartInspect Console

Logging exceptions

Another typical use-case for logging is recording all occurring exceptions and this is usually done by adding log calls to your exception handlers. Adding log statements to all your exception handlers can get tedious and error-prone though (e.g., you could forget to rethrow the exception after logging it), so we’ve added an aspect to automate this task for you. To use it, just add the SiException attribute to your code:

using System;
using Gurock.SmartInspect;
using Gurock.SmartInspect.PostSharp;

...

[assembly: SiException(SessionName="Main")]

...

		static void Hello(string name)
		{
			SiAuto.Main.LogMessage("Hello, {0}.", name);
			throw new Exception("This is a test exception");
		}

...

Note how the exception is now automatically included in the resulting log (within the expected method context):


The SiException sample log shown in the SmartInspect Console

The SiException sample log shown in the SmartInspect Console

Watching field values

Arguably the most advanced aspect in this library is the SiField attribute. This attribute lets you automatically record any changes to values of object or class fields. The aspect gets notified whenever the value of a field changes and then logs the name of the field along with its old and new value. You can configure whether the aspect should send the name and values as normal log messages or as SmartInspect watches (for showing the latest values of a variable in the watches toolbox or for displaying its history as a chart in the watches graph). By default, both watches and normal messages are logged.

Let’s have a look at the following example which demonstrates SiField in practice. Note how we use SiField as an attribute for the entire assembly again. We could also limit it to individual classes or fields, either by writing the attribute above the classes/fields we want to inspect or by specifying filters/attribute targets for the global attribute.

using System;
using Gurock.SmartInspect;
using Gurock.SmartInspect.PostSharp;

[assembly: SiField(SessionName = "Main")]

namespace Gurock.SmartInspect.PostSharp.Samples
{
	class Program
	{
		public static int FIELD;

		static void Main(string[] args)
		{
			SiAuto.Si.Enabled = true; // Enable logging
			Counting();
		}

		static void Counting()
		{
			Random r = new Random();

			for (int i = 0; i < 10; i++)
			{
				FIELD = r.Next(100);
				Thread.Sleep(1000);
			}
		}
	}
}

And here’s the generated log in the SmartInspect Console:


The SiField sample log shown in the SmartInspect Console

The SiField sample log shown in the SmartInspect Console

As you can see, every field change is automatically logged and shown in the Console. The watches toolbox in the lower left of the Console displays the latest value of FIELD in addition to the normal log messages in the main view. Also note the watches graph on the right that displays the field’s history as a chart (with its values over time). I believe an upcoming version of PostSharp will also be able to watch properties and we will update our library accordingly when this feature is available.

The nice thing about these aspects is that they can be freely combined as you like. You can mix and match the aspects as needed. If you plan to use our aspects and library, I also recommend taking a look at the PostSharp documentation for more details about aspects, AOP and PostSharp in general as well as how to configure PostSharp for more complex scenarios.

Feel free to send me an email or leave a comment on this blog in case you have any questions about this plugin.

Posted in Gurock Software, Programming, SmartInspect | Tagged | Leave a comment

Using a custom Log4net appender with SmartInspect

A popular logging framework with .NET developers is log4net, a text-only logging library that has been ported to .NET from the well-known Java logging library log4j. Similar to how I already outlined in our recent log4j blog posting, we regularly get emails from users who are interested in switching from log4net to SmartInspect. They want to take advantage of SmartInspect’s rich logging capabilities, our Console (SmartInspect’s monitoring and analysis tool) and all the unique protocol options available in SmartInspect (such as log file encryption, the memory protocol, backlog queues etc).

Switching to a new logging library is not an easy task if you have to replace all existing logging calls with new ones. Luckily, you can extend log4net with custom appenders that allow you to route the log messages to a new destination, for example, to SmartInspect. We have developed such an appender (two, actually) and demonstrate how to use them in a new article:

Using a Custom Log4net Appender with SmartInspect

We hope this will make it a lot more attractive for .NET developers to port existing projects that make use of the classic log4net log framework to SmartInspect.

Posted in Gurock Software, SmartInspect | Leave a comment