Open source teams already use various tools to manage their software development process such as bug trackers, project management tools and source control. Teams can also benefit from a test management software to manage, organize and execute their tests. TestRail allows contributors to participate in projects’ software testing efforts and even enables non-technical users to contribute to projects.

We decided to make free licenses of our test management software available to open source teams as a way to give back to the community. Gurock Software is using various open source projects to power the infrastructure and development process so this makes a lot of sense for us. Interested open source teams can contact us or learn more about the offer on the dedicated web page.

Learn more: Free TestRail licenses for open source projects

Additionally, we wanted to separate DelphiFeeds.com and our blog from our website for performance and security reasons, so we also put those sites on different machines. The good news is that our websites should become a lot faster because of the server move, especially DelphiFeeds.com. If you notice any problems, we would appreciate a quick email.

The new aspect library now also supports tracing and logging of property values as PostSharp introduced support for this functionality with its new version. If you have any questions about the new version or if you find any problems, please let us know!

SmartInspect for PostSharp

However, after discussing ideas about a simple API with a customer, the idea for the TestRail Mini API was born. This optional PHP script can be added to a TestRail installation and allows you to submit test changes and results via simple HTTP requests.

Until we have a complete API and support for automated testing tools directly integrated in TestRail, the mini API is a great way to submit your test results. You can download the API and learn more about its usage on the project site:

TestRail Mini API

So as of today, we have our own project hosting website at code.gurock.com. We will use this website to host product-related projects and we invite you to contribute. We are also happy to host your projects if they are related to SmartInspect or TestRail, so feel free to contact us if you are interested in this.

For now there are just two projects on the site: PostSharp for SmartInspect (which will get a major rewrite soon, stay tuned) and a new TestRail mini API (more about this in a later posting). We plan to add other projects to the site in the future and open more of our code for contributions from the community. Enjoy!

code.gurock.com

Custom Fields

When we initially designed TestRail, we decided to only include the most basic and often used fields in the user interface to make it easier to enter cases and test results. It became clear early on that this was a good design decision. Users didn’t really need all those extra fields some of our competitors include that make their products cluttered and hard to use.

However, although all the basic fields we offer are used by most customers, users emailed us that they had slightly different needs for their particular projects. They didn’t ask us to make TestRail’s user interface complicated, mind you, but adding just that one field would make TestRail just perfect for their needs.

So instead of adding all those extra fields customers asked for to TestRail, the idea for custom fields was born. Custom fields allow you to add your own fields to TestRail’s test case and test result forms, making it easy to customize TestRail for your projects.

Custom field configuration in TestRail’s administration area

To configure custom fields, a new Custom Fields section was added to TestRail’s administration area. Fields for test cases and test results are configured separately. TestRail supports various custom field types including dropdowns, dates, checkboxes, text fields and others.

TestRail also allows you to specify different custom field options for your various projects. For example, let’s say you added a Project Area field to TestRail so that users could select the project area a case is associated with (such as Installer, Backend, API etc). You only need to define this field once and can specify a different list of areas for each of your projects. TestRail’s way of separating custom fields and their project assignments/options is a powerful concept that makes this feature very flexible.

Assigning custom fields to projects with varying options

Some of the built-in types were converted to custom fields, enabling you to change or even remove and replace those fields. For test cases specifically, we have converted the Preconditions, Steps and Expected Results fields to custom fields. TestRail generally differentiates between small (such as dropdowns or checkboxes) and large fields (such as text). Depending on the field type, test case fields are displayed at the top or bottom of the form.

Cases and custom field positions

Similar to test cases, the position of custom fields in the test result dialog depends on the actual field type. Large fields are displayed on the left side of the dialog, small fields are displayed in the sidebar.

Test results and custom field positions

Test Steps

Prior to TestRail 1.3, the only way to enter test case steps was via a simple text box. While we really like this clean and simple approach and most customers were happy with this solution, some users needed a more structured way to manage their test steps. So with the new TestRail version we are introducing an optional way to enter individual test steps separately.

Entering separate test case steps

If the new alternative way of managing test case steps is a good fit for your needs, chances are good that you also want to enter separate test results for your steps. Fortunately, TestRail 1.3 makes this easy and provides a way to mark individual test steps as Passed or Failed.

Test steps in the Add Test Result dialog

Structured test steps are an optional feature. By default, TestRail 1.3 comes with the ‘old’ way of managing test steps. To use the new alternative approach for one or all of your projects, just configure the Steps and Step Results custom fields.

Other Improvements

TestRail 1.3 also comes with other improvements, changes and bug fixes. For example, the updated Add Result Dialog can now be resized. TestRail automatically remembers the dialog size, making it more convenient to work with larger monitors. We also improved the test suite page to make it easier to add multiple test cases at once by keeping the text box visible until you press ESC.

Improved inline adding of test cases

Other improvements include the option to disable Markdown formatting for text fields (such as Prerequisites and Expected Results) and new Select All links in the Select Configuration dialog.

Getting the new version

Existing customers can download the new version from our customer portal. New users can download TestRail or request a hosted trial of TestRail from our website. To view the complete list of changes for TestRail 1.3, please see our announcement forum. Please refer to TestRail’s Admin Manual on how to upgrade an existing installation.

The Windows Communication Foundation (WCF) is Microsoft’s approach to service-oriented architectures and allows you to easily built client/server and distributed applications with .NET. SmartInspect can be used to debug applications that make use of WCF and this article (and the attached demo project) explains how to do this. The demo application is based on Microsoft’s WCF tutorial, so if you haven’t worked with WCF before, the tutorial is a good way to start.

The service application

The service application that is part of the demo project exposes a simple service that offers various calculation methods (add, subtract, multiply, and divide). We use SmartInspect to log the method execution, parameters and results:

class CalculatorService : ICalculator
{
	public double Add(double n1, double n2)
	{
		SiAuto.Main.EnterMethod(this, "Add({0},{1})",
			new object[] {n1, n2});
		double result = n1 + n2;
		SiAuto.Main.LogValue("Result", result);
		SiAuto.Main.LeaveMethod(this, "Add");
		return result;
	}

	// [..]
}

The client application

The client application connects to the service and uses the exposed methods to calculate various results. The client application is also instrumented with SmartInspect log calls and logs the calculation results returned from service:

static void Calculate(CalculatorClient client)
{
	SiAuto.Main.EnterMethod("Program.Calculate");

	// Call the Add service operation
	double value1 = 100.00D;
	double value2 = 15.99D;
	double result = client.Add(value1, value2);
	SiAuto.Main.LogMessage("Add({0},{1}) = {2}", value1, value2, result);

	// [..]

	SiAuto.Main.LeaveMethod("Program.Calculate");
}

Debugging with the SmartInspect Console

When you launch both demo applications, the SmartInspect Console automatically receives the log messages via named-pipes (make sure to start Visual Studio as administrator, as this is required by WCF to register the service). You can also redirect the logging data to log files or other protocols by setting the connection string accordingly.

The SmartInspect Console makes it easy to see the log messages and data in context or to separate the log messages of the client and service. The Console also helps with tracking the involved threads and processes (e.g. you can see in the screenshot that .NET starts and recycles two threads to serve the four service calls).

Log messages of the client and service in the SmartInspect Console

Tracing WCF

WCF also has some tracing and logging capabilities of its own. This allows you to trace the messages and packets WCF sends and receives. Inspecting the exchanged messages is especially useful if you need to debug interoperability issues with other platforms and frameworks. WCF uses .NET’s built-in tracing API to log messages and packets.

Fortunately, SmartInspect makes it easy to receive .NET debug messages as it comes with a custom trace listener. To redirect WCF diagnostic messages to SmartInspect, simply add the Gurock.SmartInspect.DebugTrace assembly as a reference to your project and add the following code to the App.config (the demo projects already come with all the required bits pre-configured; you just need to uncomment the config settings):

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.diagnostics>
    <sources>
      <source name="System.ServiceModel.MessageLogging" >
        <listeners>
          <clear />
          <add name="SmartInspect"
              type="Gurock.SmartInspect.DebugTrace.SmartInspectTraceListener,        Gurock.SmartInspect.DebugTrace" />
        </listeners>
      </source>
    </sources>
  </system.diagnostics>
  <system.serviceModel>
      <diagnostics>
        <messageLogging
             logEntireMessage="true"
             logMalformedMessages="true"
             logMessagesAtServiceLevel="true"
             logMessagesAtTransportLevel="true"
             maxMessagesToLog="3000"
             maxSizeOfMessageToLog="2000"/>
      </diagnostics>
    </system.serviceModel>
</configuration>

The full demo application will be included in the next SmartInspect release, but you can also download a copy of the demo project from this site in the meantime.

I will also document some of the more interesting demo applications here on this blog and I’m going to start with an article about WCF logging and tracing. If you have any suggestions for new SmartInspect demos you would like to see, just let us know. Enjoy!

Michael Baarz recently emailed us as he was a looking for a modern .NET logging tool for his company’s complex telecommunication and PBX software. Michael was about to redesign his company’s flagship product and was planning to use Microsoft’s dependency injection framework Unity.

He told us that he really enjoyed working with SmartInspect and because Unity also supports some aspect oriented programming features, Michael decided to write an unofficial library to integrate SmartInspect with Unity so he could benefit from this useful combination. This posting introduces Michael’s library and will explain how to get started with the extension to integrate and benefit from SmartInspect and Unity.

Unity, a dependency injection container

Unity is Microsoft’s dependency injection container and framework and is part of their Patterns & Practices initiative. If you are new to dependency injection in general, I recommend taking a look at this article by Martin Fowler that explains the basic concepts and benefits. Besides common dependency injection features, Unity also supports aspect oriented programming by allowing you to intercept and wrap method calls.

The big difference between dependency injection (DI) frameworks and classic AOP tools such as PostSharp is that DI frameworks are configured at run-time, whereas PostSharp injects and builds its aspects at compile time. Both approaches have their pros and cons, so it really depends on the circumstances which one you should use (DI frameworks are usually more flexible as you can change the behavior at run-time, whereas ‘classic’ AOP tools support more features and are usually faster).

Using the SmartInspect Unity Interception Extension

Michael’s library to integrate SmartInspect and Unity is called the SmartInspect Unity Interception Extension and can be downloaded from CodePlex. To get started with the library, either check out the source code via Subversion or download one of the releases (there wasn’t a release at the time of this writing yet).

To use the library in your solution, you first need to add a few assembly references to your projects. First of all, you need to reference the Microsoft.Practices.Unity and Microsoft.Practices.Unity.Interception assemblies; these are part of Unity and are automatically installed and registered when you download and install Unity. The next assembly reference you need to add is Gurock.SmartInspect, as this is required to use SmartInspect logging. Last but not least you need to add Michael’s SmartInspect.Extensions.Unity assembly as a reference to your project, either by adding a reference to the compiled DLL or by adding the project to your solution and using a project reference.

Tracing method execution

One of the more useful aspects (no pun intended) of AOP is the capability to easily trace method execution, so this is the first thing I’m going to demonstrate. Unity is a dependency injection framework so we will need to properly register and instantiate all classes we want to intercept for logging. The first thing we need to do is to define the interface for the class we want to instantiate through Unity. Because one of the design goals of DI is that we can easily replace the actual implementation for an interface, using interfaces is mandatory here. For our Hello, World example, I’ve defined an interface and implementation class:

interface IHello
{
	int Count { get; set; }
	void SayHello(string name);
	string SayGoodbye(string name);
	void RaiseException(string name);
}

The actual implementation already makes use of SmartInspect to log various messages:

class Hello : IHello
{
	public int Count { get; set; }

	public void SayHello(string name)
	{
		SiAuto.Main.LogMessage("Hello, {0}.", name);
	}

	public string SayGoodbye(string name)
	{
		string msg = String.Format("Goodbye, {0}.", name);
		SiAuto.Main.LogMessage(msg);
		return msg;
	}

	public void RaiseException(string name)
	{
		SiAuto.Main.LogMessage("Hello, {0}.", name);
		throw new Exception("This is a test exception");
	}
}

Now to intercept and trace the method calls, we first need to make sure that we register and instantiate the interface and implementation with Unity. To do this, we use an instance of UnityContainer and add and configure the Interception extension. Here’s the full Main method of our example program:

static void Main(string[] args)
{
	// Enable SmartInspect logging
	SiAuto.Si.Enabled = true;

	// Create the Unity dependency injection container,
	// register the Interception extension and register our
	// our example class
	IUnityContainer container = new UnityContainer();
	container
		.AddNewExtension<Interception>()
		.RegisterType<IHello, Hello>();

	// Now tell Unity how to intercept IHello method calls
	container.Configure<Interception>()
		.SetInterceptorFor<IHello>(new TransparentProxyInterceptor());

	// Instead of creating our Hello class directly, we ask
	// Unity to resolve the dependency for the IHello interface
	IHello hello = container.Resolve<IHello>();

	// Call The IHello methods and assign property values
	hello.SayHello("World");
	hello.SayGoodbye("World");

	Random r = new Random();
	for (int i = 0; i < 10; i++)
	{
		hello.Count = r.Next(100);
		Thread.Sleep(1000);
	}

	hello.RaiseException("Hello World");
}

Now to actually use the the SmartInspect Unity extension and trace method calls, we just need to add a single attribute to the interface, namely the [SiMethodCall] attribute (which can be found in the SmartInspect.Extensions.Unity namespace):

[SiMethodCall]
interface IHello
{
	// [..]
}

The attributes come with various properties to influence the behavior of the log calls, such as changing the log level, adding name prefixes/suffixes and so on. For a complete list of properties just see the Intellisense help within Visual Studio. Once we launch the example application, we see the following output in the SmartInspect Console (also take a look at the viewer toolbox at the bottom right of the window, it shows the method arguments of the selected log entry):

Log messages and method tracing in the SmartInspect Console

Logging exceptions

Logging occurred exceptions is another useful thing you can automate with AOP. The SmartInspect Unity extension comes with its own attribute to log exceptions, namely the [SiException] attribute. The attribute doesn’t handle the exceptions (it just logs them) so it doesn’t change the behavior of your code. To use the attribute, just specify it for an interface or method like this:

[SiException(LogArgumentsToViewer = true)]
interface IHello
{
	// [..]
}

The resulting log messages in the SmartInspect Console look like this:

Logged exception in the SmartInspect Console

Watching property values

SmartInspect allows you to easily log and watch field and property values. The SmartInspect Unity extension comes with an attribute to automatically log changes of properties as they occur. To benefit from this, simply add the [SiWatch] attribute to the set method of your properties:

interface IHello
{
	int Count { get; [SiWatch]set; }
}

This blog posting is intended to give you an overview of the capabilities the SmartInspect Unity extension library provides, but it cannot explain all the useful options and features that the library comes with. So to get started with the library and learn about all its capabilities, I recommend downloading it and start playing with it.

Please also note that the SmartInspect Unity extension introduced here is not a Gurock Software product and is not official supported by us. To send feedback and to contribute to the project, please take a look at the CodePlex project page. Thanks Michael for writing the library and for contributing to the SmartInspect ecosystem!

“Around a year ago we desperately needed a name for the new test management software we had been working on. We aren’t very good with names. In fact, we used a codename for the project until the very last minute, so that we didn’t have to come up with the product name earlier. Still, even with many months to think about a name, it was difficult to find one that we liked.”

Read more in a guest posting I wrote for the Successful Software blog:
How to find a great software product name