Our PostSharp aspect library has a new home: SmartInspect for PostSharp

There are various AOP frameworks available for each of the three platforms SmartInspect supports, so why did we decide to start with PostSharp for the first part of this blog series about apsect-oriented logging? PostSharp has become the de-facto standard AOP framework for .NET and there are many things to like about PostSharp:

• PostSharp is really easy to use. It integrates nicely with MSBuild (and therefore Visual Studio) and you just need to complete the PostSharp setup and are good to go.
• It provides different APIs depending on what you are trying to accomplish. You can use a low-level API for injecting snippets of MSIL code for complexer tasks (PostSharp.Core) or a very simple high-level API (PostSharp.Laos) to develop aspects with custom .NET attributes.
• It’s very well documented. The APIs are fully documented and there are getting started tutorials, videos, a plugin repository and lots more. Can’t get a lot better than that.
• It’s a compile-time AOP framework. Unlike other tools which inject the AOP code at runtime, it post-processes the assemblies directly after compliation which is more efficient and allows for more flexibility.
• Last but not least, some SmartInspect customers were already successfully using PostSharp in their applications and told us that they liked the combination of both tools (that’s how we became aware of PostSharp in the first place).

We will be using the simpler Laos API for our SmartInspect PostSharp aspects, as we found it sufficient for all our needs. The resulting library is able to automatically trace method execution, log exceptions and watch changes of field variables. The library comes with three aspects for the different tasks, each implemented as a custom .NET attribute.

Using the SmartInspect aspects

The general procedure of using a PostSharp aspect is to install the PostSharp framework and to add the PostSharp.Public and PostSharp.Laos assembly references to your application. PostSharp injects itself into the MSBuild build process and automatically processes all assemblies that reference PostSharp.

For our logging aspects, you further need to add references to the Gurock.SmartInspect and Gurock.SmartInspect.PostSharp assemblies (the logging library and PostSharp aspects, respectively). The logging library is included in the SmartInspect trial download and the SmartInspect AOP plugin can be downloaded from the SmartInspect resources page.

Tracing method execution

The first aspect I am going to demonstrate is for tracing method execution. This aspect makes it really easy to generate a complete call trace of your application, showing you the entire application flow and letting you inspect stack traces at each execution point. It automatically generates two log messages for each method call: one for entering a method and one when a method exits. You can also log the passed method arguments (when entering a method) and the method result (when exiting a method), respectively.

To use this aspect, simple add the SiTrace custom attribute to your application as follows (note how you can continue to use SmartInspect for complementary log messages, of course):

[sourcecode language="csharp"]
using System;
using Gurock.SmartInspect;
using Gurock.SmartInspect.PostSharp;

[assembly:SiTrace(SessionName="Main")]

namespace Gurock.SmartInspect.PostSharp.Samples
{
class Program
{
static void Main(string[] args)
{
SiAuto.Si.Enabled = true; // Enable logging
Hello("World");
Goodbye("World");
}

static void Hello(string name)
{
SiAuto.Main.LogMessage("Hello, {0}.", name);
}

static string Goodbye(string name)
{
string msg = String.Format("Goodbye, {0}.", name);
SiAuto.Main.LogMessage(msg);
return msg;
}
}
}
[/sourcecode]

The attribute can be specified on a per-class/per-method basis or, like in our example, for the entire assembly. You can further specify filters for including/excluding certain modules/classes (called attribute targets in the PostSharp terminology), set aspect priorities (useful with nested aspects) and much more.

Our aspects have a few additional SmartInspect-related options. First, as you can see in the example above, you can set the session name used for sending log messages. If you do not want a hard-coded session name, you can tell the aspect to make use of the logging context and automatically adjust the name based on the current namespace or class. You can do so by setting a so called session policy. For example, to automatically use a session name that equals the name of the current class context, you can use a session policy of SessionPolicy.TypeName and omit the session name:

[sourcecode language="csharp"]
…
[assembly:SiTrace(SessionPolicy=SessionPolicy.TypeName)]
…
[/sourcecode]

There are additional options for including the method arguments (IncludeArguments) and method return value (IncludeReturnValue). With both options enabled, the resulting log looks as follows in the SmartInspect Console (SmartInspect’s viewer application). Note how the automatic enter/exit messages are mixed with the manual log messages:

The SiTrace sample log shown in the SmartInspect Console

Logging exceptions

Another typical use-case for logging is recording all occurring exceptions and this is usually done by adding log calls to your exception handlers. Adding log statements to all your exception handlers can get tedious and error-prone though (e.g., you could forget to rethrow the exception after logging it), so we’ve added an aspect to automate this task for you. To use it, just add the SiException attribute to your code:

[sourcecode language="csharp"]
using System;
using Gurock.SmartInspect;
using Gurock.SmartInspect.PostSharp;

…

[assembly: SiException(SessionName="Main")]

…

static void Hello(string name)
{
SiAuto.Main.LogMessage("Hello, {0}.", name);
throw new Exception("This is a test exception");
}

…
[/sourcecode]

Note how the exception is now automatically included in the resulting log (within the expected method context):

The SiException sample log shown in the SmartInspect Console

Watching field values

Arguably the most advanced aspect in this library is the SiField attribute. This attribute lets you automatically record any changes to values of object or class fields. The aspect gets notified whenever the value of a field changes and then logs the name of the field along with its old and new value. You can configure whether the aspect should send the name and values as normal log messages or as SmartInspect watches (for showing the latest values of a variable in the watches toolbox or for displaying its history as a chart in the watches graph). By default, both watches and normal messages are logged.

Let’s have a look at the following example which demonstrates SiField in practice. Note how we use SiField as an attribute for the entire assembly again. We could also limit it to individual classes or fields, either by writing the attribute above the classes/fields we want to inspect or by specifying filters/attribute targets for the global attribute.

[sourcecode language="csharp"]
using System;
using Gurock.SmartInspect;
using Gurock.SmartInspect.PostSharp;

[assembly: SiField(SessionName = "Main")]

namespace Gurock.SmartInspect.PostSharp.Samples
{
class Program
{
public static int FIELD;

static void Main(string[] args)
{
SiAuto.Si.Enabled = true; // Enable logging
Counting();
}

static void Counting()
{
Random r = new Random();

for (int i = 0; i < 10; i++)
{
FIELD = r.Next(100);
Thread.Sleep(1000);
}
}
}
}
[/sourcecode]

And here’s the generated log in the SmartInspect Console:

The SiField sample log shown in the SmartInspect Console

As you can see, every field change is automatically logged and shown in the Console. The watches toolbox in the lower left of the Console displays the latest value of FIELD in addition to the normal log messages in the main view. Also note the watches graph on the right that displays the field’s history as a chart (with its values over time). I believe an upcoming version of PostSharp will also be able to watch properties and we will update our library accordingly when this feature is available.

The nice thing about these aspects is that they can be freely combined as you like. You can mix and match the aspects as needed. If you plan to use our aspects and library, I also recommend taking a look at the PostSharp documentation for more details about aspects, AOP and PostSharp in general as well as how to configure PostSharp for more complex scenarios.

Feel free to send me an email or leave a comment on this blog in case you have any questions about this plugin.

Another issue with logging is that adding all those log calls takes time and getting a good log coverage can get tedious when instrumenting a large existing application. That’s not necessarily the case if you add logging from the very beginning of development, but let’s face it, most applications have already reached a mature state before you think of adding diagnostics and logging features. Adding such features often comes as an afterthought (which it shouldn’t, ideally), so when you or a customer experience a problem which you cannot easily reproduce, you are instrumenting your application or parts of it and that isn’t any fun at all. We usually recommend instrumenting the most important parts of an application first and then building from there, but it’s still, well, time-consuming.

Fortunately, there are approaches and techniques to solve both of these problems, with one of the most promising being aspect-oriented programming (or AOP for short):

“Aspect-oriented programming (AOP) is a programming paradigm that increases modularity by enabling improved separation of concerns. This entails breaking down a program into distinct parts (so-called concerns, cohesive areas of functionality). All programming paradigms support some level of grouping and encapsulation of concerns into separate, independent entities by providing abstractions (e.g. procedures, modules, classes, methods) that can be used to implement, abstract and compose these concerns. But some concerns defy these forms of implementation and are called crosscutting concerns because they “cut across” multiple abstractions in a program.”

It’s been several years since I first heard about AOP. Back then, AOP was being marketed as one of the next big things in programming (okay, it’s been several, several years now). Supporters of AOP claimed it would change programming and software development as much as object-oriented programming did 20 years ago. We now know that it hasn’t and probably never will to that extend, but it’s for sure a nice addition to the developer’s toolbox.

So, what’s so special about AOP? The most beautiful aspect of AOP is that it improves the modularity of your application by putting parts that logically belong together in one central location. You can implement security concepts, caching, transactions, design by contract and of course a complete logging solution without touching any of your existing classes or methods. This means there’s no need for repeated boiler plate code for checking method arguments for null values, no repetitions of the same begin/commit/rollback transaction code or for adding exception logging code to every exception handler. You simply tell the AOP framework which parts of your application should make use of which features (also called aspects in AOP terminology) and the AOP framework does all the work wiring those aspects to your application modules.

In the coming months, we will take a look at some of the available AOP frameworks for the different platforms supported by SmartInspect here on this blog and show how to build a sophisticated logging layer with these frameworks and our own logging tool SmartInspect. Stay tuned!

Articles published so far:

• Aspect-oriented logging for .NET with PostSharp and SmartInspect
• Aspect-oriented logging for .NET with Unity and SmartInspect

Unfortunately, Delphi never had good (built-in) run-time support for stack traces. The features for stack traces during debugging in the IDE are/were okay, but there were nothing in the language or framework which helped you to find out programmatically where an exception occurred at run-time and, more importantly, how it got passed to your exception handler (besides the original exception address, maybe). So, I was happy to see that Delphi 2009 finally introduced a new StackTrace property which, I hoped, would return a full-blown stack trace when you caught an exception.

The initial happiness soon wore off when I realized that the StackTrace property was really just a placeholder to return a stack trace from a possible stack trace provider rather than a real stack trace implementation. So, without such a provider (and there’s none that comes directly with Delphi), there is still no way to get a stack trace for your exceptions. Though a bit disappointing, the good thing is that there’s now finally a standardized way to get a stack trace, even if it’s not implemented by default.

Exception reporting tools such as Eurekalog or madExcept or debug helpers such as the JclDebug unit can register themselves as providers and use their engines to return a stack trace when an exception is raised. I’ve built a small unit which demonstrates this with the Jcl in combination with our logging tool SmartInspect and I’ve heard Fabio of Eurekalog is working on a similar feature for his component:

[sourcecode language='delphi']
unit StackTrace;

interface

uses
SysUtils, Classes, JclDebug;

implementation

function GetExceptionStackInfoProc(P: PExceptionRecord): Pointer;
var
LLines: TStringList;
LText: String;
LResult: PChar;
begin
LLines := TStringList.Create;
try
JclLastExceptStackListToStrings(LLines, True, True, True, True);
LText := LLines.Text;
LResult := StrAlloc(Length(LText));
StrCopy(LResult, PChar(LText));
Result := LResult;
finally
LLines.Free;
end;
end;

function GetStackInfoStringProc(Info: Pointer): string;
begin
Result := string(PChar(Info));
end;

procedure CleanUpStackInfoProc(Info: Pointer);
begin
StrDispose(PChar(Info));
end;

initialization
// Start the Jcl exception tracking and register our Exception
// stack trace provider.
if JclStartExceptionTracking then
begin
Exception.GetExceptionStackInfoProc := GetExceptionStackInfoProc;
Exception.GetStackInfoStringProc := GetStackInfoStringProc;
Exception.CleanUpStackInfoProc := CleanUpStackInfoProc;
end;

finalization
// Stop Jcl exception tracking and unregister our provider.
if JclExceptionTrackingActive then
begin
Exception.GetExceptionStackInfoProc := nil;
Exception.GetStackInfoStringProc := nil;
Exception.CleanUpStackInfoProc := nil;
JclStopExceptionTracking;
end;
end.
[/sourcecode]

The unit merely starts and stops the exception tracking of the Jcl, implements a minimal stack trace provider and registers for the Exception provider events. The GetExceptionStackInfoProc and CleanUpStackInfoProc functions are automatically called by the RTL to give the provider the opportunity to initialize and cleanup the stack trace after an exception occurred. GetStackInfoStringProc is called indirectly when you access the StackTrace property and is responsible for returning the actual stack trace.

So, how do you use this unit? Let’s have a look at the following example:

[sourcecode language='delphi']
…

uses
…, StackTrace;

type
TForm1 = class(TForm)
Button1: TButton;
procedure Button1Click(Sender: TObject);
private
procedure SomeMethod;
end;

implementation

procedure TForm1.Button1Click(Sender: TObject);
begin
try
SomeMethod;
except
// Log the exception: We use SmartInspect here because it has
// built-in support for Exception.StackTrace but you could also
// access the StackTrace property here directly.
SiMain.LogException;
end;
end;

procedure TForm1.SomeMethod;
begin
raise Exception.Create(‘A test exception’);
end;

…
[/sourcecode]

As you can see, using this unit is just a matter of adding it to our uses clause. It won’t get any simpler than that. The unit will take care of registering/unregistering itself as a stack trace provider and when you now access the StackTrace property of an Exception object, you will get a detailed stack trace. To include the method names and line numbers in the stack trace, make sure to let the linker include debug symbols into your application and to enable the ‘Use debug .dcus’ option in case you also want line numbers from the VCL and RTL methods.

The SmartInspect Console showing the stack trace of an exception

Now, when you use SmartInspect for logging and have a stack trace provider registered, all your logged exceptions automatically include the exception’s call stack. Pretty useful, isn’t it?

I understand that there are many Delphi users who want to upgrade to Delphi 2009 to benefit from additional features but don’t care about the Unicode improvements (or already use a different technique to support Unicode such as the TNT Controls). Adding this compatibility mode would therefore be a great short-term solution for them but would result in huge technical debt for CodeGear.

For CodeGear, this would mean supporting two parallel versions of the VCL for at least the next few versions of Delphi. What a support and maintenance nightmare! Likewise, third-party vendors would have to ship two different versions of their products for Delphi 2009 and beyond. One compiled with Unicode support and one without. Again, a maintenance nightmare.

I admit that porting real-world applications to Delphi 2009 can be quite a bit of work. The just released version of SmartInspect now uses Delphi 2009 for the SmartInspect Console. Previous versions were built against Delphi 2006 and I can say that the transition wasn’t as smooth as I hoped for. Like Babniks product, the SmartInspect Console already supported Unicode by using WideStrings and the TNT Controls, so there wasn’t a need to update to Delphi 2009 just because of Unicode. But we wanted to fix a few Vista related bugs and other glitches and Delphi 2009 includes fixes for most of them. So we decided to give Delphi 2009 a try.

It turned out that converting the Console itself wasn’t that big of a problem. The warning messages of the Delphi 2009 compiler are really helpful and we got most of the Console running within about two weeks. A bit more problematic were some no longer maintained third-party controls and it took two more weeks to get the Console to a really stable state. That’s more than I expected but still okay for such a one-time conversion (the Console has about 50K LoC + forms, the unmaintained third-party controls were about 10K I guess).

In retrospect, I think it was well-worth the time and Delphi 2009 feels like a good version. The IDE is much more stable than Delphi 2006 and the language and VCL finally got some much-needed improvements. We are also happy that we could throw away the WideString and TNT related code. Using Delphi 2009 makes the Console code much cleaner. It also got a bit faster because of the performance improvements of UnicodeString compared to WideString.

So, while I can understand that not everyone wants to invest the time for the Delphi 2009 conversion, I do not think that adding this compatibility mode would have been a good decision, especially not in the long run.

We’ve also upgraded the Console to make use of the latest development tools and third-party components available, integrating it better with Windows Vista and preparing it for Windows 7.

If you have been using the beta versions, you won’t notice a lot of changes as we have only added a few additional bug fixes since the last beta. Here are the main changes of SmartInspect 3.2 compared to the previous stable release:

• New: Connection variables for automatically replacing variables in a connection string
• New: Width format option for text protocol pattern string
• New: Custom timestamp format for text protocol pattern string (platform-specific)
• New: Exception.StackTrace support for LogException (Delphi only)
• New: Support for protocol objects in addition to raw streams in the Dispatch method of the memory protocol
• New: ‘View | Create View’ Console feature for quickly opening new filtered views
• New: Console command line parameter /configdir and /pipename
• New: Internal string pooling which drastically reduces the overall memory consumption of the Console
• Improved: Log file loading performance increased and reduced memory consumption
• Improved: Bookmarks in the Console now indicate whether they are already in use
• Improved: Better view performance when displaying large Log Entry titles
• Improved: Increased performance of title and data viewers

This is also the first release that includes the memory consumption improvements we implemented in the Console and explained on our software quality blog. You can download the release from My Gurock.com and the updated trial is available on our website. As usual, you can simply install the new version over your existing installation. For a complete list of changes, please see the online help or our announcement forum.

SmartInspect 3.2 is a great release and contains quite a few new features and lots of smaller improvements and fixes. You can expect a few blog postings about some of the new features to appear here soon.

We also updated the SmartInspect for PHP library. It contains a few bug fixes and can be downloaded from our SmartInspect resource page. Please see the smartinspect.php file for changes.

The SmartInspect Router is a Windows service application which can receive SmartInspect log packets via TCP/IP or named pipes and routes them to different log destinations. The supported log destinations are the same as those from the SmartInspect libraries, namely log files, TCP/IP or named pipes. Two typical uses of the SmartInspect Router are to combine multiple client logs into a single log file and to collect logs in a central location. The SmartInspect Router is intended to provide a common logging infrastructure for distributed systems, multi-process applications and web applications.

Since SmartInspect is used more and more for web applications and distributed systems it became clear that we needed a server application that can receive logs via TCP/IP or named pipes and stores them in a central location. Many customers have expressed their need for such a solution.

We began work on the SmartInspect Router a few months ago and are happy to finally release the first version in this EAP. After installing the new EAP version, you can find the setup of the SmartInspect Router in the deploy directory of your SmartInspect EAP installation. The SmartInspect Router can be configured with a simple XML configuration file (please see the included help for details).

Releasing a completely new application is always a challenge and there might be a few things that don’t work correctly. If you encounter a problem with the SmartInspect Router, we would appreciate a comment in our SmartInspect EAP forum.

Every now and then we get questions about beta versions or technology previews of upcoming SmartInspect releases. So far, we have been hesitant to release beta versions. But in order to become a bit more transparent in what we are working on for SmartInspect, we decided to start a SmartInspect Early Access Program (EAP). In this EAP, we will provide interested customers with technology previews and beta versions of SmartInspect. We plan to release new EAP versions on a frequent basis.

To download EAP versions of SmartInspect, you need to have a customer account on our My Gurock.com customer portal and an active SmartInpect support plan. After logging in to My Gurock.com, you should notice a new menu item called “EAP” with the latest EAP releases. Note that EAP versions might include incomplete or buggy features and/or might not work as expected. They haven’t gone through the entire QA cycle yet and are only provided for testing purposes. These versions are not intended to be used on production systems and are therefore not fully supported.

What’s new in this EAP version?

The first version of the EAP is a preview version of the upcoming SmartInspect 3.0 major release. Please note that this EAP release is not a feature-complete preview version of SmartInspect 3.0. It contains most major features of the upcoming major version but lacks several smaller features which will be added over the next few weeks. We will post more about the 3.0 version soon. Here is a short list of new major features that are already available in the first EAP release:

• New named pipe protocol: The new named pipe protocol is the new default protocol instead of TCP/IP and provides a great performance increase for local live logging. Also, since it’s very fast to test if a pipe is available, automatically reconnecting is really useful for production systems now. This means that you can start the Console when your application is already running and you will immediately receive live logging packets.
• Improved SmartInspect sessions: It is now possible to specify log levels on a per-session basis. Sessions can now also be configured via SmartInspect configuration files. This is really great for production systems like ASP.NET applications, as you can now enable or disable logging for specific users directly via config files.
• Support for log file encryption: SmartInspect now supports log file encryption. You can specify an encryption key for log files in the SmartInspect libraries and the logging data is automatically encrypted with AES. This feature can be useful if you log sensitive information that has to be protected.
• Asynchronous logging: You can now specify for each logging connection if you want to log asynchronously. The libraries will create a (configurable) buffer and process logging data in a separate thread. This feature is very useful for high-performance production systems or for logging via TCP/IP with regular reconnects.
• Configuration timer: The new configuration timer allows you to automatically reload SmartInspect configuration files when the config has changed. This allows applications to pick up logging configuration changes without the need to be restarted.
• Sorting of packets: An often requested feature is the ability to merge log files based on the timestamps of the log packets. The SmartInspect Console now supports this with its new Sort feature.
• Improved Console navigation: We also added new navigation features to the Console like Go to Time, Synchronize and back-/forward actions.

There are other new features that are not mentioned in the list above. Please refer to the What’s New section of the online help for a complete list of new things.

Installing EAP and official versions side-by-side

It is generally possible to install EAP and official releases on the same machine, but please be aware of the following issues:

• You should install EAP releases into a directory different from your official SmartInspect installation. Doing otherwise overrides your existing installation.
• Do not activate the IDE library integration in the EAP setup if you plan to further use the official versions of the libraries. Doing otherwise can override your existing integration.
• After installing an EAP release, double-clicking log files in the Windows Explorer opens them with the new EAP versions of the Console (same for config files and the Configuration Builder).

To avoid the possibility of interfering with an existing installation altogether, it is highly recommended to install EAP releases on virtual machines only. Please note that you can reuse your existing SmartInspect 2.x license key for this EAP version.

Will SmartInspect 3.0 be free for existing customers?

If you have an active support plan when SmartInspect 3.0 is released, you can download and use the new SmartInspect version without paying any upgrade fees. And because every SmartInspect license already includes one free support plan year, you will get the 3.0 for free if you purchased SmartInspect recently (or if you plan to purchase new licenses now). If your support plan expired, you can renew your support plan by logging in to My Gurock.com.

Providing feedback

If you are experiencing problems with this (or a future) EAP version or if you have suggestions or feature requests, please post them it in the related forum or email us. We would really love to hear your feedback!

Although we don’t officially support Delphi 5 with SmartInspect 2.x, we had a version of the SmartInspect Delphi library for SmartInspect 1.4 which runs fine with Delphi 5. This library doesn’t support some of the newer features of SmartInspect 2.x like Unicode (this is actually the main reason we don’t support Delphi 5 with SmartInspect 2.x because Delphi 5 lacks a lot of Unicode related functions), the Watches Graph, the backlog protocol feature and others. The good news is that the current Console version is fully backwards-compatible and can work with the data from older SmartInspect libraries. You thus still benefit from some of the new features like the marker functionality or the improved filtering and navigation capabilities even when using the old library.

So, if you need a logging solution for your Delphi 5 application, send us an email and we are happy to help you evaluate SmartInspect with Delphi 5.