SmartInspect 2.0: Backlog functionality

This is the first posting of a series talking about upcoming features of SmartInspect 2.0. The first new feature that we want to introduce is the so called backlog functionality. The backlog feature of the SmartInspect libraries is a flexible packet queue that minimizes the I/O costs by saving log entries in memory until a log packet with a certain configurable log level is logged. We got the idea for this feature from one of our customers who implemented a similar functionality with the help of the library filter event.

If the backlog of a protocol is enabled, log packets are first written to a queue in memory rather than directly to the protocol specific destination (a log file or TCP/IP connection, for example). The queue can be configured to have a certain trigger log level and a maximum amount of memory that it consumes. If a log packet arrives with a log level equal to or greater than the specified trigger log level, the queue is flushed and all packets in the queue are written to the actual protocol destination. If the packet queue has been filled up with packets and a new packet is about to be stored, old entries are automatically discarded. The new backlog functionality can thus drastically reduce the I/O performance costs involved in logging.

We designed the backlog functionality especially for production usage. If you enable the backlog and configure it to save log packets to a log file in case of an error, you will be able to analyze all errors that occurred over the time and inspect them in their context. Unimportant log entries that were generated long before an error occurred are automatically discarded. This approach guarantees the best performance during regular production usage and enough information when an actual error occurs.

Depending on the used protocol and the complexity of the logging functions, the logging performance can be improved by a factor 10 and more. Since the logging performance plays a very important part on production systems, we are really happy to have this feature in SmartInspect.