• Optimizing Memory Consumption with String Pools: Part I
• Optimizing Memory Consumption with String Pools: Part II
• Optimizing Memory Consumption with String Pools: Part III

Integrating string pools into the SmartInspect Console means that you will be able to open and analyze even larger log files, as string pools can reduce memory usage by 40% and more. If you haven’t yet subscribed to our new blog and are interested in software quality, testing, usability and related topics, make sure to subscribe:

» No bug left behind Feed

So instead of publishing all the different things here on this blog, we’ve just launched No bug left behind, our new blog with the stupid name about software quality, testing and related topics. We have quite a few plans for the new blog and are also looking for guest postings, relevant links and articles. So if you have something interesting to share, please let us know!

If you are interested in software quality, usability, testing and other related topics, make sure to subscribe to our new feed.

No bug left behind »

Recent studies show that up to 40% of system failures are caused by software bugs and that common memory and concurrency related bugs account for 60% of system vulnerabilities and security problems. So reducing software bugs in your application is the best way to increase the stability, reliability and security of your software.

During the development of our logging tool SmartInspect, we used many techniques to keep the quality of our product high and this list contains some of the techniques we use. So without further ado, here is my list of 12 practical tips for building bug-free software (or at least software with fewer bugs):

1. Code Reviews

Four eyes see more than two. That’s why you should let other developers review your source code on a regular basis. Pair programming on the other hand, a technique where two developers write code together for longer periods, isn’t for everyone and is often not needed. But complicated, important or security related code greatly benefits from code reviews and will improve your code quality a lot.

2. Beta Tests

Beta tests play an important role with keeping your software’s quality high. But most times, it doesn’t make sense to release beta versions of your software for minor updates. Major releases on the other hand, should be tested by end-users and customers before going gold. You can test your software as much as you want, if you cannot control the execution environment, the chance is high that end-users will find bugs and problems with all the different computer configurations out there. Also make sure that your software reached a high quality standard before giving it to beta testers. You don’t want to waste the testers’ time by letting them find and report bugs that you already know about.

3. Automated Tests

Automated tests like unit tests or automated GUI tests can be used to guarantee the functionality of application modules, application programming interfaces (APIs) and user interfaces. You don’t have to be a test-driven development wizard to make good use of automated tests. Using unit tests for key parts of your application can go a long way towards building more reliable software. There are tons of unit testing frameworks, web and GUI testing tools out there that you can utilize.

4. Logging

Using log files or live logging during development and production usage is an important and useful technique to identify bugs, find concurrency problems and to analyze and find out why an application crashed. Advanced logging tools are also able to log complete objects, trace threads and distributed systems and offer rich viewer tools to monitor your application. Instead of writing your own basic logging framework, you should use proven and advanced libraries and tools. Many open-source and even some commercial offerings have been released over the years, including our own .NET, Java and Delphi logging tool SmartInspect.

5. Error Reporting

To find and resolve errors and exceptions, you first have to know what kind of errors your users and customers are experiencing. Many users of trial software won’t get in touch with you to report any errors. Instead, they will just uninstall your application and test a competing product. To make error reporting for end-users easier and more useful to you, you should use automated error and exception reporting techniques. When an unhandled exception occurs, your application should show a friendly dialog offering the user to send an error report back to the developer. Error reports should contain all kind of information to help you identify the problem, including error messages, call stacks, version numbers and log files.

6. Customer Feedback

Similar to error reporting, you should make giving feedback as easy as possible for your users and customers. In SmartInspect, for example, we have buttons in the menu and toolbars to send feedback and questions. You can also use a short (optional) survey when a user uninstalls your application. This survey should only have a few questions and besides other things ask why the user uninstalled your software. If a lot of users report that they uninstalled your software because of stability problems, you will know that your application isn’t as high-quality as you have thought.

7. Use Proven Code

You should build the core functionality and main features of your applications yourself, because only then are you able to easily and quickly modify and improve it. But for many other parts you can reuse existing and proven code. It will take you years to build a stable, easy-to-use and feature complete reporting engine or setup builder, for example. Often times it’s better to use proven existing code, either from internal libraries, third-party companies or open source solutions if the license permits this.

8. Dedicated Testers

If possible, you should have dedicated testers in your organization for quality assurance. In fact, you should have lots of them. For simple standard applications, one tester per developer is a good rule of thumb. For applications that are complicated and time-consuming to test, two or more testers per developer are needed. Many small organizations cannot afford dedicated testers. If this is the case, developers should test each others’ code. It’s important that others test your code and functionality, because general wisdom says that developers do a really bad job testing software that they wrote themselves.

9. Virtual Machines

To test your software on as many different environments and operating systems as possible, you should use virtual machines with tools like VMware, Virtual PC or other available virtualization software. Besides allowing you to test your software on all kinds of configurations, you will save tons of time because you can easily copy, share and reset the virtual machines. It’s a good idea to create many different standard images for all the operating systems you regular test on and put them on a file server. When you need a specific configuration to test something, you can then start with one of your base images without installing the operating system, drivers and required software and so on.

10. Write a Specification

Many bugs are caused by badly designed class hierarchies, inaccurate interfaces, wrongly understood requirements and the therefore required workarounds. That’s why an experienced developer or architect should write a specification with all the collected requirements and technical implementation details before writing the first line of code. But as always, requirements change during the lifetime of a software application. That’s why it’s important to keep the specification up-to-date and plan the architecture changes for any new or altered requirements.

11. Use a Good Debugger

If you use an IDE like Visual Studio, Eclipse or Delphi, you already have access to a powerful debugger that you should use. But with many development environments and development platforms like PHP, Windows Scripting, Python and Ruby, many developers aren’t using a debugger. In those scripting environments, developers often try to squish bugs by try and error, changing code parts, adding a few print statements and so on. This is not only a very cumbersome and time-consuming way of identifying and solving bugs, it’s also very dangerous if you don’t fully understand your code and are able to step through it with a debugger. Do yourself a favor and get a good debugger for your development platform (and there are debuggers for almost everything).

12. Debug and Strict Options

Many development environments allow you to enable special debugging options like range checking, overflow checking and memory corruption checking. Such options should be enabled during development and sometimes even during quality assurance to identify hard to find bugs. Many script languages like Perl or PHP on the other hand allow you to enable certain rules and warnings that will force you to declare variables before using them. This is especially useful if a script language is case-sensitive and it’s easy to make typos.

As I already said, these tips won’t make your software magically bug-free, but they allow you to make your software much more stable, reliable and usable if you put the work into it. And on top of it, it will make you a better developer, too.

1. Do you use source control?
We have been using a source control system from day one and wouldn’t develop even the most simple program without it. It’s just so convenient to have backups of every single version we ever checked into the source control system. Another important feature for us are branches. When we have to fix a bug in the currently released version and when we made changes to our code that we don’t want to ship right now, we create a new branch and fix the bug in it. We use Subversion as our source control tool and although it could be faster, it’s the best source control system I ever used.

2. Can you make a build in one step?
As Tobias outlined in the Explaining our build system blog posting, we use Cygwin and Makefiles for our build setup. We can make a build in one single step by executing the main Makefile. The scripts do everything from getting the sources, building the applications, help files, libraries, setups and so on. It has been a great timesaver for us.

3. Do you make daily builds?
We don’t have scheduled daily builds as it doesn’t really make sense for us at this stage. We start builds by hand when we check in some bigger changes into the source control system. But since we have days without a single source code change, we don’t need to have the build machine running 24/7. Besides, breaking the build is very rare as we are just 2 developers and the chances of breaking something is much smaller.

4. Do you have a bug database?
We use Trac as our bug database (and wiki). So far it’s been okay but it could be better. One thing I’m missing in Trac is a hierarchical overview of projects, bugs and feature requests (we had that in our home-grown bug database). Since our home-grown bug database was desktop based and we needed something web-enabled, we switched to Trac. We haven’t switched to something else because Trac has such a great Subversion integration and a built-in wiki and we haven’t found a good alternative for it yet.

5. Do you fix bugs before writing new code?
Most of the times we do fix bugs before writing new code. But there are some situations where this is not true. Take Windows Vista, for example. We tested SmartInspect on Vista and fixed most of the problems immediately. For some problems we decided to wait for later RC versions or the final Vista version as we aren’t sure if the problems are bugs of Vista or of SmartInspect. So in general I would say that we adhere to this rule but I don’t think this rule should be enforced strictly for every case.

6. Do you have an up-to-date schedule?
We do have an up-to-date schedule and are very strict about keeping SmartInspect in a ready-to-ship state. So if we have to release a new version of SmartInspect, we can do so pretty quickly. We set us milestones with planned features and release dates and try to stay close to the schedule. We have been pretty bad about this with SmartInspect 1.0, but improved considerable since we released the first version.

7. Do you have a spec?
Strictly speaking, we have to say ‘no’ to this one. We do talk a lot about planned features, their implementation and the best design. We also write down use cases and user stories when we plan a new release. But we don’t write 50-page documents with detailed discussions about every single feature that we want to put into a new release and I doubt that it would be worth it. We do write small specs about planned future products when we find a good new idea and want to preserve it for the future, but we don’t have specs for new versions of SmartInspect.

8. Do programmers have quiet working conditions?
I’m happy to say that this one is true for us. As we are just two developers and both of us have a quiet private office, it’s pretty easy to confirm this.

9. Do you use the best tools money can buy?
I would say ‘yes’ on this one. We have multiple monitors, good desks and good chairs, all the software that we need and up-to-date machines. We joined the Empower Program, for example, to get all the operating systems for testing and production usage that we need. We also have other not-so-cheap tools that make it easier for us to generate the SmartInspect library documentation which saves us a lot of time.

10. Do you have testers?
We obviously don’t have any dedicated testers but we compensate this by having a huge number of automated test cases for the SmartInspect libraries and comprehensive test plans. We execute the test cases and test plans on a regular basis and keep the bug count surprisingly low (our customers confirm this).

11. Do new candidates write code during their interview?
As we haven’t interviewed any candidates, this one doesn’t really apply to us. I guess this one doesn’t apply to most of the MicroISVs out there and is more appropriate for bigger teams.

12. Do you do hallway usability testing?
We did quick usability test of SmartInspect in the past and plan on doing it in the future. It has helped us a lot and we changed some of the features and changed default settings after seeing people having problems with the GUI. One of the things that the usability testing made us realize is that many people have problems with docking windows. That’s why we disabled docking in the SmartInspect Console by default and added a Reset Docking feature to undo any docking with one click.

All in all we rate 8 out of 12 on Joel’s Test with 2-3 rules not really appropriate for a very small ISV like us. That’s not too bad I guess and we are pretty happy about our development system. How do you rate on the test with your MicroISV?

Multi-Monitor Programming Pitfalls

Creating Custom Exceptions in .NET