Archive for the 'VMware' Category

Does your software application have bugs? Of course it has, every software application that’s out there has bugs and bug-free software is a myth. But it’s still possible to greatly minimize bugs, security problems and errors in your application by following a few tips and techniques I outline in this posting.

Recent studies show that up to 40% of system failures are caused by software bugs and that common memory and concurrency related bugs account for 60% of system vulnerabilities and security problems. So reducing software bugs in your application is the best way to increase the stability, reliability and security of your software.

During the development of our logging tool SmartInspect, we used many techniques to keep the quality of our product high and this list contains some of the techniques we use. So without further ado, here is my list of 12 practical tips for building bug-free software (or at least software with fewer bugs):

1. Code Reviews

Four eyes see more than two. That’s why you should let other developers review your source code on a regular basis. Pair programming on the other hand, a technique where two developers write code together for longer periods, isn’t for everyone and is often not needed. But complicated, important or security related code greatly benefits from code reviews and will improve your code quality a lot.

2. Beta Tests

Beta tests play an important role with keeping your software’s quality high. But most times, it doesn’t make sense to release beta versions of your software for minor updates. Major releases on the other hand, should be tested by end-users and customers before going gold. You can test your software as much as you want, if you cannot control the execution environment, the chance is high that end-users will find bugs and problems with all the different computer configurations out there. Also make sure that your software reached a high quality standard before giving it to beta testers. You don’t want to waste the testers’ time by letting them find and report bugs that you already know about.

3. Automated Tests

Automated tests like unit tests or automated GUI tests can be used to guarantee the functionality of application modules, application programming interfaces (APIs) and user interfaces. You don’t have to be a test-driven development wizard to make good use of automated tests. Using unit tests for key parts of your application can go a long way towards building more reliable software. There are tons of unit testing frameworks, web and GUI testing tools out there that you can utilize.

4. Logging

Using log files or live logging during development and production usage is an important and useful technique to identify bugs, find concurrency problems and to analyze and find out why an application crashed. Advanced logging tools are also able to log complete objects, trace threads and distributed systems and offer rich viewer tools to monitor your application. Instead of writing your own basic logging framework, you should use proven and advanced libraries and tools. Many open-source and even some commercial offerings have been released over the years, including our own .NET, Java and Delphi logging tool SmartInspect.

5. Error Reporting

To find and resolve errors and exceptions, you first have to know what kind of errors your users and customers are experiencing. Many users of trial software won’t get in touch with you to report any errors. Instead, they will just uninstall your application and test a competing product. To make error reporting for end-users easier and more useful to you, you should use automated error and exception reporting techniques. When an unhandled exception occurs, your application should show a friendly dialog offering the user to send an error report back to the developer. Error reports should contain all kind of information to help you identify the problem, including error messages, call stacks, version numbers and log files.

6. Customer Feedback

Similar to error reporting, you should make giving feedback as easy as possible for your users and customers. In SmartInspect, for example, we have buttons in the menu and toolbars to send feedback and questions. You can also use a short (optional) survey when a user uninstalls your application. This survey should only have a few questions and besides other things ask why the user uninstalled your software. If a lot of users report that they uninstalled your software because of stability problems, you will know that your application isn’t as high-quality as you have thought.

7. Use Proven Code

You should build the core functionality and main features of your applications yourself, because only then are you able to easily and quickly modify and improve it. But for many other parts you can reuse existing and proven code. It will take you years to build a stable, easy-to-use and feature complete reporting engine or setup builder, for example. Often times it’s better to use proven existing code, either from internal libraries, third-party companies or open source solutions if the license permits this.

8. Dedicated Testers

If possible, you should have dedicated testers in your organization for quality assurance. In fact, you should have lots of them. For simple standard applications, one tester per developer is a good rule of thumb. For applications that are complicated and time-consuming to test, two or more testers per developer are needed. Many small organizations cannot afford dedicated testers. If this is the case, developers should test each others’ code. It’s important that others test your code and functionality, because general wisdom says that developers do a really bad job testing software that they wrote themselves.

9. Virtual Machines

To test your software on as many different environments and operating systems as possible, you should use virtual machines with tools like VMware, Virtual PC or other available virtualization software. Besides allowing you to test your software on all kinds of configurations, you will save tons of time because you can easily copy, share and reset the virtual machines. It’s a good idea to create many different standard images for all the operating systems you regular test on and put them on a file server. When you need a specific configuration to test something, you can then start with one of your base images without installing the operating system, drivers and required software and so on.

10. Write a Specification

Many bugs are caused by badly designed class hierarchies, inaccurate interfaces, wrongly understood requirements and the therefore required workarounds. That’s why an experienced developer or architect should write a specification with all the collected requirements and technical implementation details before writing the first line of code. But as always, requirements change during the lifetime of a software application. That’s why it’s important to keep the specification up-to-date and plan the architecture changes for any new or altered requirements.

11. Use a Good Debugger

If you use an IDE like Visual Studio, Eclipse or Delphi, you already have access to a powerful debugger that you should use. But with many development environments and development platforms like PHP, Windows Scripting, Python and Ruby, many developers aren’t using a debugger. In those scripting environments, developers often try to squish bugs by try and error, changing code parts, adding a few print statements and so on. This is not only a very cumbersome and time-consuming way of identifying and solving bugs, it’s also very dangerous if you don’t fully understand your code and are able to step through it with a debugger. Do yourself a favor and get a good debugger for your development platform (and there are debuggers for almost everything).

12. Debug and Strict Options

Many development environments allow you to enable special debugging options like range checking, overflow checking and memory corruption checking. Such options should be enabled during development and sometimes even during quality assurance to identify hard to find bugs. Many script languages like Perl or PHP on the other hand allow you to enable certain rules and warnings that will force you to declare variables before using them. This is especially useful if a script language is case-sensitive and it’s easy to make typos.

As I already said, these tips won’t make your software magically bug-free, but they allow you to make your software much more stable, reliable and usable if you put the work into it. And on top of it, it will make you a better developer, too.

We planned to move our physical servers over to virtual VMware machines for quite some time now and finally purchased and set up our new system at the end of last year. As our experience might be useful for some people who plan the same, I decided to describe our solution and comment on the actual performance.

Our new physical server contains a Core 2 Duo E6600, 4GB RAM and 4 x 320 GB disks set up as RAID 5 with a spare disk. It’s very important to have lots of RAM for VMware, as each individual virtual server needs as much RAM as a normal physical box, so the more RAM the better. We decided to go with 4GB as it’s currently the most economical choice and should be enough for our needs.

On the physical box we just installed Windows Server 2003, Active Directory including DNS/DHCP and VMware Server. We decided to install Active Directory directly on the physical machine instead of a virtual one because the server is also acting as our file server. And because virtual machines don’t provide the same good IO performance as physical machines, we wanted to have the file server directly on the server.

We then set up the different virtual servers and moved all services and data files like our Subversion repository, emails and so on to the new servers. We also moved our virtual build machine to our new server. The build machine contains many IDEs and is responsible for compiling new versions of SmartInspect and building the help and setup files. When it’s building a new version, it consumes a lot of CPU and IO performance and we were curious how it would perform on our new system.

After using our new server(s) for some time now, I can say that the performance is much better than we actually hoped for. We can start a build on our virtual build machine and everything else like our email and Subversion servers are as fast as normal. And the 4GB RAM gives us enough capacity for more servers and development/tests machines in the future. The actual performance and required hardware obviously depends on the actual number of users you have and our two users are probably not very representative.

But based on my observations and tests, I suspect that such a server should easily handle all the common services like Subversion, E-Mail, File Server, Active Directory and web based applications for up to 10 users or more and therefore makes it a very economical solution for small ISVs. And besides the space and cost savings such virtual servers provide, it becomes ridiculous easy to backup entire servers just by copying or burning the virtual machine files.

Until now, the free VMware Server was only available as a beta, but today VMware Inc. finally released the 1.0 of the product. This makes it possible for ISVs to save money and time by hosting multiple servers on one physical machine and start/stop/control the servers remotely. We have been waiting for the final release and can now start converting our servers to virtual ones.

VMware is announcing their $200.000 Ultimate Virtual Appliance Challenge. Anyone can submit a self-built virtual appliance and get the chance to win up to $100.000 as the first price. VMware defines a virtual appliance as:

“Virtual Appliances are not general purpose Virtual Machines. They are designed to solve a specific problem and are intended to simplify the installation and management of the solution.”

All submitted virtual appliances will be rated by a panel of industry experts and by community votes. It will be interesting to see what people come up with.

When we first heard about the free VMware Server, we got really excited. We have been planning to replace our aging servers with some new hardware and thought about reducing the server count. Besides our internet server which is located at our provider, we have three internal servers working as a file server, intranet and database server, source control server, mail server, SUS server, domain controller and a bunch of other things. Three servers may sound a lot for just two users, but we wanted to separate things a bit for security and performance reasons.

Now that VMware Server is going to be released for free, we are thinking about getting a nice dual-core server with 4 GB RAM and just hosting different servers with VMware. I see some advantages of using VMware Server instead of using separate boxes:

• Adding a new server is just a click away as long as there is enough free RAM and general performance available
• Backing up the different servers becomes ridiculous easy with VMware, as we can just burn the virtual machine images to DVDs
• The hardware is used much more economically as we just need one really fast server which should be enough even when the server is under high load

Some services such as the file server will still be running on the native Windows server for performance reasons. But all non-Windows services like our email and intranet servers will run on Linux or FreeBSD virtual machines.

Has anyone used something like this before? I know that Mike Gunderloy is using a VMware GSX server for his server infrastructure and he seems to be happy with it. He wrote on his blog:

“For example, I’ve taken a single Dell 1850 dual-processor server and used VMware GSX to simultaneously run a Subversion server, a Cruise Control .NET server, a Windows Software Update Services server, a Data Protection Manager Server, and several more servers - without the hassles of worrying about whether any of that software will conflict or fight over ports. In addition to keeping the software separate, this also enables me to make much better use of server hardware by running it at a higher average load, which in turn means paying for less hardware.”

If anyone has experience with such a system and can report on the performance of such a setup, it would be great if you could comment on it.

The new free VMware Player could be a great tool for ISVs to help prospective customers evaluate their software easier and faster (just as I noted in my last posting). ISVs could offer fully preconfigured virtual machines with their trial software and interested users could get the software up and running in just a minute. This is especially useful for software where you need to fulfill some major requirements.

One example for this scenario could be the help desk software HelpSpot from UserScape. Ian Landsman, the founder of UserScape, just released HelpSport and he is offering a trial download for potential customers to test his software. As HelpSpot is web based, it has some major dependencies like a web server, database engine and script language attached to it. Now to evaluate HelpSpot, it could take hours and hours to install and configure a web server, a database and a scripting language. To get a competitive advantage, Ian could configure a Linux virtual machine with Apache, MySQL, PHP and the HelpSpot trial and offer it as a free download (additionally to the normal trial download). This would make the whole evaluation process much easier for his users and could lead to more paying customers in the end.

VMware Inc. just announced the release of their new product VMware Player. VMware Player can be downloaded for free and lets anyone use preconfigured virtual machines without buying VMware Workstation. For those who do not know what VMware Workstation is, it’s a virtualization software to ‘emulate’ a computer. You can run multiple copies of Windows on the same computer, test other operating systems and use it to test your software on different systems. Among other differences, the Player cannot be used to create new virtual machines.

I think the release of VMware Player is a smart move. VMware Inc. tries to convince ISVs to build preconfigured virtual machines to distribute their beta software and production ready applications preinstalled as virtual machine. This would allow customers to quickly evaluate new applications and test beta software without the need to install them. I’m sure many more people would test out beta software if they could just download a virtual machine (maybe even bundled with VMware Player) and then securely test it without putting their machines at risk.

It could be a big advantage for VMware Inc. if ISVs would start distributing their software as virtual machines. VMware Inc. has another set of server products to run and manage multiple machines on a central server. Since all virtual machines that are created with one of the VMware products work on all other VMware systems, customers could just download a virtual machine of a purchased software, copy it to a VMware server and it would just work. This is especially appealing for complex server software where you need hours to install and configure the system and the software.

This would also be beneficial to ISVs. Potential customers could evaluate complex software much faster and without the hassle to install and configure it. The only downside that I can see is that it won’t work with Windows software, because you are not allowed to distribute the Windows operating system. But for software running on Linux systems (especially server software), this could be a win-win situation for both ISVs and VMware Inc. Check out the Virtual Machine Center where you can already download preconfigured virtual machines form Oracle, MySQL and others.

For a welcome change from testing the upcoming SmartInspect release, I decided to write some notes about how we test our software on multiple platforms.

Let’s begin with the simpler part, the SmartInspect Console. Since the tests for the Console are currently not automated, we maintain a large test plan which needs to be executed manually. We process this plan on every supported platform before we release a new version. Testing the Console on one platform basically means to grab a fresh copy of a Virtual Machine for this platform, install the current SmartInspect build on it and process the entire test plan. It’s simple but quite tedious. It normally takes an entire day or sometimes more to process one platform (including different platform versions).

For the libraries, we developed lots of unit tests for each library which test most of their functionality. Executing these tests takes only a few minutes. However, we faced a problem when we thought about running them on multiple platforms. Since our test frameworks originally needed a compiler to run, we would have needed to install a compiler for every language on every supported platform. This would have been highly impracticable. So we decided to adjust our build system to create nice precompiled packages for our tests. Testing a library for a specific platform basically means now to copy these packages to a new Virtual Machine, unzip them and start the automated tests.

We also thought about automating the testing of the libraries even further. The new VMware version contains a tool which can be used to start Virtual Machines directly from the command line. Combined with a few scripts, this could be used to automate the testing of the libraries on all supported platforms completely.