On my latest project, I’ve been working with our developers to test defect fixes as we prepare for our initial release. Normally, I’m the only one on the team who tests defect fixes, but as the result of a recent push, our developers are actively working with me on testing.
Testing with our developers has helped us discover some huge benefits. Let’s look at some of them.
Get TestRail FREE for 30 days!
Developers Feel My Pain
The biggest benefit so far has been that working in the test environment has given them a more realistic understanding of how challenging installation and configuration can be.
The developers typically work with pre-configured software. They are saved the trouble of installing the system and manually inputting initial configuration to get to the task they are interested in. This helps them save time while working on a specific feature, but it left them immune to parts of the user experience.
Once they began setting up their test environments, they realized that over time, the installation process had become burdensome. Additionally, the workflow for initial product configuration had evolved and become somewhat brittle; it now required a specific order that didn’t feel intuitive. These were issues that I had brought up and tried to advocate for, but until the developers experienced it themselves, they didn’t realize how important it was to fix the problem.
Developers Learn About Risk
Once the developers finished installation and configuration, it was time to start testing in earnest. Our focus was on testing defects that involved the calculations made by the program. We wanted to test these defects carefully to make sure that the results generated by the program were correct, so our testing needed to be especially thorough.
Although the defects were well written, the developers soon discovered that the scope of changes made to fix defects aren’t always limited to the defect that was reported. Sometimes the change made to resolve a defect doesn’t really fix the defect, or it fixes the defect but causes another problem, or it reveals a new defect that was hidden by the behavior of the first. Deciding whether a defect is “fixed” and what constitutes a new defect can be challenging. To help the developers decide what to do when a defect resolution was imperfect, I used the language of decision-making and risk.
At this stage of development, after the features have been completed and the release is imminent, every change to the project presents a risk to the release. We need to help the product manager make decisions about that risk. In the case of the defects that we included in our testing scope, the decision has already been made that the risk of changing code to fix the defect is less then the risk of delivering incorrect results to the customer. So, if the behavior described in a defect report has not been fixed and results in incorrect results being delivered to the customer, we can just return the defect to the backlog with notes for it to be fixed.
If the behavior described in the defect report has been fixed but we find new undesired behavior during testing, then the original defect report can be successfully closed, and the new defect needs to be written up in a separate report. This means that the product manager can make a decision about the risk of fixing this new defect versus including the undesired behavior in the release.
If, during testing the defect resolution, we find that the results are still subpar but the behavior has changed significantly enough that the problem is now different from what was described in the defect report, we need to think about the decision-making process. If I were a product manager considering this change, would this be the same decision-making process as it was for the original defect and results, or would this represent a significantly different risk evaluation?
If it’s difficult to decide, usually the right thing to do is call over the product manager and talk about what you’ve found. Let the product manager make the decision.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
Developers Make Great Testers
After discussing the notion of risk with the developers, I learned that they are in a unique position to evaluate risk. As they test, often they’re thinking about what it would take to fix problems they’ve found. They realize that it may not be a good idea to make those fixes ad hoc, but because they already have a fix in mind, they can speak to the risk of the potential fix in great detail.
Once again my team finds that by working together, we can reap the rewards of having all of our knowledge and strengths applied to each stage of development—including testing.
This is a guest posting by Carol Brands. Carol is a Software Tester at DNV GL Software. Originally from New Orleans, she is now based in Oregon and has lived there for about 13 years. Carol is also a volunteer at the Association for Software Testing.
Test Automation – Anywhere, Anytime
