Tester’s Diary: Who Values What

how to measure quality in testing

This is a guest post by Rachel Kibler.

Back when I was a new tester, I assumed quality was a standard that could be measured by things like bugs found. Those were the days, right?

Gerald Weinberg famously asserted, “Quality is value to some person.” People talk about quality meaning value, but they often neglect the kicker: “to some person.” Throughout the jobs I’ve had, nailing down who the “person” is was sometimes a challenge.

Round 1: No Mission, Some Analytics

At my first testing job, I worked for a bank on their mobile team. We had separate apps for consumer and business banking, and we worked closely with a vendor for the bulk of the apps. Bugs that we filed against the vendor took months to resolve, and bugs we filed against our own work usually would be resolved within a couple of weeks.

Looking back on that job, no one ever explicitly said what our mission was — what the point was of allowing customers to do their banking on their phone. We certainly didn’t make the bank a lot of money, and it felt like the apps were just an expected convenience for our customers rather than a statement about what we believed. Maybe it’s OK that we were just following the industry standard.

We did have analytics about feature usage, which helped prioritize things a bit. And I can see that our decisions about what to build and fix were geared mostly toward consumers rather than businesses, which perhaps did imply that people come first. But nothing was explicitly stated.

Round 2: So Much Mission, So Little Data

I left the bank for a startup, and that was a different ballgame. The company was based around an app that did serve a mission: to help people create and maintain privacy online. That helped focus some decisions.

However, we were so good at giving people privacy that we had very little data about our users, so figuring out “value to some person” often meant informally polling our colleagues, friends, and family to find out what was important to them. Our backlog of bugs was horrendous, but instead of fixing a bunch, we would do feature work to enhance privacy. We viewed bugs through the same lens of privacy and wanting a person to feel safe, and the ones that had an impact on that mission were fixed almost immediately.

It was exhilarating work, and I felt like what I did matter to the people who were using our software, though it was almost entirely anecdotal. The mission of the company took root in me, and it felt important to have that to guide me.

Modern Test Case Management Software for QA and Development Teams

Round 3: Strong Mission, Strong Analytics

And now I work for 1-800 Contacts. We sell contact lenses, and all of our tech is designed to facilitate that. We have phenomenal customer service (live people answer every call!), which helps to sell more contacts. The team I work on lets people take an eye exam online, have it reviewed by an ophthalmologist licensed in their state, and get a new prescription for contacts. This also sells more contacts. But at the same time, we’re working in telehealth, which is exciting, and it makes it a lot easier for people who don’t have eye issues to renew their prescription.

Unlike my previous experience, we have all the analytics we need here. Our data is used to find pain points for our customers. On this team, we essentially measure our quality by the percentage of people who complete the exam. If people find value in the exam and in getting a prescription, they will finish it if they are able. Our goal is to increase completion percentages. That drives our decision-making process.

Having data to back up what we do on a daily basis is really empowering. We see concrete proof that the decisions we’re making, the features we’re building, and the bugs we’re fixing all matter. Instead of a loose, fuzzy image of “some person,” we watch our metrics and have a clear idea of who our customer is.

Find Your Person

I have come to recognize how strongly I resonate with being able to identify the person. I want to know that my work matters to someone. But more than that, once you know whom you’re building, fixing, and doing things for, you’ll also have a better idea of what that person wants and needs — so you’ll be able to deliver true value.

All-in-one Test Automation Cross-Technology | Cross-Device | Cross-Platform

Rachel Kibler is a tester at 1-800 Contacts. She can be found online at racheljoi.com and on Twitter @racheljoi.

Comments