when-to-stop-testingOften we are asked to explain what a tester does and whether someone might be a good fit for the job. It can be challenging to explain to someone without making it sound too simplistic such as a checking job or so difficult that they are intimidated. Over the next few months, I would like to address a few questions and challenges I face as a Software Testing Manager. In the October and November 2012 editions of Testing Circus, I addressed the question “do you think I could be a tester?” In the December 2012 edition, I addressed the question “how can I learn to test?” In this article I will address the often-asked question “how do I know when to stop testing?”

I will address this question from what you can do as a tester and if you are mentoring a tester. New testers typically want to test everything to make sure nothing is missed. However, it is impossible to test every test case from both a time and cost perspective. So how do you know when you should stop testing? Some of the answers may be driven by your company’s policies and procedures. Below are a few general guidelines that you might find helpful.

Set Time Guidelines

If you are mentoring a new tester, provide her with a block of time to perform the initial tests. Then review the test results with her to determine next testing steps based upon the requirements and what she learned by performing the initial tests. Continue this process until testing has been completed. By going through this process with the new tester will help train her how to think through the requirements, testing results, risks, identifying new testing ideas, and determining when to stop testing.

If you are the new tester, talk to your manager to understand who will review your test results and how you can work closely with that person. Sometimes the initial testing assignments are small enough to be completed in one or two testing sessions. As the assignments become larger it is beneficial to test in a continual feedback mode with a more experience tester.

Risk Level of the Testing Issue

Understanding the risk level of the testing assignment can help determine how much time to spend testing. The higher the risk typically means you will spend more time testing. However, that needs to be balanced with the scope of the problem. For example, a high risk change that touches many areas of the product will require more testing time than a high risk change that touches a very small, isolated part of the product.

Diminishing Returns and Use of time 

It is important to understand the law of diminishing returns. The definition from the Merriam-Webster online dictionary:

1. A rate of yield that beyond a certain point fails to increase in proportion to additional investments of labor or capital
2. Benefits that beyond a certain point fail to increase in proportion to extended efforts

How I would loosely translate to testing: spending more time performing additional tests does not necessarily equate to added test value. There becomes a point where sufficient information and value has been gathered through testing and spending additional time testing will not necessary produce more valuable information. Instead, spending this additional testing time on a different testing problem could be a better use of time. When thinking about what is the best use of my testing time, a tester needs to consider time constraints and what testing problems are still outstanding. For example you have to test problem A and B. You have already spent a lot of time testing problem A and have gathered a lot of valuable information. At this point, is it better to spend more time performing additional tests or should I move on to test problem B?

We want to stop testing before we encounter diminishing returns on our testing time.

Experience and Time

It can take many years to learn the art of software testing and a good tester is always learning more about the product and challenging her testing skills. As a newer tester, it is important to ask questions about what you are testing because this will help you understand
how much time to spend testing. Below are a few discussion questions you may have with a more experienced tester for feedback on your approaches and whether you are performing too much testing.

• Based upon the requirements, I am planning on performing the following tests. Do you agree with my approach?
• Based upon what I learned during testing, I am going to perform the following tests and I am not going to perform the following tests. During the conversation, you will discuss what information you gathered during testing to identify the additional tests.
• As an overview of my testing I performed the following tests and I did not test the following functionality. Do you agree with my overall testing approach and are there other areas I should test?

The best way to learn is by hands-on-testing with reviewing your results and questions with a more experienced tester. Do not be afraid to ask questions, as that is how you will learn.

This article was published in our January 2013 Edition.

https://i0.wp.com/www.testingcircus.com/wp-content/uploads/when-to-stop-testing.jpg?fit=500%2C217&ssl=1https://i0.wp.com/www.testingcircus.com/wp-content/uploads/when-to-stop-testing.jpg?resize=150%2C109&ssl=1Bernice Niel RuhlandArticlesTesting Article,When to Stop TestingOften we are asked to explain what a tester does and whether someone might be a good fit for the job. It can be challenging to explain to someone without making it sound too simplistic such as a checking job or so difficult that they are intimidated. Over the next few months, I...