In preparation for testing, some testers will translate the requirements to test cases documenting the input and expected output in order to cover positive, negative, and boundary testing. Test cases can be time consuming to write and maintain and will not always allow the flexibility to introduce different testing scenarios. Over time the test cases can provide diminishing returns since the testers may be retesting the same keystrokes on a mature piece of code whereas exploring different testing paths will identify new bugs.

Testers can choose different testing options such as exploratory testing allowing them to explore areas of the application they feel may be vulnerable. At times a more structured approach is needed to help the tester organize their testing to ensure they adequately cover the features while allowing them the flexibility of exploratory testing.

What is a Test Matrix?

Testing-Matrix-Software-Testing

A testing matrix can be created for features that undergo regular testing (ie., regression testing) or for new features that require multiple rounds of testing. The goal is to reduce the need to gather this information each time the feature is tested allowing the focus on generating test ideas and performing testing. This approach is also helpful for features that are tested across multiple browsers or modules within the application. A testing matrix can be created using a spreadsheet or table format to take advantage of the row and column relationship providing a way to keep track of completed testing. The rows define the feature’s options and the columns represent the testing areas. This approach reduces the time to create and update the testing matrix through consolidating the feature’s options across testing areas. The tester can bundle multiple tests into one test scenario or she can test them individually based upon her test ideas, experience level in testing, and knowledge of the feature. This allows her to change her approach when retesting a feature and allows other testers to take different testing paths providing better overall testing coverage.

Creating a Test Matrix

If you are testing your application in a web‐based environment and you need to test the ability to print a report using different browsers, your testing matrix may look as follows: The above matrix provides initial guidelines on testing coverage and the supported web browsers. With this information coupled with knowledge of the application and history of fixed bugs, testing ideas can be generated. This matrix can be customized to include a section to document the testing ideas and additional columns can be added such as notes. A testing matrix should be created based upon the needs of the features being tested, the risk level, and how much documentation is helpful.

Variation of a Test Matrix

The above test matrix example defines specific feature options to be tested allowing the tester to generate their test ideas. Another variation would allow the tester to document their testing ideas. Your testing matrix may look as follows:

Benefits of This Approach

Testing-Matrix-Software-Testing2

This approach can be helpful when there are a definite set of features that must be tested such as in regression testing or when the same set of options are tested across multiple browsers or testing areas. While the tester is executing their testing ideas, this approach can help them organize their tests and document what they have tested allowing them to review their progress.

Because it is not always possible to run all test ideas across all web browsers or testing areas, a test matrix allows the tester to assess how much testing to perform. This information can be reviewed with stakeholders or other testers to discuss test coverage.

Disadvantages of this Approach

There is a cost associated with creating the test matrix by ensuring the correct information is captured and maintaining it if the features change. There is the risk that a tester will test only within the constraints of this structure and could miss important tests. Training needs to occur to ensure testers use the testing matrix as a way to organize their testing and not constrain the testing.

Conclusion

Whatever approach is used to organize testing, it is important that the testing is not limited by the approach. The goal of the test matrix is to provide organization and not become a checklist that is blindly followed. The value of creating a test matrix must outweigh the time to create and maintain if it is to be reused. A test matrix can be beneficial for regression testing and when testing a feature across multiple browsers or testing areas. In the end, how a tester organizes her tests can be a personal preference or company‐driven especially in regulated environments.

https://i0.wp.com/www.testingcircus.com/wp-content/uploads/Testing-Matrix-Software-Testing2.png?fit=824%2C385&ssl=1https://i0.wp.com/www.testingcircus.com/wp-content/uploads/Testing-Matrix-Software-Testing2.png?resize=150%2C150&ssl=1Bernice Niel RuhlandArticlesTesting ArticleIn preparation for testing, some testers will translate the requirements to test cases documenting the input and expected output in order to cover positive, negative, and boundary testing. Test cases can be time consuming to write and maintain and will not always allow the flexibility to introduce different testing...