Looking for Clarity on testing terminology (Collaboration Request)

Question

I need to explain the testing terminology to others because I want everyone to be on the same page; have the same understanding. After continuous study of the testing world for over more than a month, I am coming to the conclusion that terminology in the testing world is nothing more than the wild west; do what you want to do and make up whatever testing terminology you desire. This is, to put it mildly, is frustrating.

I thought I finally figured out a way to understand all testing terminology ... successfully fit all the pieces of the puzzle together. Using a matrix. An analogy to this matrix would be a matrix of dog breeds; where x-axis (or y-axis) represents the name of the dog breed and the other axis lists characteristics which define that dog breed. So for example, Smoke test is a dog breed ... a characteristic would be that it is a type of functional testing, and by it's definition, it is also of type regression ... smoke test can be done on UI, API, or back-end area, it can either be testing acceptance or system requirements.

I want this way of understanding the different testing to be correct. This is the best way I have found to understand the testing world. The matrix helps you to ensure that you have covered every combination of testing.

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

QUESTION:

How Correct is this matrix? The amount of testing terminology in the testing world is mind boggling, overwhelming. It seems like the testing world is being made more complex than it needs to be because of lack of agreement on terminology making newcomers life worse than it needs to be. Collaboration would be greatly greatly appreciated.

[Testing classification matrix]: The header row for this table (smoke , sanity, security ...) are just a few examples of what could be on the top row. The Blue columns is supposed to successfully cover any imaginable type of test possible under the sun (nothing is left out ... everything falls under one or more of the blue categories).

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

REASON FOR TRYING TO ORGANIZE TESTING TYPES:

Ensure full gamut of testing is carried out to ensure full coverage. My initial thought/hope was that research within the testing world had been done on how to arrive at best testing process which best covers and protects the code and was hoping that a starting place would be by ensuring you are not leaving out any type of testing and to ensure the proper team is conducting the testing.

[Testing responsibility matrix]

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

After putting together this matrix, I did find other documentation attempting to show a comprehensive understanding of the testing types very similar to the matrix which I put together.

Answer 1

As you have discovered: there doesn't exist a comprehensive rule set of what to do in what circumstance. And as someone who is in software development and testing for over 20yrs, I don't think we will get to that ever. There are different 'schools' or philosophies in testing with different ideas on what testing means and how it could be organised. Besides that: things change and evolve. People have different ideas and opinions and experiences and needs... As there is no single coding language and practice that defines 'programming'.

This isn't the answer you are looking for, and I can only speak for myself, but as a "Context Driven Testing" (CDT) minded agile tester (if such a beast exists) I would not spent time to create a matrix to cover 'all' testing or search for the best practice: e.g. something that works always in every case. Every situation is different. Every organisation is different. Every team is different.

I guess there is a reason @trashpanda stated: "It's an absolute minefield" :-)

Note: have a look here: books on testing

Answer 2

I see what you're trying to get at.

In my own opinion, I've found that the easiest way to understand testing is by looking at test levels and test types. For example:

Testing levels:

• Unit testing
• Integration testing
• System testing
• System integration testing
• User Acceptance testing
• Operational acceptance testing

Testing types:

• Installation testing
• Compatibility testing
• Smoke and sanity testing
• Regression testing
• Acceptance testing
• Alpha testing
• Beta testing
• Functional, etc.

You can execute one or more test types within the testing levels (think of them as phases), and some will apply more than others. For example, you would exploratory testing during the system integration phase, but not during unit testing.

If you can get your head around the different test levels and test types on Wikipedia, then you can start looking at test approaches and processes afterwards... it's an absolute minefield.