depth vs. breadth

Question

I am on a small team (3 developers and one UI person, no QA resource). We have a template that we maintain and use to put together small registration sites for a variety of external clients. I am the UI person, and tasked with making improvements to the template.

For the testing phase, we will make use of our team plus some internal business users that also work closely with our product. This has worked fine in the past, but this time the changes are more widespread than usual and I am worried the test coverage will be spotty without some kind of test plan. I need many browser/OS/device combinations tested.

My question: For each tester, should I assign depth or breadth?

Is it better to divide up the test plan so that tester01 tests all of the scenarios on one browser/platform, and tester02 the same scenarios on a different browser/platform? OR does it make more sense to have tester01 test a handful of scenarios on all browser/platforms, and tester02 different scenarios on all platforms?

Answer 1

As always, Depends!

On the following factors that I can think of:

• The knowledge and expertise of the test group - Are all of them familiar with all features being tested? Are they all at the same level of expertise across your system? If the answer to both these questions is yes, then assigning each person to testing all the features on a dedicated browser might be an efficient way. This way they can test individual features as well be able to notice issues that span across the features or if a change to one feature results in an issue somewhere else in the application.

• How do various features under test in your application relate to or depend on each other? - If there is a high level of modularization, there would be a benefit in dividing tests by features rather than by browser.

• Time and priority - If time is constrained and you need all the features to be tested, something that might work is assign each tester to one or more browser and have them create a priority list of test cases. User A can concentrate of Set A of TC, while user B can concentrate on set B. This way each person does an in-depth testing for a particular set of test cases and a high level testing for rest of the cases. You can also go ahead and prioritize TC for a particular browser.

Will add more options as they come to my mind

Answer 2

To start with, what Suchit said.

Also, some other things to consider:

• How knowledgeable are your internal business users - will they use the site the way they usually do, or will they try different things?
• What level of detail will your testers and business users need? As a general rule, the more autonomous your testers, the more sense it makes to keep them in functional domains (e.g. Tester A does module 1 on all browsers, Tester 2 works with module 2 is better if you don't need to explicitly script all your steps, where it makes less difference for closely scripted testing)
• You mentioned these are registration sites - is it feasible to give each tester/user a set of data and have one tester check that all the data reaches its destination (presumably a database) correctly?

Without knowing your business domain, I can't give advice on how to prioritise beyond that if you have data on your most commonly used platforms it makes sense to prioritize those: as always go for the 80/20 rule (the 20% that gets 80% of the use - or as close to that as you can get). If Windows 7 PC with Internet Explorer 9 is your biggest group of users, functionality and appearance on that combination would be your first priority target, and so on down the line.

I'd also suggest if you run short of time to look at focusing on later versions rather than older ones, and classes of device before breaking into specific devices (e.g. "android version X, phone" rather than "HTC One", "iOS tablet" rather than a specific iPad model)