Dividing line for developers on cross browser testing before handing off to QA

Question

We're trying to define the process for cross-browser testing testing done by developers before handing off to QA. There's a point where you don't necessarily want a developer doing all the testing themselves, but they should be doing some basic verification on the work they've done. My question is what is a good way to define that point where the dev has done enough testing to hand off?

Answer 1

TL DR: Establish a baseline, tinker, and react.

In general developers should not worry too much about cross browser testing. Yes, it's a problem, and yes, they need to be aware of it, but I would keep it to a minimum. Let them focus on great functionality first, and not that developers shouldn't care about quality (because they totally should!) but in general they should defer that to the experts in it.

However, there are obvious cases where they should. If they're fixing a bug specific to a browser or browser version, clearly they need to test on that browser and version. Ideally they check the other browsers in this case too, since cross browser bugs are most likely to produce other cross browser bugs when you fix them.

Okay, so there's one side that says leave it all to the QA, and one that says obviously there's a little that devs have to do. Where is the happy medium? And the only answer to that depends greatly on your individual organization. You can learn this only through uncontrolled experimentation.

Start off with very little cross browser testing (and, in general, other types of unexpected developer testing). After gathering metrics for a few sprints or releases, identify the types of testing that would have caught these bugs faster. It might be cross browser testing. It might only be one or two browsers that often cause problems. It might be ones that interface with a certain other system. Modify the developer's test scripts they follow to include those specific kinds of tests. Then wait to see if your rates in those areas go down significantly. And the significantly part is important. If you've increased your developer testing time by 10% for a 10% drop in bugs in that area, I'd say that's a loss (because that developer time could have been spent on a dedicated QA person who would have probably caught more). If you drop by 50% in that area, suddenly it starts to pay off more. But what I think doesn't matter. It's about what's best for your organization that does.

Where the thresholds are will be determined by the costs incurred by your organization in terms of how much faster bugs get fixed, how many fewer bugs are reported, and more importantly how the schedule is impacted. (That's the goal, right? Reduce bugs so developers don't spend their time re-coding what they already did?)

Like so many things in SQA, you have to establish a baseline, tinker, and react.

Answer 2

This is a good question with no "right" answer as a great many factors come into play and the right answer depends on your individual situation.

Factors to consider are:

• developer:qa ratio
• developer group size
• company size and resources
• amount of javascript in the application
• current browser usage of your software
• developer uniformity of local browsers used
• automation and continuous integration of UI tests
• ability to run automated UI tests with alternative drivers
• tools used for cross-browser testing and their availability
• tools available to developers for easy cross browser testing
• availability of developers to spend time not coding the back end
• ability to run automated UI tests with alternative browser drivers

I call it 'the dev-qa slide' ;)