Note: I've asked this over on SO and p.SE and it got closed. I'm not trying to game the system, just looking for a home :)
I'm seeking a little "wisdom of crowds" estimation or a pointer to an authoritative reference regarding insight into best practices for allocating effort into testing.
While there is a wide range of different types of testing, given the following list:
- Manual exploratory testing (random ad-hoc testing either thru the user interface or APIs)
- Manual end-to-end testing (predefined user-interface driven tests)
- Automated end-to-end testing
- Manual integration testing
- Automated integration testing
- Automated unit testing
and given the following technologies and application:
- some kind of LAMP stack
- consumer facing ecommerce website
- REST based business logic middle-tier for use by the website and internal facing back-office applications
- back-office applications (inventory/order management, warehouse logistics, BI)
and given the following organization:
- product management defining the user-facing website
- the middle-tier is java
- back-office applications are currently PHP and migrating to java
- back-office apps are defined by the internal consumers (purchasing, finance, warehouse, etc).
Let's think about the total number of hours spent performing testing activities (developers creating unit and integration tests, test engineers creating and executing test plans, test engineers creating tools and harnesses and running them).
Does anyone have a pointer to an authoritative study regarding this kind of resource allocation?
At my company, for example, I'd guess this is our current mix:
60% Manual exploratory testing 10% Manual end-to-end testing 10% Automated end-to-end testing 5% Manual integration testing 5% Automated integration testing 5% Automated unit testing
As far as my suggestion for a better practice, I'd prefer to see something more like:
5% Manual exploratory testing 5% Manual end-to-end testing 10% Automated end-to-end testing 5% Manual integration testing 20% Automated integration testing 55% Automated unit testing
I'd be most appreciative for any help in finding some shoulders to stand on to help guide my team.
Postscript: I'm confused about the whole "subjective" thing, and attempted to craft this question in line with the six guidelines for SQs (note that this question was closed on programmers, so that's not the answer). Anyway, given the six guidelines, I thought this satisfied them. Just to review:
- inspire answers that explain “why” and “how” -- admittedly a little weak here, but I'm pretty clearly asking for "how"
- require long not short answers -- again, someone could answer with a string of five numbers, but I would hope to see some insight into why they landed on that particular mix
- have a constructive, fair, impartial tone -- I hope that providing the estimates of my current and hoped for situation don't torpedo this characteristic
- encourage experiences over opinions -- I think this is pretty clearly asking for what folks are doing and what's working for them
- insist opinion is backed up by fact -- I would hope that folks would tell the truth about their testing mix (analysis identifying why their mix is successful would be a bonus, see #1,2 and 4)
- not just mindless social fun -- I'm hoping to learn something here