What experience do you have with the idea "QA designs, Developer develops"

Question

There is the opinion that QA should design cucumber scenarios, define the right data for tests, communicate with business and end users to fully understand use cases, focus to semantics and look also for quality assurance of governance processes and legal compliance – but should not do development of the automation code themselves, instead driving the developers to do so because the developer has the deep knowledge of product APIs and trained programming know-how. Let’s call this approach “QA designs, DEV develops”.

While fully understand this philosophy of “separation of concerns”, but I have my doubt here. The implementation of cucumber steps is development as any other development. So the QA engineer needs repetitively change the code & run the code in IDE until the cucumber script is perfect structured and working. It is not possible to create a clear, structured cucumber scenario from the scratch! That means the idea that “QA designs, DEV develops” is not applicable in practice – it’s just a theory.

To be more specific: QA starts with a raw cucumber scenario that they received from business analyst. Now, the QA engineer needs to refine this scenario, apply the right parameters, add new validations, group statement accordingly, etc. As any development, it is essential to run the code after every small change. That way the QA moves successively from imperative notation to declaratively styled scenario formulation.

Or what do you think? It is possible to separate clearly these responsibilities? Maybe even some real experience?

Answer 1

I read the "design the cucumber scenarios" as being just one tiny step past writing explicit test cases.

It is not possible to create a clear, structured cucumber scenario from the scratch!

I disagree with this assertion. If someone's written strong Cucumber steps, the implementation seems like it should be pretty trivial (insert the appropriate selectors, etc, especially if you're in a team that isn't defining these contracts up front so that tests can be written parallel to dev).

The part I'd push back on as an SDET/SEiT is that I wouldn't be comfortable handing off the implementation of the automation at that point, especially in a BDD world, where the steps/definitions can often mask underlying behavior, but if I was a manual tester, this might be the only option.

QA starts with a raw cucumber scenario that they received from business analyst.

There's clearly a question as to what QA is adding above and beyond the BA - depending on the company, you might push for the BA to take on more of the test case enumeration (and eliminate the QA writing the detailed cucumber specs).

Answer 2

QA certainly does not design anything but some tests.

Design of a feature is responsibility of people who have substantially deeper knowledge about the inner workings of the system, like system analyst, system architect, database and system administrators (for their parts) and senior, most experienced designers. Then, developers design the parts they develop withing the limit of the overall design.

These same people are also better positioned to describe the scenarios which needs to be tested.

Most of these tests will be manual.

Some (but certainly not all manual tests) of the recommended test will be deemed a good candidate for automated regression test. Automated regression test is code, and needs to be developed and maintained exactly as any other code: Tests should fit into overall architecture, share common libraries, etc.

I am talking about automated regression system level tests (end-to-end tests) which might be responsibility of QA (or devs, depending of the company approach to quality) and exercise the tested system from the POV of a user (only the surface exposed to the user). There should be (and your dev team should start with - without any involvement of QA) deeper test first: unit tests, which test small parts of the system and mock everything else. Unit tests are your first layer of defense, because they can pinpoint the error much closer.

Automated regression tests usually test "happy path" (with some small detours), to make sure that basic functionality of your system, your bread-and-butter, is functioning correctly. Usually it is not cost-effective to automate (system regression tests) very peripheral functionality, or all possible combination of user inputs, because developing and maintaining system test is time consuming: If you add some feature, you need to review and possibly change all the test which use the page where change happened.

But certainly you want to automate as much of the functionality you test repeatedly in manual mode, that is obvious.

So the more system level regression tests you have, the more of your effort is on maintaining them (making changes in response of the newly added functionality) and less efforts are remaining for developing tests for new features, so obviously you will reach the point of diminished returns.

I read somewhere that any system-level regression test which did not indicated a bug (did not break as a result of a bug) is a candidate to be removed, because maybe some other test is already testing the same functionality and you have duplicated efforts (double the maintenance with no increased coverage).

Code coverage is another good tool to consider even evaluating which test to add (and which to remove).

Answer 3

The process you describe sounds like an archaic waterfall-process with a lot of handovers from role to role. Which tends to be very ineffective.

The best architectures, requirements, and designs emerge from self-organizing teams.

http://agilemanifesto.org/principles.html

Figure out together who has what responsibility in the team. Personally I prefer if QA and Developers pair both on programming as well as test-design. Quality should be a whole-team-approach.

Same goes for requirements, work together with stakeholders, business-analist, developers and QA at the same time. For example in a Specification by Example workshop, a session that could result in Cucumber like scripts.