2

When creating a negative automation test script, should this be separated from my positive script, or can I also include the negative scenario in my script. I am using protractor for this. Thanks

Improve this question
1
  • Do you mean happy vs sad paths? I'm not sure exactly what negative means.
    – Michael Durrant
    Jan 24, 2016 at 13:27

2 Answers 2

Reset to default
4

In my opinion you should not mix positive and negative paths because every test has to be independent and give a clear result.

Suppose that you create 100 scripts both positive and negative and all tests with negative paths fail, in theory you'll never know if they would fail even with positive ones until you fix your code. My suggestion is having a reasonable granularity of the test suite but focus mainly on maintenability of the suite itself e.g. balancing the amount of automated and manual tests.

Improve this answer
1
  • so it means for a login app - positive test would be the entry of valid credentials(this would be now a separate test script). Then I would also create a separate test script for negative test which would be the entry of invalid credentials. I would execute them separately. I am not sure if this is my correct understanding of what you said? Thanks :)
    – Marj
    Feb 8, 2016 at 21:54
4

The trade-off is between script length and diagnostic value:

  • Script length: one script with everything in it may be shorter (and should never be longer) than a positive script plus a negative script, because multiple scripts will duplicate setup steps. Because it has fewer steps, the everything script will probably be faster to execute than multiple scripts.
  • Diagnostic value: diagnosing bugs is a narrowing-down process; you try different variations on a test until you find the shortest possible variation that reproduces the bug. The script with everything is probably not the shortest possible variation, and so it has less diagnostic value than the positive-only or negative-only script.

None of us can tell you how to make that trade-off. If you have lots of developers and only one tester, you may choose to optimize for script length. If you have enough testers, and it tends to take a long time for a developer to narrow down a bug to its cause, you may choose to optimize for diagnostic value.

Most likely, you have both problems, i.e. not enough testers and too long to narrow down the bugs, and so you will have to make the trade-off using your own judgement.

Improve this answer
2
  • I am the only tester in the team, and 5 of them. Could you please explain more about the Diagnostic value trade-off? I am already thinking of separating positive and negative test scripts.
    – Marj
    Feb 8, 2016 at 21:50
  • 1
    Imagine two scripts: one with only a single step and one with ten steps. If the single-step script fails, you know which step triggered the failure. If the ten-step script fails, you don't know which of the ten steps triggered the failure. You may need to try variations of each of the ten steps to narrow down the problem. In that sense, the one-step script has more diagnostic value than the ten-step script.
    – user246
    May 3, 2018 at 10:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.