4

I'm in a disagreement with my superior.

Background:

Basically, for our application, which is very complex at times we have:

  1. App specification in .doc documents

  2. JIRA requirement tickets that repeat the spec but the description goes into greater detail and may (or may not) contain technical implementation details.

  3. Test Cases in a separate system

The Test Case format is simple:

Step   | Expected result
------------------------

Issue: A lot of our test cases (1000+) have the business logic copy-pasted or paraphrased in our test cases. Example:

     Step                 |       Expected result
    ----------------------------------------------
1) Click button           | The field should become disabled 
2) Click something else   | (because if A is true, then B affects C, and 
3) Check some other field | because of that the field becomes disabled)

My reasoning: That's unneccessary duplication (the phrase in the brackets above) and should be cut out (and I did, hundrends of them) and if the tester needs to understand further what he/she is doing - JIRA ticket links are a click away (in test cases themselves or in the head of test suites).

Superior's reasoning: Without the additional explanation the tester then does not understand the business logic and just blindly executes clicking, which is bad. Test cases should educate the tester (especially new testers) as they execute regression testing.

My answer to that: I'm all for the tester knowing the business logic (otherwise the tester is mostly useless) but a test case is not a place to keep business logic, it needs to be concise and to the point, otherwise the tester is forced to read through much more text and thus manual regression testing becomes much slower

Discussion hit a dead end at this point.

  • I understand this question may attract "opinion-based" answers, but perhaps there is a widely-accepted concept/approach? – Andrejs Sep 20 '16 at 11:15
  • Vote to close : Purely opinion based Q. – NarendraC Sep 20 '16 at 12:13
  • 2
    +1 for question. Vote to leave open: yes, it is opinion-based but this is common problem and even opinion-based solutions have benefit for community. – Peter M. - stands for Monica Sep 20 '16 at 14:20
  • 1
    Can you elaborate on "the spec + JIRA ticket links are all over the place"? That would make a big difference to how I'd answer this. – Kevin McKenzie Sep 20 '16 at 19:58
  • @Kevin, thanks for pointing that out, question edited – Andrejs Sep 21 '16 at 6:29
8

The short version - It depends

The longer version

There are merits to both approaches. Some general heuristics:

  • Everything in one document is a priority - if your company believes it is important that testers have all the information they need in the test case document, then they will want business logic in the test cases.
  • DRY is a priority - if your company is a proponent of Don't Repeat Yourself, then they will want the test case document to reference the business logic rather than repeat it.
  • Testing is/may be outsourced - if there are plans to outsource testing, there may be a desire to include all the logic in the test case documents in order to limit the amount of internal documentation that must be sent out of the company.
  • Automated regression is planned - if there is a plan to automate the regression testing, the business logic may be needed in the test case documents as a guide to the automation team.
  • Other documents don't have the correct detail - this happens more often than anyone would like, but it's still an issue: testers working on a test case notice that the application logic doesn't cleanly match the documentation. They don't have the authority to update the documentation, and they don't know whether it will be updated before the next round of regression testing, so they update the test documents so that whoever tests next time knows what should be happening. Sooner or later this becomes organizational policy and business logic is expected in the test cases.
  • Nobody reads the specifications - Again, not ideal, but I've seen this happen more times than I care to count. If nobody reads the specifications, it's possible the only time the business logic gets looked at is when the testers are running the test cases.

Ideally, everything should be documented exactly once and then referenced as many times as needed, but life is not ideal and there could be any number of good reasons why your superior wants the repetition - perhaps there's a history of testers blindly executing test cases and this is part of the mitigation strategy.

I'd start by looking at all the reasons your superior wants/needs business logic in test cases, then try to work with him to find a potentially better way to deal with the problems he's trying to solve.

  • Great answer. Thanks. Note re Automated regression - IMO, test automators actually just need to know what to click and what then to assert. If they want to know more - link to the requirement ticket is there. "everything should be documented exactly once and then referenced" -- yes, that's what the team should strive for, and not argue against the DRY principle with "nobody reads that bit". – Andrejs Sep 20 '16 at 12:31
  • 1
    Also, spreading duplication over a 1000 test cases causes 1) maintenance hell 2) Invalid bugs because "oops, someone forgot to update that duplicate business logic paragraph in the test case" – Andrejs Sep 20 '16 at 12:32
  • @AndreyDoronin - I disagree that test automators only need to know what to click and what to assert. That produces maintenance hell in the automation code (I speak from experience here...). Automators who understand the business logic can craft their code to be more efficient and make sure they assert the correct things. – Kate Paulk Sep 20 '16 at 12:35
  • @AndreyDoronin - you won't see any arguments from me about maintenance hell! As I said, ideally everything should be documented exactly once. The issue is that your superior has found this maintenance nightmare to be helpful, so is keeping it. You need to know the underlying problem before you can help to move away from what is almost certainly a flawed mitigation strategy – Kate Paulk Sep 20 '16 at 12:37
  • 1
    @AndreyDoronin - there was a problem, and it got better by putting business logic in the test cases (this is the mitigation strategy to make the problem less problematic). It's flawed because it causes maintenance hell etc. but you're not going to change it unless you can work with your superior to find a better way to fix the problem he's trying to fix with business logic in test cases. – Kate Paulk Sep 20 '16 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.