Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

We are in the process of changing our barcode format to Code 128 and use all kinds of "special" ASCII characters in the barcodes to maximize the data we can encode in it. A barcode scanner works like a keyboard device and sends the string as key-presses to the application. The application itself can see the difference between normal key-presses and those of the barcode scanner and handles it accordingly and uses the data to fire an action.

During exploratory (manual) testing I found two combinations that would scan incorrectly, they have been fixed. But now I am afraid that more combinations might give trouble in the future. My developers tell we have around 9.333 × 10^157 possible combinations. This is impossible to test manually. Testing a barcode means you need to print, scan it and verify the result.

If we release this new format and a client finds a combination that doesn't work, it could lead to a fix that renders all (or a lot of) previous bar-codes invalid.

Question: How should I handle a possible unlimited combinations like this, how far should I go in testing to feel comfortable enough to release a new feature like our new barcode format?

Update:

  • The found issue was a Operating System keyboard setting issue ' + e leads to é instead of 'e
  • In the end we have automated with AutoIt and tested all up-to four character combinations after each other. And the Min/Max value boundaries.

Learnings

share|improve this question
1  
Wont using the Equivalence Partitioning and/or BVA test techniques help? Just wondering. –  Yash Apr 25 at 9:52
1  
Try looking into combinatorial testing –  user246 Apr 25 at 11:54
1  
was there anything 'special' about the two combinations you found? Can you assume that the straightforward bar codes ( using 'standard' characters and numbers ) are OK which would drastically reduce the combination? –  Phil Kirkham Apr 25 at 17:27
    
Is it possible to automate the part that scans or does it have to be done manually? –  Phil Kirkham Apr 25 at 17:28
    
Yes, we can leave out all of the alphanumeric only cases, smart! :) In our main desktop application I think we can automate and simulate the keystrokes a scanner generates. If we accept the barcode scanner works as intended this should be no problem for the tests. The barcode scanner itself has never given any problems in the past, yet. We also have more complex situation where we have a windows mobile phone with a scanner build in which is harder to automate, but I am confident that if the desktop barcodes work it will also work on our PDA's. –  Niels van Reijmersdal Apr 25 at 17:49

2 Answers 2

up vote 3 down vote accepted

If at all possible, this would be an ideal test to automate. If you decide to do that, below is an article by James Bach and Patrick Schroeder that is illuminating. The core idea of the article is that selection of a good pairing algorithm is not straightforward and a simple randomization approach is a good technique for getting effective coverage, whether doing manual or automated testing.

Good luck! I hope this helps.

http://www.testingeducation.org/wtst5/PairwisePNSQC2004.pdf

share|improve this answer
    
Good point and a great article, thanks. If we accept the barcode scanner works as intended with Code 128, then we could simulate the barcode scanner with something like AutoIt autoitscript.com , I will try next week :) –  Niels van Reijmersdal Apr 25 at 17:37

First of all, this is NOT a combinatorial testing problem. Combinatorial tests are based on 2 or more input parameters that accept multiple argument values.

In this case, you have a single argument which is the barcode. Your problem is a permutation problem. Essentially, you are attempting to explore various permutations of the Code 128 barcode format.

In this situation, I would create a simple random barcode generator and loop through as many permutations as feasible or that provide confidence to your team.

The random generator would be reasonably simple to make given there are 3 possible start characters, 2 stop characters, 103 characters in the encoded data, and a checksum.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.