Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Problem

I have a system with two logical threads, living on different machines and operating concurrently and asynchronously. First one receives data from an external system and puts them into DB. The latter receives action requests from the external system, and based on the data stored in the DB, it takes one decision or another, and then sends requests to third-party system.

The problem we have is because devs assumed that: data has been put in DB before the second thread receives request. In practice, it happened to me that there was no data in DB and the second thread took the wrong decision that impacts third party system severely.

I wonder how to test against such bugs in a systematic way?

Proposed solution

The reason I have no found this defect before is that I took the same assumption the devs did: I waited for the data to be available in DB, before sending an action request. The way I found it was by (a) removing this wait condition from my automated test, plus (b) some coincidence: first thread was more overloaded than the latter and thus it delayed saving the data into DB.

My first ideas:

  • Remove unreal wait conditions from tests
  • Simulate such coincidences I described... But how?
share|improve this question
add comment

2 Answers

up vote 3 down vote accepted

When executing tests (either manually or automated), you must be careful about your assumptions.

Always explore if an application might be time/wait sensitive as you are conceiving and executing your tests.

  • Always consider - what would a single user do (regarding wait period)?
  • What would many users do?
  • What happens if I go far faster than typical?
  • What happens if I go far slower than typical?

Automation can help here, as long as you simulate closer to real-world conditions. With most tools, you can easily simulate many users arriving randomly, and waiting random periods of time.

share|improve this answer
add comment

Your ideas might be valid, but they use prior knowledge that might not be present for future bugs. Usually load and stress tests tend to catch similar issues, and so are long stability tests especially if they use somewhat above average load.

share|improve this answer
    
So stress, load and long stability tests can help discover not only performance problems but also functional problems, e.g., inconsistent data, etc.? –  dzieciou Apr 8 '13 at 6:23
    
Definitely yes, I found more than one bug this way –  Rsf Apr 8 '13 at 15:36
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.