2

Recently we found a concurrency problem in our Web app. It came out that a class in a third-party library is not thread-safe and when it is shared between multiple threads, then access to it must be somehow synchronized. This finding was a by-product of load testing of our Web application with many concurrent users.

I wonder if for finding concurrency issues I should run more focused tests in addition to load tests? Or maybe it is enough to run traditional performance tests? Or maybe I should change something in performance tests so they detect such issues better? E.g.,

  • make test scenarios more complex, including more endpoints in one scenario.
  • run threads with random delays to increase the chance of interesting interleavings between accessing the shared resource.
Improve this question

2 Answers 2

Reset to default
2

This is a black art. It is hard to test for concurrency problems in a methodical way. In other words, it is hard to construct functional test cases that will verify whether concurrency works correctly. We tend to find concurrency problems as a side-effect of load testing.

I think it is a good idea to combine as many kinds of operations as you can in the same test, i.e. instead of all threads doing the same thing, different threads do different things.

The time intervals during which concurrency problems occur may be very small. Factors such as network latency and thread contention will already introduce random delays. I'm not sure that adding intentional delays will matter.

Improve this answer
1
  • I like the wording "black art" :-)
    – dzieciou
    May 2, 2015 at 14:45
1

I'm a strong believer of the approach of running performance tests on production environment complete replica as it will allow to cover the following cases:

Once you encounter a performance problem, bottleneck, degradation additional investigation will be required to get to the bottom of the problem, identify point of failure and report or fix associated problem.

Improve this answer
2
  • Thank you, Dmitri. I miss, however, "concurrency" word in your answer. How does it answer my question? Do you mean that only on production-like environment I will be able to discover those concurrency issues that may appear in production? If I have limited budget then what aspects of prod infrastructure I should copy into my test environment?
    – dzieciou
    May 3, 2015 at 9:33
  • 1
    I meant that your load scenario should be as realistic as possible. Like 20% of users are just browsing having reasonable think times, 30% are search something, 15% are checking something out, etc. I don't know specifics of you application to provide more or less exact values. Usually it's enough to simulate the peak expected load and see how it goes.
    – Dmitri T
    May 3, 2015 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.