0

If I am running my performance tests in test environment that has 50% of hardware we have in production, can I somehow extrapolate results from test environments to production? What are possible projection models?

For instance, one common projection model is linear extrapolation: if test environment can handle 1000 users per second (response time < 1s), then I can say production can handle 2000 users per second. But I guess it many cases it might be oversimplified.

  • 2
    Extrapolation is rarely a good idea in performancetesting. I have seen situations where more memory means less performance. And its nog only hardware that limits performance. – Patrick Jun 26 '18 at 12:39
  • @Patrick can you tell more about "more memory = less performance"? Can you give an example? – dzieciou Jun 26 '18 at 12:58
  • @Patrick, sure, also software implementation can limit performance, but given two identifcal implementations, hardware is the one that matters. – dzieciou Jun 26 '18 at 12:59
  • 1
    The more memory example was related to garbagecollction that took more time. It was for a system where there were very strickt requerements on responstime. And during every garbage collection those where not met. – Patrick Jun 26 '18 at 13:28
  • If a non hardware element (like max number of database connections setting) is the limiting factor, increasing hardware will have not increase capacity when factor is not changed. – Patrick Jun 27 '18 at 5:49
2

Maybe

The one thing you can say is that if your test environment can handle 1000 users/second with a response time < 1s, then your production environment can handle at least the same.

You can't do a straight linear extrapolation unless every part of the production is x times as powerful as the test system. That doesn't just mean double the memory and double the CPU speed. It also means double the BUS limits, double the disk access speed, double the network capacity, double the task switching speed... and possibly more depending on precisely what resources the system uses.

The prod system may have limits that aren't reached in the test system, such as request limits, queue limits, and so forth. Those can be hard limits based on the operating system, or they can be configurable but cause other issues if they are changed from the default.

The only thing I would say for certain is that if the test environment handles 1000 users/second with response time < 1s, the production environment will handle more than that. How many more will depend on too many variables to be able to give a confident estimate.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.