How does response time react to increasing load?

Question

I'm testing my application locally with JMeter, and I have a doubt about how the response time changes as far as users are increasing.

If I test my service with, for example: 100 Threads, 100 Ramp-Up Time and 1 Loop,

I observe that the average response time is 5ms and the 99° percentile is 10ms.

But if I test my service with, for example: 200 Threads, 100 Ramp-Up Time and 1 Loop,

I observe that the average response time is 9ms and the 99° percentile is 16ms.

Is this a normal behaviour? Or should the response time be always equal?

Answer 1

Ideally you should have the same response time for any number of users.

In reality the situation is different, even applications are capable of scaling - the scalability factor will never be ideal so performance will degrade.

Normally if you start with 1 virtual user and gradually increase the load application will be handling more and more requests with the same response time (throughput increases). At some point throughput stops increasing, at the same moment response time starts increasing - this is called saturation point - and at this stage the application under test performs the most efficiently. If the number of users / requests per second fits into your NFRs and/or SLAs - you can stop testing here.

If not - you found the first bottleneck and now you need to find the reason and if possible fix it. The reason could be in:

• lack of resources (CPU, RAM, Network, Disk, etc.) - make sure to monitor essential metrics of the operating system plus your application specific metrics to ensure it has enough headroom to operate. You can do this using an APM tool or JMeter PerfMon Plugin
• incorrect configuration of middleware (application server, database, OS network stack, etc.). You need to conduct fine tuning of your application and associated software components
• inefficient code of your application itself - make sure to have profiler tools telemetry enabled during the load test run - this way you can tell what are the "heaviest" functions, long running DB queries, largest objects, etc. so you will be able to narrow down the investigation area and optimise the most frequently called methods.

Answer 2

It completely depends on your configuration and use.

Generally as user demand grows over time a company will add resources (servers, memory, faster database, key-value stores) to ensure that user response time stay within what they determine (or allow) to be acceptable limits. When front-end revenue starts to drop this is a big place to look.

So in the short term, yes one server getting increasingly load will at some point start to slow down. But on some servers that could start to show at 10 users and others not until 1000 users. It depends on threads, memory, garbage collection, what the code is and does and many other factors.

Overall a company should determine what it wishes its users performance to be and then provides the necessary infrastructure to support that as usage grows. With auto-scaling in the cloud this is increasingly easy.

Also - make sure that you are testing a production or production-like system to have any meaningful conclusions about performance. Also make sure that your jmeter threads provide enough similarity (i.e. variety) in content to real requests on the web.

Answer 3

It may. If it is increasing as observed it suggests that some part of your system under test (SUT) is approaching overload. A common approach to load testing is to run the test at a slowly increasing number of threads. Expect to see fairly constant response times from the start of the test until reaching some number of threads and after that the response times increase steadily. As the number of threads continue to increase the SUT may become overloaded and then you may get lots of failing requests or possibly the SUT may crash.

Running the test at two loads does not tell you much. Running it at several different loads (including running a test that steadily ramps up the load) allows you to draw a graph of load against response time and thus see easily where the system behaviour changes.