JMeter Concurrent Users Count Practice

Question

I have a web application which is working in production and there are 2000+ users of my application. Now I have to perform performance testing on the system so is there any idea how many concurrent users will be sending requests at a single time? It is presumed that not all the 2000+ users hit the server at the same time.

I am going to use JMeter for load testing of my system so how many concurrent users should I assume must be hitting the server at the same time?

Is there any practice being followed in the world or it varies application-to-application?

Any help is welcomed.

Answer 1

Is this a real-world question, or an academic one? Are you trying to find the answer for your particular application that you are testing? Or are you looking for a general formula as an answer?

If you have an application that is already in production, then you can analyze the logs and determine the usage pattern of your 2000+ users. That will tell you how many typically are concurrent users and can help model the answer to your Jmeter question. You might start by considering how folks are using your system now, and how you expect that to grow over time. That could provide you with some targets for your experiments.

To answer your specific question, in my experience, it varies application-to-application.

Answer 2

As far as I can tell, you are trying to answer the question "How do I take the information I have from usage of my current system - number of concurrent users and hits per second - and convert that into how many concurrent threads in JMeter?". I'll answer the best I can, if that is not your question, please correct me.

Tools like JMeter don't allow you to simply tell it "Hit this page X times in Y seconds", instead they execute tests as quickly as they can and you throttle the total number of hits per second by increasing and decreasing the number of threads.

You can have a number of goals for your stress testing, one of them may be "ensure we can continue to support the current production load". For this case, you generally need to experiment a bit with how many threads until you hit a number that will generate approximately the same amount of load you would see in your production environment. It is not guaranteed that if you run the tests for any amount of time that the requests per second will stay consistent. For example, if there is a slow memory leak in your application, the number of requests per second from your tests may decrease over time as it begins taking longer and longer to get responses back.

I have experimented with writing tests that will send only 1 request per X seconds so that I can "throttle" the requests to an exact number. You can do this by starting a timer, sending a request, getting a response and then sleeping until the timer hits X seconds. Then, you can pretty easily calculate how many threads you would need to execute in order to get an exact number of requests per second.

I will say that after experimenting with this approach I stopped using it because it really didn't add much value. I could usually meet all of my stress testing goals without using this trick.

Answer 3

What you're describing is a workload model, a model of the measured or expected load on a system.

Be wary about being fixated on the number of users, often a more useful metric to understand is the number of business transactions or the number of page/individual requests. Knowing how many users are on a system does not describe the load on the system, only the concurrency. For example; you could have 2000 users who log in over an hour and do nothing (2,000 total page requests), or you could have 50 users hitting a page each every second (180,000 total page requests).

The other part of a workload model is what business transactions you include or not include. Having all of your users visit the homepage of a website, when primarily they'll be buying and searching for products isn't very realistic. Aim to cover the transactions with the highest expected volume, those which are architecturally significant (e.g. are known to put a lot of load on the system), or are known to have prior performance issues. You cannot load test all the functionality of any non-trivial system. Here's an example of a basic workload model which used expected transactional throughput and converted that into the "transactions per thread per minute" required by a JMeter Thread Group:

Because you have an existing system you are at an advantage; there should be existing recorded information about the workload that you can work from. This might be reviewing access logs, speaking to key business stakeholders, whatever is required.

The second part of the problem is how do you apply that workload model to your JMeter scripts? It's not intuitive, but without any fancy add-ons I've developed the following approach:

The "Pacing" object is just a user action which contains a Constant Throughput Timer and a Uniform Random Timer:

The Target Throughput for the Constant Throughput Timer I've set to a variable called ${pacing} which I set at the Test Plan level.

The Random Delay Maximum (in milliseconds) field in the Uniform Random Timer I set to:

${__javaScript(1000 * (3600 / '${pacing}' / 60),maximum_delay)} 

... which calculates the maximum delay for +/-50% variability (a good target for pacing) on the figure you set pacing to. The Constant Delay Offset (in milliseconds) field in the Uniform Random Timer I set to:

${__javaScript(-('${maximum_delay}' / 2))}

This then randomises the pacing +/- 50% based on the the value calculated for maximim_delay. The weird part of how this works is that the constant delay is calculated and then the random delay is applied on top of that at runtime - it's not particularly logical to read it but it does work!

So going back to the workload model, for the "Browser" thread group I would set the pacing variable to have the value 0.22 times per minute (13.2 times per hour). The maximum_delay would be 272727 milliseconds (4 minutes 32 seconds). The Constant Delay Offset would they calculate -136363 milliseconds. What this means is that the pacing will be randomised between 272 - 136 seconds, and 272 + 136 seconds, or between 136 and 408 seconds.