Trying to simulate users logging in but staying idle

Question

I'm trying to simulate the scenarios where 1000+ users log in at once, and stay logged in for a set amount of time. While those 1000+ users are still logged into the server, another set logs in as well.

Essentially a failover test? Trying to see how much the server can hold before it just dies.

Will the Ultimate Thread Group in jmeter help me achieve this? I read the documentation, and I'm a bit confused with the "Hold load" part. To me, it means that the http request will last for the set amount of time (which is what I need - to simulate users staying idle). Is this correct?

Answer 1

Typically on a Web application an idle user will consume almost no server resources. The constraint will be the amount of memory or database space used to store the users session. If the server can therefore support 2000 users you shouldn't have any problem.

That said a more real concern should be the impact on performance of the server due to the extra load and in particular how the 'login storm' is handled as this can consume significant resources causing timeouts on connecting and spikes in load. Sometimes it pays to have a random back off built in to spread this load.

To test this realistically your existing users should be active and responses timed In order to determine impact of fail over event. On the other hand if it's a truly rare event depending on the nature of your application you may only worry that it all comes back up in the end...

Answer 2

Agree with Andy here and think you need to take great care ensuring that the test reflects what may actually happen.

First I'd look at session timeout and memory release. Ensure that your first batch of users remain active preferably doing something useful say every 30 seconds. This will ensure no session timeouts and also give you some measurable responses.

Then log on further users and as Andy says look at the initial users response times and the new users log on times, and see if there is any deterioration . If there is then check if there is memory available, any shortage of cpu, and any excessive system activity related to managing the large number of users.

You should look at how much cpu is used for actual user work and how much for system overhead (not always as easy as it sounds).

I have seen live systems lose part of their concurrent users due to sub network failure. The users when the network comes up try to log on again. It is interesting to note if they find their previous sessions (assuming these have not been mopped up). And of course how long the now orphaned sessions last.

I haven't used it but from the documentation the Ultimate Thread Group in jmeter should allow you to set up a realistic test. But do allow sufficient time for the workload to stabilise before firing the second batch.

Answer 3

It depends on what do you mean by stay logged.

To simulate 1000+ unique virtual users you need to provide separate username/password combination. It's likely that you will also need a HTTP Cookie Manager to represent user cookie.

In regards to Ultimate Thread Group, the Hold load bit means that test threads will be looping over for specified amount of time, all the requests under Ultimate Thread Group will be executed continuously for the time slot set.

If you need just log in -> sleep -> log out a Constant Timer may be what you looking for.

If you need to specify the exact load in "requests per second" - consider using Constant Throughput Timer.

If you need to release all 1000 threads at once - use Synchronizing Timer.