How to determine breaking point of my application when performance testing from jMeter

Question

I am doing some performance testing with jMeter on my application. I've check marked Retrieve all Embedded Resources and Clear cache on each iteration.

For the thread group I've got: Number of threads 10, Ramp up period: 60, Loop Count 3.

Question

• How can I best determine the breaking point of my application? (i.e. the point at which it becomes unresponsive)
• Which listener will best be able to tell me the breaking point of my application (i.e. how many concurrent users it can handle)

Answer 1

First you need to define what the "breaking point" is. There are a number of possibilities. The obvious problem would be if the web server itself crashes and stops responding, but there are a number of problems that can occur well before that point that in most cases would still be considered "broken".

Some other problems that could tell you your application is "broken":

1. Response times become unreasonably long. A general rule of thumb is if a web page takes longer than 6 seconds to load, from a user perspective, they will lose interest and become impatient.
2. Some (but not all) requests begin returning non 200 response codes. You will need to set up validations for your responses in jmeter to watch for non 200 response codes.
3. The web server crashes and restarts automatically causing a few requests to fail, but all existing sessions to immediately expire.

None of the Jmeter listeners will tell you anything about the health of your application, it just tells you about some of the symptoms such as slow response times, or error response codes. In order to actually monitor the health of the web server, you need to monitor the system it is running on to watch memory usage, cpu usage, network usage, disk IO, etc. Many tools like Tomcat have some performance counters you can tap into and get more information as well.

To actually determine where you application breaks is a blurry line as well. If you immediately pound a server with 1000 requests, it could break, but if you ramp up slowly to 1000 threads it may handle it. In addition, you may be able to run 100 threads just fine, but if you run 100 threads for an hour straight, it may break. Again, the most obvious and the most common scenario is that you run a certain number of threads and it breaks when you reach some threshold.

To determine where that threshold is, you can often use a step load pattern where you start at some number of threads, and every X time period increase the threads by Y. So for example, every 10 minutes I might increase threads by 5, and I might run that test overnight and come back in the morning to see how far it got before the system "broke". The closest out of the box functonality in Jmeter to do this kind of load, would be the "ramp up". You would need to put the number of threads up to something very large (number depends on the project), maybe 1000+ and change the ramp up time to something very large like 43200 (12 hours) so that it continually ramps up until the application breaks.

Another option would be do multiple runs where you manually increase the number of threads for each run. For Jmeter, I would actually prefer creating or using a plugin to do the step up load since their out of the box functionality does not provide an ideal solution.

Answer 2

To determine the breaking point of you application increase the thread count gradually with each test run. At present you have it as 10. Now try testing with 20, then 50 in third iteration. Now suppose you find out that you application broke at 50 threads, then reduce the thread count to 40 and run the test again. Let's say your application works fine with 40 threads, then run it with 45 threads and repeat the process till you get the correct breaking point.

You can use the view results tree and summary report listeners for understanding the results better!