Jmeter's threads got stuck and never end

Question

I'm working on some Jmeter scripts, everything seems to work just fine; they got recorded and run properly (for a couple of users anyway).

The problem appears when I run the scripts for more than 50-60 users. In that case, the script runs, the listeners show me results and according the threads are finishing, they go to sleep, until the last 10% of them. At that point, the threads just hang and never stop. I don't see more results in the listeners, I don't receive any error or warning (neither on a dialog nor in the log panel).

Example

A script based on: 1) enter mainpage; 2) click on a link; 3) waits for load; 4) stop recording after page stop loading. Added some listeners (Tree view, Table and Summary report). Then, the script is run for one and 10 users/threads (launched at the same time), to check integrity. Everything works fine. The script is saved, the listeners cleaned up.

The "real test" is meant to be run for, let's say, 100 users (the idea is go up to 500 users in the next stages of the project). The script is launched, listeners collect data, log panel shows me the threads' transactions, some of the threads start to finish. But, the last 10-15 threads never stop (in average, 10% of the original number of launched threads). I wait, literally for 2 hours and nothing happens.

I tried on two computers, an Intel Core 2 Duo at 1.7GHz with 2GB of RAM, and then I tested on my main rig (Phenom X4 965, 4GB RAM), both with Windows 7 Pro. I run a third time on my notebook using Debian, Intel Pentium dual core at 2GHz, 4GB. In the three machines the exact same thing happens. It's something that doesn't happen for major sites (let's say, Wikipedia, NY.com).

So, everythings point to a server problem, but what could be happening? And why Jmeter doesn't inform me about the situation (it just hangs, nothing more happen)?

Thanks in advance.

Answer 1

AirieFenix, welcome to SQA. Here are some possible problems:

1. Your JMeter machine ran out of resources.
2. Your server is overwhelmed.
3. An intermediary (e.g. a proxy server or a load balancer on a DMZ machine) between your JMeter machine and the server is overwhelmed or is intentionally throttling network traffic.

Here are some ways to narrow things down:

1. One Jmeter hangs, try interacting with the server using a browser on the JMeter machine. If that works, the problem is probably with JMeter.
2. If the browser fails, try interacting with the server using a browser on a different machine. If that works, the problem is probably with JMeter, a resource problem on the JMeter machine, or an intermediary. On the other hand, if the browser fails, the problem is on the server or an intermediary.

If you narrow down the problem to the server (and not to an intermediary), you should start looking at bottlenecks on the server. For example, the server may be working as hard as it can (out of CPU cycles), or it might be trying to exceed the maximum number of database connections, or it might be waiting on a long-running database query. How you measure those things depends on information you didn't include in your question (e.g. operating system, database vendor, database interface, programming language). If you don't know how to do that yourself, one of your developers ought to be able to help you.

Answer 2

I'm taking a stab in the dark here; are there any errors in your jmeter.log file?

To narrow down whether or not there is a memory constraint make sure you edit the jmeter.bat file and increase the Java heap size. The lines that define the heap size are:

set HEAP=-Xms2048m -Xmx2048m

Change both values. In my example I've set a 2GB heap.

Also note that 32-bit systems are limited to 3.2 GB (ish) of memory total, so you will struggle to assign more than 2 GB (In fact from experience I got to ~ 1.7 GB)

The default is 512 MB (I think) which won't give you much to work with.

Answer 3

Can somebody suggest me, how to end the test, which have been started from console and (at some time point my object of testing crashes) requests aren't sending, console is freezing and I'm unable to interact with it. Main goal is to save the report, because I'm able just to close the window with console, but in that case I will lose the report, which is generating at the end of the test, as far as I know.