3

First time I'm doing load testing on our server. I wrote a small program to keep posting some JSON files with httpclient.

However when threads are more than 50, the program says

INFO: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect.

Meanwhile, if threads are around 20 to 30, the server returns me 200(OK).

Does it mean that basic load testing is already failed?

OR still I need to analyze CPU or memory or other performance detail, then I can say the load testing is failed, if the CPU or memory are too high.

Not sure how to determine if the load testing is failed. Need some suggestion.

3

Has the test failed? That depends on the expected use of the app in Production.

If 20 threads is acceptable, then you have probably proved the app is performant. If the expected production load is greater that 50, then it isn't ready.

Firstly, you need to know what the expected load will be. Once you have that, you have something to benchmark against.

You say you get a 200 for 20-30 threads, but what I would do is time how long it takes to responsd for a single threaded call. You should in your java app be able to find some way to record how long every call takes.

Then, as you ramp up load, by running several tests you can see how the response time degrades and at what point the app fails.

Regarding the server, yes you should monitor server resources such as CPU and memory usage.

If the server is unix or Linux, then I'd suggest installing NMon. This is a fab application. This app will continually monitor server resources whilst a test is running. You can define how frequently to check resources, e.g. Every 10 seconds for a 30 minute period. Once your test has finished you can pull the file off the server. There is a spreadsheet you can download which will read an nmon result file and give you a nicely formatted output.

When gearing up a for a test, I would have nmon running before, during and after the test. That way you can see the point the server is putting under strain. E.g. For a 30 minute test, I would have nmon running for 40 minutes, which is five mins before and five mins after.

So, to sum up, my approach would be to run several tests... one thread, ten, twenty, thirty, forty and fifty. Each test to run for 30 minutes, with Nmon to run for forty minutes. Then you can analyse the server resources and from the test harness see how load affects the response times.

| improve this answer | |
  • Yeah. In real production, it requires even more than 500 threads. That is why I think it it is failed. – Lehtia Sep 17 '17 at 9:53
  • Hi, I have update my original post just now with additional pointers. You say 500, then you may need to consider additional servers and have the load balanced. – Chris Adams Sep 17 '17 at 10:09
  • Once you eastblisb how performance degrades, you can then look at the app for bottlenecks. If the server is behaving well,it could be the app that is slow. You'd need to look at the code to see if there any inefficienct areas. It might be data access, if there is a database behind the app. If there is a db, you should also ask your Dba to run an AWR report to see if the the db needs optimising. – Chris Adams Sep 17 '17 at 11:06
  • Well, we have already found out the reason. That is because the S3 bucket of AWS. We are working on a better efficient way to solve this! Thanks a lot! – Lehtia Sep 18 '17 at 6:04
  • Good stuff. Once resolved and you have your tests running, you might also want to try JMeter as Dmitri suggests. I'd be surprised if JMeter isn't suited for your needs. It prob more th case that it might not be clear how to achieve what you need. – Chris Adams Sep 18 '17 at 7:07
1

I don't think you should be re-inventing the wheel as most likely your "small program" won't be displaying charts with correlation of increasing load with decreasing throughput, how many virtual users are active at the moment, how many server hits per second you managed to produce, what are server side health metrics during test execution, etc.

So I would recommend the following approach:

  1. Implement your test using a special dedicated load testing tool. There is a variety of free and open source load testing tools available in the market so you won't have to spend extra budget for that.
  2. Implement your test using this load testing tool. Remember that it should represent real user as close as possible with al its stuff like Cookies, Cache, handling of images, scripts, styles, simulating AJAX requests, etc.
  3. Verify that your test is doing what it is supposed to be doing by running it with 1-2 virtual users and 1-2 iterations
  4. Implement monitoring of baseline OS health metrics on application under test side to report i.e. CPU, RAM and Swap usage, Disk, Network IO, your application-specific metrics if any
  5. Do the same for your load generator(s)
  6. Start your test with 1 user and gradually increase the load until until response time will start exceeding acceptable boundaries or errors start occurring whatever comes the first. Mind how many virtual users were online at this stage - this is your application bottleneck.

Now you should have more information to analyse, i.e. you will be able to say how many users your application can handle and where it breaks. You will be also able to state if it is caused by banal lack of resources. On the other hand if your application responds slowly but there is a headroom in terms of CPU, RAM, etc. it indicates either wrong configuration of the application (or underlying infrastructure) or implementation performance problems - in this case you will need a profiler tool to identify "heavy" methods and objects.

| improve this answer | |
  • Hi, thx for the reply. I thought about using JMeter, but it may not fit my situation. We are using AWS server, and there are an app installed on PC to keep posting some JSON files to the server. So no cookies, no images and so on. The problem is the data in each JSON file is random generated. I am not sure if Jmeter can do this. In addition, I need to sign each POST request which is already done before. So I decided to write a small program. – Lehtia Sep 18 '17 at 6:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.