Load testing web service with SoapUI to confirm android app's slowness

Question

The AUT is on Android platform, which has experienced slowness or long time loading a section's contents (more than 17s).

I can notice the app only has long time loading the first time I enter that specific menu, after that moving in/out the menu does not have that high loading time anymore. (Something of cached contents? I'm not sure)

From the web service perspective I know, that this action to enter the menu contains of 3 separate requests and waiting for feedback from the web service.

That's why I used SoapUI to make a simple load test to confirm this slowness.

However, the result for the load is not promising:

• max response time: around 20000ms
• average response time: 500ms/1200ms/5000ms (for each 3 requests)
• That is just for 50 threads, which result in 0 to small percent of time-out errors.
• If I try to use 400 threads instead (which is a requirement from customer), the timeout errors will be unacceptable (50%).

So, my questions are:

1. Is the slowness on the app a result of the sum of these 3 requests: around 7000ms ~ equal 7s? (It is still way much lower than 17s on real device).
2. Also, does this appear, that the web service can not afford a high number (400) of concurrent users in real working environment?
3. Could it be, that or am I doing something wrong here?

Answer 1

Is the slowness on the app a result of the sum of these 3 requests: around 7000ms ~ equal 7s? (It is still way much lower than 17s on real device)

Yes

That is most likely the cause of your issue. The application likely catches and retries when it times out which will give you these very long times. It sounds like there is some severe issues, most likely in the database level that is causing this. I would suggest attempting to run the same query(ies) or stored proc(s) that the AUT is running and see how it handles it.

The 17s vs 7s can be caused by:

• Network Latency
• Application formatting
• Application Lag
• Time Outs and Retries

I think that all of the above, to some degree, plays a part in your issue.

Also, does this appear that the web service can not afford a high number (400) of concurrent users in real working environment?

Absolutely

Your AUT has some major issues.It needs severe performance tweaks and desperately needs a DBA to review what is happening. Unless your test server is just horrible compared to your production server, it will be massive issues. And even if it is, the amount of a peak for that single screen demonstrates the best area for system enhancements in order to improve customer satisfaction and reduce company expenses. It's a win win for the fix.

Could it be that or am i doing something wrong here?

Yes, your Soap requests might be crazy and cause 1000x more processing that the average request but as long as it is similar to what an actual user would possibly be doing than you are probably doing everything right. It sounds like a pretty bad issue that while it may not take the highest possibly priority should at the least be an area of concern for the long term and I would DEFINITELY ensure that you practice CYA and notify your management and team of this potentially fatal risk.

Answer 2

welcome to SQA. One thing that it looks like you are not taking into account is the network speed of the device you are testing on. If you are running load runner on a network that has high internet connection speeds, or more probably is on the same network as the service you're testing, then you will get almost no additional time due to network latency, whereas on a device you will likely see quite a bit.

There are tools that you can use while you're executing load runner to simulate network latency, Load runner may actually have this built in, but I'm not super familiar with the tool.

Lastly, adding network latency to the requests will likely exacerbate the issues you're seeing with timeouts because now the web service will have to wait longer before it gets a response back from the client and will have more threads it's keeping around in memory for a longer period of time.