Tool used - HP LoadRunner 12.5; Protocol used - Ajax TruClient. The application is hosted on Nederland location.

we are performing load tests with the same number of scripts simulating the load from Nederlands (40 vusers), China (20 vusers) and USA locations (40 vusers). Given the response time of these tests return similar results, we have noticed a very drastic difference in the Throughput. The Nederlands location has returned very less throughput and the other two locations, even though there is a difference in load returned similar throughput. The difference is by a margin of over 150%.

The test duration was 1 hour for each of the test which were executed one after the other (Nederland -> USA -> China). Client side caching was disabled and there is no caching activity was turned on in the servers or the Load Balancer as well.

Please help us by giving a perspective to look at in resolving this mystery.

Given that your application is hosted in the Netherlands and your application throughput/response time is worst from this location versus the other two distant locations (China/USA) across complex networks, the items to consider most would be local issues.

You have noted that you are using TruClient for your testing. This type of virtual user is very resource intensive on the load generators, as you are running full browser instances. As such, it is quite easy to exhaust the resource pool on your available load generators. One way you can check against this possibility is to include a control load generator in your load generator pool in the Netherlands which runs on a single virtual user. This should be running on complete independent hardware from the rest of your load generators.

If your control group and your global group (well, Netherlands based virtual users) all degrade at the same rate then you are likely looking at a network issue impacting all of the users inside of your local LAN. However, I do not think this will happen because of the apparent good throughput from the USA and CHINA during your tests.

What will most likely happen in that your control user will bounce along quite happily during the test, but your load generator with the remainder of your Truclient population (assuming one load generator here instead of three or more which would be preferred with one control generator in the group) will degrade. This is a direct sign that your load generator is bottlenecking and resulting in the slowed performance for your virtual users, not your application.

Always include control factors in your test design. I know that this is sort of a "scientific method 101" type discussion, but it seems that this element is oft overlooked in automated and performance test designs.

  • Thanks James; we tried very hard to do a root cause analysis but failed to find one. Strangely enough, when we changed the LG machines hosted in NL location, this issue is fixed. The problem was solely with the hosted LG machine in NL but the reason is still unknown and we have abandoned it because it wasnt worth our time. – Mishal Sep 5 '17 at 11:31
  • make sure you have multiple load generator machines in each location to act as a control element against one load generator being overloaded and coloring your results. – James Pulley Sep 5 '17 at 18:22

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.