1

I have a web performance test created in VSTS 2013 ultimate and 1 execution round of this web performance test creates 1 entry in Database, no matter input data is unique or not, so same input data can be used to created multiple records.

Now, I need to create/configure my load test (containing 1 scenario and above mentioned 1 web performance test only) in a way, so that it generates 200 concurrent users load & executes the complete load test in sequential way 5 times i.e.

  • Iteration1 starts = 200 users come in at once execute all requests 1-by-1, Iteration1 ends
  • Iteration2 starts = 200 users come in at once execute all requests 1-by-1, Iteration2 ends
  • Similarly Iteration3...4...5...

So that at the end of test, there should be 200*5 = 1000 rows affected in Database.

What should be the value of Scenario Iterations, Users and Test Iterations in this case? My CSV file have 100 rows of input data and load test should restart through this CSV file once it reaches file end for next 100 or more users.

Update: users who has finished Iteration1 can jump to Iteration2 without waiting for other users. But in total each user should go through the test 5 times.

1

Your requirements could not be fulfilled just by setting users and iterations. Here's why. By default, users act asynchronously. So when user 1 completes iteration 1 it will proceed with iteration 2 without waiting for other users to complete iteration 1.

It looks like you specifically require for a user not to start the next iteration until all other users are done with the previous iteration. To achieve this behavior, you need to synchronize users by inserting a rendezvous point at the end of every iteration. This feature is available in the tools like LoadRunner and our StresStimulus, but is not available "of the shelf" in VSTS. Here is how to implement rendezvous point programmatically in VSTS.

  • You are misunderstanding "scenario iterations". It tells how many test cases to execute within the scenario, not how times to execute the scenario. – AdrianHHH Apr 12 '16 at 21:13
  • If I set users = 200, and Scenario Iterations = 5 then load test executes only 5 times with affecting only 5 Database rows. – Dhiman Apr 12 '16 at 21:18
  • Tanks for correcting. – Vadim Kleyzit Apr 12 '16 at 21:51
1

Assuming your question requires the following:

  1. 200 Virtual users.

  2. Each runs the same test case.

  3. Data driving is used, with 100 sets of data.

  4. Each user shall run one test case then pause until the next round starts.

  5. There will be five rounds.

  6. Thus a total of 5*200 = 1000 test cases should be executed.

There is no built in facility for this load pattern.

If each test takes the same time to execute then an approximation to the required pattern can be obtained by setting a long think time between iterations(perhaps the same as the test execution time) and make sure that think times are as stated, not normally distributed. Then set the run settings to 1000 executions and the scenario settings to 0 iterations. However, variations in test execution time would mean that each round of tests gets further and further out of alignment.

To get better alignment of test starts, create a scenario that has a constant load of 200 virtual users and an iteration count of 200. Create 4 copies of this test (copy and paste the scenario in the load test editor). Set the Delay start time property of each scenario so that the rounds of executions are as required. Set the iteration count of the run settings to 1000.

As the data source file has only 100 entries then at each round two virtual users will process the same set of data.


A comment asks how to proceed if point 4 is removed. It was the bulleted list, discussing iterations, in the original question that made the question difficult and led to point 4. Without point 4 the whole requirement is just like any standard load test. The settings are then simple.

  • Set the scenario to 0 iterations (meaning not limited within the scenario).
  • Set the load to a constant 200 users.
  • Set the run settings to 1000 executions.

With these settings, whenever a virtual user finishes one test case that user will immediately start executing another test case. This continues until 1000 test cases have been started. On average can expect each user to run 5 test cases. However, suppose the one of the first test cases takes an very long time and another takes a very short time. Then it is possible that the first user will only run 4 test cases and the second will run 6.

  • Lets say I remove point 4 from your mentioned scenario i.e. A user who has completed its iteration can jump or start with next one, then what should be my parameters for concurrent user load of 200 users for 5 rounds? – Dhiman Apr 12 '16 at 21:16
  • Just to confirm if I configure my Load test with Usercount = 25, Max. Scenario Iterations = 25, Test Iteration count = 25, then at the end of my test only 25 rows will get added to Database, Right? And Load will be concurrent in this scenario (considering, executing load test 1 time will add only 1 row in DB). Second, if Usercount = 25, Max. Scenario Iterations = 25 and instead of Test Iterations I specify Run Duration (more than enough time for 25 iterations) then at the end of Test still only 25 rows will get added and load in this case too will be concurrent user load, Right? – Dhiman Apr 15 '16 at 15:36
  • @Dhiman Surely you have been reading the extensive Microsoft documentation about how load tests work. You have also stated that you have been experimenting with various settings. Hence you should be able to answer this last question from the documentation and confirm that answer by experimentation. – AdrianHHH Apr 17 '16 at 13:53
  • But it will be good if someone who has used VS for performance testing, can confirm my these observation/assumptions. Also, for the Concurrency part I am not able to verify that, as I don't have access to server logs to do that or if I can validate it from VS results, please let me know how I can do that. – Dhiman Apr 19 '16 at 19:53
  • @Dhiman First version, correct. Second version correct, but run duration long enough for the longest single execution, not 25 times that long. To validate see the "Detail" page in Visual Studio results. Check that Run Settings has "Timing details storage" set to "All individual details" before the run. – AdrianHHH Apr 19 '16 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.