Load test Configuration in VSTS 2013

Question

I have a web performance test created in VSTS 2013 ultimate and 1 execution round of this web performance test creates 1 entry in Database, no matter input data is unique or not, so same input data can be used to created multiple records.

Now, I need to create/configure my load test (containing 1 scenario and above mentioned 1 web performance test only) in a way, so that it generates 200 concurrent users load & executes the complete load test in sequential way 5 times i.e.

• Iteration1 starts = 200 users come in at once execute all requests 1-by-1, Iteration1 ends
• Iteration2 starts = 200 users come in at once execute all requests 1-by-1, Iteration2 ends
• Similarly Iteration3...4...5...

So that at the end of test, there should be 200*5 = 1000 rows affected in Database.

What should be the value of Scenario Iterations, Users and Test Iterations in this case? My CSV file have 100 rows of input data and load test should restart through this CSV file once it reaches file end for next 100 or more users.

Update: users who has finished Iteration1 can jump to Iteration2 without waiting for other users. But in total each user should go through the test 5 times.

Answer 1

Your requirements could not be fulfilled just by setting users and iterations. Here's why. By default, users act asynchronously. So when user 1 completes iteration 1 it will proceed with iteration 2 without waiting for other users to complete iteration 1.

It looks like you specifically require for a user not to start the next iteration until all other users are done with the previous iteration. To achieve this behavior, you need to synchronize users by inserting a rendezvous point at the end of every iteration. This feature is available in the tools like LoadRunner and our StresStimulus, but is not available "of the shelf" in VSTS. Here is how to implement rendezvous point programmatically in VSTS.

Answer 2

Assuming your question requires the following:

1. 200 Virtual users.

2. Each runs the same test case.

3. Data driving is used, with 100 sets of data.

4. Each user shall run one test case then pause until the next round starts.

5. There will be five rounds.

6. Thus a total of 5*200 = 1000 test cases should be executed.

There is no built in facility for this load pattern.

If each test takes the same time to execute then an approximation to the required pattern can be obtained by setting a long think time between iterations(perhaps the same as the test execution time) and make sure that think times are as stated, not normally distributed. Then set the run settings to 1000 executions and the scenario settings to 0 iterations. However, variations in test execution time would mean that each round of tests gets further and further out of alignment.

To get better alignment of test starts, create a scenario that has a constant load of 200 virtual users and an iteration count of 200. Create 4 copies of this test (copy and paste the scenario in the load test editor). Set the Delay start time property of each scenario so that the rounds of executions are as required. Set the iteration count of the run settings to 1000.

As the data source file has only 100 entries then at each round two virtual users will process the same set of data.

---

A comment asks how to proceed if point 4 is removed. It was the bulleted list, discussing iterations, in the original question that made the question difficult and led to point 4. Without point 4 the whole requirement is just like any standard load test. The settings are then simple.

• Set the scenario to 0 iterations (meaning not limited within the scenario).
• Set the load to a constant 200 users.
• Set the run settings to 1000 executions.

With these settings, whenever a virtual user finishes one test case that user will immediately start executing another test case. This continues until 1000 test cases have been started. On average can expect each user to run 5 test cases. However, suppose the one of the first test cases takes an very long time and another takes a very short time. Then it is possible that the first user will only run 4 test cases and the second will run 6.