Visual Studio Team System 2008 Test Load Agent - Query on "Usefulness and Licensing"?

Question

I m a tester using VS 2008 - Test edition for performing web testing on web applications. So got a suite of web tests for different web applications in our company.

Recently I was asked to perform Load testing for around 2500 to 5K users.

Currently VS 2008 can allow upto 1K Vusers as load but to create some load like 2500 to 5K Vusers, with respect to the information on MSDN I need to use Load Agents ( atleast 5 Load Agents) and which Microsoft provide only 190 day trail or 25 test runs for the load agents software. Once I have run 25 tests then I think I cant use it unless i change the users ( which might breach the license).

Microsoft's recomendation in thier product configuration and installation section is to get a license per processor ( under test), but thier is no clear information on how much they cost. Since I need to raise a call to my manager ( who maintaince the volume licensing) about how much they cost and convience them, I need some information which I couldnt find it googling or on MSDN.

Mostly on MSDN they are recommending VS2010 Ultimate version which is expensive and my management has rejected my request.

So if anyone who has or currently using VS2008 test edition, Can please let me know what might be the licensing cost for the VS Load Agent 2008 which I think is per processor ( for load agents rather than cost per load agent)?

Any experience of using the Visual Studio Team System 2008 Test Load Agent ( like advantages or disadvantages) would be really helpful.

Answer 1

Microsoft licensing is one of the most complicated things on this planet, especially when you start talking about enterprise agreements. Your best solution is to go straight to the source

You can call 1-800-426-9400, Monday through Friday, 6:00 A.M. to 6:00 P.M. (Pacific Time) to speak directly to a Microsoft licensing specialist, and you can get more detail information from there. Worldwide customers can use the Guide to Worldwide Microsoft Licensing Sites to find contact information in their locations. http://www.microsoft.com/licensing/index/worldwide.mspx

from MSDN forums.

Answer 2

What does a test for 5K users mean? is that 5 k users over day? over an hour period, with each user on the system for about 10 minutes? or all hitting the system 'at once' (well, as much 'at once' as a serial network connection and TCP/IP will allow). Depending on what the true details of the scenario is, then you can do it without agents.

I'd read the MS patterns and Practices book on loadtesting, get a better handle on exactly what kind of test scenario you will be doing.

You might not need agents. Remember that a given vuser can simulate the load from many individual users over time (one after the other) Also sometimes all it takes is a few hundred people at one time doing stuff that is db intensive (logging in, putting stuff in shopping carts, etc) to really load a system. I've brought some systems to their knees with only a few hundred vusers.

create your scripts, unless the site is exceptionally well made, you will likely start exposing problems very early on. You don't need those agents until you get to the point where you want to run all your scripts that are part of the scenario at the same time, with more vusers per script than you can handle from a single system. Only once you have all your scripts running great, and can individually scale each one up to a load in excess of what you would need in the big combined scenario, do you need to start using Agents..

At that point, you might be able to get some good useful work done before you run out of test runs, and can prove the value of what you have done and allow management to know how many agents you really need and that they are worth it to them. (and trust me on this, 'expensive' as they may be, Loadrunner is far more expensive)

Also, for your base system, move the db off onto another box so it is ONLY doing the loadtest work and if it has a robust processor and lots of RAM you may be able to scale above 1000 vusers (depends on your scripts)

Also do be aware that MS has completely changed their licensing model for VS2010 where the load stuff is concerned, and may not be very interested in selling you anything to work with 2008. (and yeah Ultimate is pricey, but again, LR is more)

Answer 3

The low tech/low budget solution would be to simply set up Visual Studio on multiple machines and run tests from each of those. Set up one of them to be the "main" run that collects performance counters, etc.

Just out of curiosity, what are you testing that requires that amount of load? Are you running your load tests against a set of servers or a single server? Normally you would pull a single server out of rotation and run your tests against it so you could use a lower number of concurrent users in your tests.