Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It only takes a minute to sign up.
Sign up to join this community
How effective is Visual studio test suite than load runner?
I would like to learn performance/load test a web application, but i got really confused to choose which tool to go for.
Does the scope of VSTS 2010 is better than Loadrunner, considering the cost of the license too?
PS: I'm new to performance testing, please recommend a tool which is good for a beginner. I tried Jmeter but its just a long way to go.
I like Visual Studio Load Testing, it is my preferred tool, however I have not had a chance to use Loadrunner to do a proper comparison. What I can say is that for web services running on Windows machines, Visual Studio has a ton of built in metrics and reporting that make it really nice, while there is almost no support for Linux servers.
In terms of ease of use, I would say it is somewhat easier to get started with than JMeter which I also have had experience with. From what I saw, there was a lot more documentation and training around for Visual Studio Load Tests.
For Visual Studio, the pricing is radically different so the two are not comparable. You need to have Visual Studio Ultimate, which is an enterprise decision, vs LoadRunner which is tied to a single controller instance. Depending upon your enterprise you could wind up more expensive under Visual Studio vs LoadRunner, particularly with the change in pricing which is effective as of December 2013.
The two tools are very different philosophically as well. Visual Studio is aligned with Microsoft's overall philosophy on testers: There are no performance testers, but there are developers who performance test. It is tightly bound with Microsoft's overall platform and systems philosophy, so if you wind up testing a lot of Java infrastructure items using CORBA/RMI/ORACLE, essentially anything not produced or endorsed by Microsoft then you are going to run into structural challenges on monitoring , diagnostics, production of a specific interface type which is "non Microsoft" in nature.
HP's tools (as well as Borland) are very platform agnostics and interface agnostic. You need to test SAP? Fine Corba? Fine. .Net? Fine. Web? Fine.... And these are geared towards professional systems engineers and performance testers who happen to have foundation skills in platforms, systems analysis, test and development, a superset of the skills imposed on developer focused tools like JMETER and Visual Studio. Analysis and monitoring come into play as well here.
If you are just looking to dabble, pick up a free tool. If you are evaluating a tool for enterprise use, leave cost as the last issue. Look at the capability of the tool for the production of load, the monitoring of your infrastructure, the ability to report against your requirements and how it provides access to information for diagnostic purposes. Take a hard objective look at the skills of your team and how this matches to the tools under evaluation. if you pick wrong and the tool is not effective at it's job or the users don't have the skills to be effective with the tool then you have virtually asked your team to assemble and entire house using only a jewelers screwdriver set: In short, you have guaranteed a negative ROI no matter how much the tool costs.
This mismatch of skills to an effective tool is the number 1 reason for failure of performance testing practices across all tools in the marketplace today. Open Source and commercial. Get the match right and you are virtually guaranteed a positive ROI on even the most expensive tools because the cost of the performance testing effort plus the tool will never exceed the cost of failure in production.
