13

We have an a browser based application that is built on the Microsoft Stack, specifically .NET 3.5. It also uses Ajax, iframes, jQuery etc.

At the moment we are just guessing at the concurrency rates that our app/server architecture can provide. This is making everyone nervous and it would be really good to work out a more exact figure. By concurrency, I mean X number of users logged in at the one time. Specifically, I need to know how many concurrent users the server/application can take before the user experience gets really bad or the server crashes etc.

Are there any simple tools/services that we can use to get this information?

I have no prior experience of performance/load testing as it's always been done by specialized people. Is this the sort of thing that's going to need months of learning/ setting up a tool?

Improve this question
3
  • 2
    Note that performance testing has a much wider meaning (0-100 of a car, network throughput etc.). Better use something like web "site/app performance testing"
    – Rsf
    Commented May 12, 2011 at 17:28
  • IMO, "load testing" clarifies the intent well enough. You could look at Amazon Web services, people discuss using EC2 to generate multiple "users" hitting your site - but you'd need an external testing site for that.
    – Ethel Evans
    Commented May 12, 2011 at 18:04
  • 2
    Are you talking about "Concurrent users" or "simultaneous users"? The two are different in that concurrent are users clicking the same buttons at roughly the same time while simultaneous are users that are all on the site at the same time but perhaps in different areas, doing different tasks.
    – TristaanOgre
    Commented May 12, 2011 at 20:01

9 Answers 9

Reset to default
5

There are a lot of performance testing tools:

Setup for the most part is very easy. You could run each of these packages on a local VM; however, if you want to simluate large loads of traffic you need to have a machine with a little more RAM and Processor.

Since this is an AJAX application you might have to simulate the resources on disk that might be on contention. A functional test might do this for you, but if you're interested in how your server and stack are responding how you craft your requests for load are important.

Improve this answer
2
  • 5
    Rather than just addtional power on the machine, I've found it usefull to run multiple machines at once if possible. Too many requests from 1 machine and I've seen my results become skewed.
    – Lyndon Vrooman
    Commented May 12, 2011 at 15:48
  • Very good point.
    – terryp
    Commented May 12, 2011 at 15:51
4

You should not focus on the concurrency rate, rather try to get traffic statistics, such as visits, amount of activity, and so on. If you already have a site running, you can analyze the server logs or any audit trails your application might write. When your site is still offline and you just want to go live, you have to come up with some expectations for traffic.

These numbers will give you an idea what to simulate.

But if I got your question right, you are looking for a way to get the possible max throughput out of your application to know what you can expect of it, rather coming with numbers and trying to prove that you can make the numbers. Right?

This basically calls for a performance test, where you have to set your expectations first:

  • what is still acceptable performance (response time limits)
  • what are the standard scenarios for the application, what kind of usage patterns do you anticipate
  • what kind of data growth can you expect based on the usage patterns
  • what is your standard deployment scenario in the beginning, what hardware do you want to us (because you do not know yet, what you need, you have to set something first)

Based on that you can develop you performance scenarios and run it. If you have never done a test before, you should not expect to get an answer to you question rather to find a ton of issues around the application. After fixing that, you will get an answer to your question and get an idea, not a fixed number.

You might also like that article of mine about getting numbers for load testing including some ideas about concurrency: http://blog.xceptance.de/2011/06/07/get-the-right-load-mix-out-of-a-few-numbers/ This is a little ecommerce centric... but might give you still an idea what you might need or what you can do without numbers.

Improve this answer
3

As you are using the Microsoft stack I have to recommend the Microsoft Visual Studio tool kit. The load testing tools that come with visual studio will hook into your performance counters on the boxes. The tools include guidance from the asp.net team baked in in the form of key warnings when certain thresholds are reached like an excessive number of garbage collections occurring.

You should also look into the Microsoft patterns and practices documents on ASP.net performance as there is a lot of good guidance for performance issues in there.

Improve this answer
3

I have worked in VSTT and did performance testing for OLTP application. VSTT has good information and articles on web testing, load testing

Improve this answer
1
  • Just one comment. The quick reference guide is up to version 3.6, found here: vsptqrg.codeplex.com
    – Sam Woods
    Commented Nov 8, 2011 at 19:46
2

JMeter is the open source performance testing tool that might be useful in this situation. There will most certainly be some ramp-up time to learn how to use the tool as well as some time required for general performance testing education (best practices, etc).

Improve this answer
1

I think a simple Google search can give you tons of tools which might help, but you need not to look at them before your goals and requirements are clear. What I believe you're really asking is 'how-to' instead of a list of tools. So I'll try answering that way.

Are there any simple tools/services that we can use to get this information?

I started out with the same questions and concluded this:

No, there is no shortcut.

Because in order to conclude from your performance tests - you'll need to determine the most important metrics for your business. Our company being in web apps has these key metrics or KPIs (Key performance indicators):

  1. Page Load Time
  2. No. of queries ran at every call
  3. Slowest Query
  4. Memory Usage

Once you determine these metrics, you should analyze and address bottlenecks found during performance tests, and then proceed to load testing or concurrency.

Note: I'm not saying tools aren't helpful but they just make the job easier - infrastructure and rules has to be laid down first.

Improve this answer
1

Well, I'm a happy Selenium user these days. I don't see any reason you couldn't use Selenium to do load testing, although you would need to invent data-collection and summary mechanisms. Still, it would probably be worth it, because Selenium would be useful for all sorts of automated testing.

Improve this answer
1
  • 3
    Selenium is not really a performance testing system (although it can be abused as one). If you want to use Selenium as a load testing tool you'll need a hell of a lot of hardware, or you'll need to use somebody like sauce labs (saucelabs.com). I would suggest you are better off going the JMeter route!
    – Ardesco
    Commented Mar 1, 2012 at 15:45
1

In addition to the tools and advices above you may also try to use Tsung – open source multi-protocol distributed load tool - for your load-/stress-testing activities.

The purpose of Tsung is to simulate users in order to test the scalability and performance of IP based client/server applications.

The main features to note are:

  • Tsung is written in Erlang to take advantage of that language’s concurrency support: Tsung will not drive the load as fast as Passport can take it but will send defined number of requests every second even if system-under-test cannot cope, so you can expose bottlnecks.
  • Tsung scales well (it’s been used to simulate tens of thousands of users).
  • OS monitoring (CPU / memory utilization, network traffic) using SNMP, munin-node agents or Erlang agents on remote servers.
  • Tsung supports forms and HTTP sessions, includes some niceties like proxy recording, ‘thinktime’ support, and a choice of random or ordered traffic.
  • Tsung is a distributed load testing tool that is available free of charge under the GPLv2 license as an open-source software product.
  • Tsung is protocol-independent and can be used to stress HTTP, WebDAV, SOAP, PostgreSQL, MySQL, LDAP, Jabber/XMPP servers.

More detailed description you can find on the tool's site.

Tsung can simulate very large numbers of users per server, making it ideal for analysing and testing the performance of large-scale applications, such as instant messaging solutions.

Several good guides and points to start:
• Test the Performance and Scalability of Your Web Applications with Tsung
• Using Tsung for Load Testing Rails or Web Apps
• Tsung Installation
• Tsung ProcessOne page

As for me I've used it successfully for both load- and stress-testing web-services on the project. Tsung may be a bit tricky for first use but really effective in case of high concurrency testing.

Improve this answer
1

I am not sure, whether you meant "Concurrent users" or "Simultaneous users", but it seems you want to address risks related to 'Speed', 'Scalability' & 'Reliability' of yours web site when multiple active sessions are in use and server resources are being used concurrently. There are different types of performance testing required at different time for validating different scenarios / issues in the application. In the below url, I tried to differentiate multiple types of performance testing. It is not the standard definition though but I tried my best to be in sync with others.

http://rgsoftwaretesting.blogspot.in/2011/01/performance-testing-types.html

From tools set perspective, following are list of few tools that I have used and find it very useful:

  1. JMeter - An open source tool but little hard to learn / use.

  2. BlazeMeter - Cloud based tool, use JMeter scripts and takes care of internet bandwidth requirements. Each instance in cloud is connected with 100Mbps / 300 Mbps line.

  3. BrowserMob - Cloud based selenium grid based tool, very easy to use and learn

  4. VSTS - Licensed tool from Microsoft, affordable

  5. NeoLoad - Licensed tool, easy to use and learn, support flash / flex
  6. LoadRunner - Licensed tool, very expensive but support many protocols

As yours application is developed on Microsoft Stack, you can use 'Perfmon' to collect performance counters from multiple servers and correlate it for analyzing it further. All licensed tools inherently supports gathering these performance counters from multiple servers and provides very easy interface for correlation and reporting.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.