Hot answers tagged performance
12
There is never a general industry-wide answer to this sort of question. What might be "Key" for your company may very well not be "Key" for my company.
To answer this question for your specific context, find out what matters to the stakeholders you serve. Figure out what is important, what is not, and then determine what indicators can help you determine if ...
10
The first thing you need to do is isolate whatever could be messing with your system. Make it as independent as possible. The fewer background processes, extra hardware and software, and even network traffic you can get, the better. Sometimes it's even a good idea to run these on a batch job that runs at night at a time when you know there's nothing else ...
7
All right, So Tristann kindly revised his original question to include more details in terms of a scenario. So I'm adding a second answer to more directly address it.
Firstly you'd probably want to ask a few more questions about what the customer is most concerned with and what they want tested, here's a small sample:
what's the duration of the shoppers
...
7
I have had some luck using Grinder. It is Java based but you can also write your scripts in Jython or Clojure.
You said that you would like to take advantage of your team's current Selenium scripts and expertise performance testing. You did not describe your regression test scripts, but you may want to reconsider whether they are appropriate. In ...
5
There are a lot of performance testing tools:
Apache Bench
The Grinder
Siege
Pylot
Setup for the most part is very easy. You could run each of these packages on a local VM; however, if you want to simluate large loads of traffic you need to have a machine with a little more RAM and Processor.
Since this is an AJAX application you might have to simulate ...
5
Desktop software usually requires installation. Web applications usually do not.
But web applications are sometimes expected to be running 24x7. This can make upgrades and maintenance more of a challenge to plan and execute (and thus test)
In addition to browser versions mentioned by others here, you may need to worry about browser add-ons
You may also need ...
4
You should not focus on the concurrency rate, rather try to get traffic statistics, such as visits, amount of activity, and so on. If you already have a site running, you can analyze the server logs or any audit trails your application might write. When your site is still offline and you just want to go live, you have to come up with some expectations for ...
4
I wrote about concurrent users and numbers in a blog post: http://blog.xceptance.de/2011/06/07/get-the-right-load-mix-out-of-a-few-numbers/
Wait… where are my concurrent users?
This is simple: “concurrent users” is
an inaccurate way of describing
traffic, so we have not used that
number yet. Why is that?
To get to the bottom of that, we
...
4
I do not think the UI testing is very different: field validation, default values, resizing, scalability, and so on.
You probably need to support more than one brand and version of web browser, and perhaps even some mobile devices. You may want to separate your business logic tests from your browser-level tests so that you do not repeat every test on ...
4
Combinatorial explosion of varieties: You may need to test each version of each browser on various hardware running various operating systems
Front-end testing can be easier because of the universality of the displayed information
When you do performance/load testing you're simultaneously testing the machine the server is on, not the current desktop (unless ...
4
Apart from "what those other guys said", all of it very good advice, some other considerations I'd recommend are:
Usability - Desktop applications tend to have a help file built in, where web applications should be more or less self-explanatory.
Load times - this one is a big pain point. Not everyone has broadband (and we won't go into how much I despise ...
3
NOTE: this answer was to an earlier version of the question which was asking of you needed a 1:1 relationship between vusers, etc. Rather than re-word it I'll let it stand as is since the info inside is still pretty sound. But now you know why it doesn't seem to be directly answering the new version of the question.
Generally if you can afford it ...
3
Oh deja vu ...
I think that Larry Elison summed it up nicely when he stated that Cloud computing is just, servers and the internet. Video is here -> http://www.youtube.com/watch?v=UOEFXaWHppE
Performance testing "Cloud" applications should be tested as though you would test any existing web application. Specifically for performance you would want to ...
3
I'm of two minds about this.
Don't load them - There are a number of problems I've uncovered with a web application where the DLL's and other such objects were not loaded in memory first. Two concurrent users hitting the app at the same time before all files were "spun up" created some really strange concurrency issues that only happened in that instance. ...
3
I have worked in VSTT and did performance testing for OLTP application. VSTT has good information and articles on web testing, load testing
Visual Studio Performance Testing Quick Reference Guide (Version 2.0) Published
Performance Testing Videos Posted to CodePlex
3
As you are using the Microsoft stack I have to recommend the Microsoft Visual Studio tool kit. The load testing tools that come with visual studio will hook into your performance counters on the boxes. The tools include guidance from the asp.net team baked in in the form of key warnings when certain thresholds are reached like an excessive number of garbage ...
3
Lyndon is half right. To clarify - Visual Studio web performance tests (coded or otherwise) will not ever execute JavaScript - either in a browser or by any other means. The article Anicho references states "You can use coded a Web performance test to simulate the logic typically performed by JavaScript"
You can craft them however to execute xmlhttp ...
3
you could use a plugin, although there is a simpler way. You can create a CSV file with the list of usernames and passwords you want to iterate through and then create a datasource. When you execute the web test, it will iterate through all of the items in the CSV file, one line for each test execution.
Based on what you outlined above I would probably ...
3
The answer to this question is obvious: it depends.
As an example, the customer complained on that the report generation pages takes a lot of time ~ 4 minutes to generate the report.
Our developers were not able to fix the performance without rewriting the legacy code from scratch. So they just added “ajax-like” loading indicator. Well… our customers were ...
3
You didn't say how much precision you need -- do you just want to know which is faster, or do you want to know how much faster?
Here are some factors that interfere with benchmarks like that:
Clock granularity. I assume you will use something (System.currentTimeMillis, System.nanotime(), or a stop watch or whatever) to measure the time interval. The ...
3
Silverlight is NOT a thin client, it allows for heavy client-side processing and so the request is quite valid.
Silverlight is a dialect / subset of WPF, the technology that replaces Windows Forms, another fat-client :)
Here are a few ideas on how to go about this "perf testing" (it's not really load testing because a client can only be accessed by 1 ...
3
AirieFenix, welcome to SQA. Here are some possible problems:
Your JMeter machine ran out of resources.
Your server is overwhelmed.
An intermediary (e.g. a proxy server or a load balancer on a DMZ machine) between your JMeter machine and the server is overwhelmed or is intentionally throttling network traffic.
Here are some ways to narrow things down:
...
2
JMeter is the open source performance testing tool that might be useful in this situation. There will most certainly be some ramp-up time to learn how to use the tool as well as some time required for general performance testing education (best practices, etc).
2
If you are interested in web sites, and mostly from a front-end point of view, the best books I have found are:
High Performance Web Sites
Even Faster Web Sites
both by Steve Souders.
2
I loved the book The Art of Application Performance Testing. It gave me a good primer the first time that I was told that I would have to set up performance tests. I also occaisionally tak ea look through the blog OnLoadTesting occaisionally for some general theory.
2
Categories can help. Categories can help with some things. You might use categories to help indicate:
Who will write or run the tests. For example, unit tests are generally written by developers (and are sometimes called developer tests). Acceptance tests might be executed by testers or product owners.
The characteristics being tested--e.g. functional ...
2
You can switch to HTML Unit Driver if your project allows this (this might involve some tests update, since FireFox and HTMLUnit drivers have different js engines).
Otherwise, I agree with @SamWoods:
launch single browser (FireFox) instance before test suite run,
reuse it in all the tests, instead of launching another one.
don't forget to clean up ...
2
Unfortunately the initialization of the firefox webdriver object takes that long and there is nothing you can do about it. You could set up your tests so that in the [TestFixtureSetUp] method you create the FirefoxDriver instance and then use that same FirefoxDriver instance for all of your tests.
The only potential problem there is that if the browser ...
2
Unfortunately Visual Studio Webtest was not built to validate the parameters but more to put load on a server as all it does is send a HTTP request and retrieve the request in raw format with all the headers and validates the request in its raw form. Hence you are getting the escaped characters due to the encoding performed.
If you want to get around it, ...
2
I think the key to your question is "web application". So we should not be directly concerned with which cloud you use or even latency.
Performance testing can mean different things to different people. I see it as:
Testing speed of pages
Load testing
You need to know how the application performs in the view of users. Slow pages lower your revenue ...
Only top voted, non community-wiki answers of a minimum length are eligible
