Hot answers tagged

13

Similar questions have been asked a number of times. The general consensus is that UI automation tools such as Selenium are not a good choice for load testing. There are a number of reasons why, but the main one is that it is a LOT slower. You can easily spin up thousands of concurrent requests in a second in a tool like JMeter which to reproduce in ...


12

It is very likely you will get differing results. Especially in response times. This is mainly due to latency caused by the Geographical distribution. This will become important if response times is one of the metrics you are using to understand the website's ability to cope/scale under load. A simple reflection of this is; go to http://amazon.co.uk/ ...


11

According to Visual Studio: Load testing in the cloud, here are the steps to switch the test from using Visual Studio Online to running locally: Simply open your existing project using Visual Studio 2013 first. Within the Solution Explorer, expand the 'Solution Items' folder, then open Local.testsettings. Edit the test settings file to configure your ...


9

I have had some luck using Grinder. It is Java based but you can also write your scripts in Jython or Clojure. You said that you would like to take advantage of your team's current Selenium scripts and expertise performance testing. You did not describe your regression test scripts, but you may want to reconsider whether they are appropriate. In ...


9

No. The items you mention scale very differently and there are far too many factors and resources that will get used. For instance if I time requests on a local server using an application I'll find things like 1 user = 10 second response average (time per request) 10 users = 2.5 second average 100 users = 2.5 second average 1000 users = 20 second average ...


9

Does this actually catch, before production, many of the "surprise" problems we might anticipate? Or is there a more fundamental flaw in the approach that will cause deleterious changes to pass testing and affect production? You are wise to have a test system that you can use for catching performance issues, but your "scaling" approach is flawed. ...


8

All right, So Tristann kindly revised his original question to include more details in terms of a scenario. So I'm adding a second answer to more directly address it. Firstly you'd probably want to ask a few more questions about what the customer is most concerned with and what they want tested, here's a small sample: what's the duration of the shoppers ...


8

Apache JMeter is really cheap, and by really cheap, I mean free. It should be able to handle the find operations you are looking for. The download link is: http://jmeter.apache.org/download_jmeter.cgi


7

From my Experience I would Say Yes to VSTS Test Edition I used VSTS Continiously ran load tests for 4 Hours, Simulated Maximum of 60 requests / sec with 4 Test machines, OLTP Application Web Service / Biztalk / SQL Server / Replication / Service Broker based architecture VSTS Tests - Easily bind data to tests. This is pretty straight forward ( ...


7

This is a pretty interesting problem, actually :) You could try using a Markov Chain Monte Carlo approach. I haven't worked with these types of models myself, but the idea is that you start with a given long-run distribution and use Monte Carlo modelling to develop the actual Markov chain. I know this is a pretty well-used technique for simulation ...


7

You could also check out Gatling, which is an open source stress tool based on akka. On the Gatling homepage, you'll find a direct comparison of how your test specification will look in Gatling vs. in JMeter. This might help you decide between these too. If you want to create massive amounts of load, I think there's nothing better than akka as an engine. ...


6

If the system needs to scale to many concurrent users or work with lots of data, I would say load testing should be started as early as possible. This way possible problems in the application architecture can be found in a phase where it is still possible to fix them. Couple examples: Let's say that the database schema is such that some queries will be ...


5

Based on the answer and comments I had received what I wound up doing was: Use a Once Only Controller with a BeanShell Sampler to read the file into an array, then when I need to pull that data I just use a BeanShell PreProccessor with: import java.util.Random; Random random = new Random(); vars.put("MyChosenVariable", vars.get("AvailChoices_" + ...


5

Randomizing the file BEFORE use is a much better approach. It will much more efficiently (especially in the case of significant amount of entries) to randomize the order of entries in your CSV file before the test - and then read prepared data sequentially, in way you like (via CSV Data Set e.g.). You can do the preparation using e.g. perl script or even ...


5

you could use a plugin, although there is a simpler way. You can create a CSV file with the list of usernames and passwords you want to iterate through and then create a datasource. When you execute the web test, it will iterate through all of the items in the CSV file, one line for each test execution. Based on what you outlined above I would probably ...


5

There are a lot of performance testing tools: Apache Bench The Grinder Siege Pylot Setup for the most part is very easy. You could run each of these packages on a local VM; however, if you want to simluate large loads of traffic you need to have a machine with a little more RAM and Processor. Since this is an AJAX application you might have to simulate ...


5

I have no doubt that the IBM Rational tools could allow you to automate functional and performance testing. However, there are many free and open source tools that also provide the same (or often better) functionality. From my own experience working with proprietary test tools such as Rational (my experience was with Rational years ago, Silk and a bit of ...


5

Assuming the web application accesses a server then the purpose of many web performance tests is to see how the server behaves under load and whether high loads cause unwanted delays in responding to user activity. Also to see whether the system has headroom while coping with that load. One part of the test could be to see that 95% of all web pages are ...


5

Presumably the database is just a piece of your overall system, and your goal is to determine whether database changes break things or slow the system down. Using a downsized database is a reasonable way to check whether database changes break things. I'm not sure it makes sense to use a downsized database for load testing. Whether this makes sense ...


4

I wrote about concurrent users and numbers in a blog post: http://blog.xceptance.de/2011/06/07/get-the-right-load-mix-out-of-a-few-numbers/ Wait… where are my concurrent users? This is simple: “concurrent users” is an inaccurate way of describing traffic, so we have not used that number yet. Why is that? To get to the bottom of that, we ...


4

You should not focus on the concurrency rate, rather try to get traffic statistics, such as visits, amount of activity, and so on. If you already have a site running, you can analyze the server logs or any audit trails your application might write. When your site is still offline and you just want to go live, you have to come up with some expectations for ...


4

Does your company have a firewall proxy? I've run into this same issue with non-Vaddin apps, and resolved the issue by launching Jmeter with proxy info. From the command line, it looks like this: jmeter.bat -H proxyDomain -P proxyPort -u proxyUsername -a proxyPassword


4

If you have an average visit length of 5 minutes, a single user could create 60 min / 5 min = 12 visits per hour. To get to the target number of 1000 visits/visitors per hour, you need 1000 visits / 12 visits per user = 84 concurrent user. In total you will have 1000 visits and 3000 page views. Just as a side note: 5 min visit length and only 3 page views ...


4

Load Runner 9.10 does not support windows 7 and IE8(even LR 9.50 is not completely supported on Win 7). If Windows 7 is your primary platform then you need to upgrade to Load Runner 11 otherwise use Windows XP. Here is a link if you need more details http://www.jds.net.au/tech-tips/loadrunner-in-windows-7/


4

I'll assume you're just using JMeter as a type of Web Crawler with a fixed set of pages to consume, and you're only measuring response times and http status. Create a thread group of 100 users Place a CSV Data Set Config under the thread group The CSV contains one URL per line The CSV loads into a parameter named "URL" A HTTP Sampler is used where the ...


4

Install Google chrome. Install Blazemeter extension in chrome. Download JMeter and set it up. Lots of tutorials available for that. Record scripts with blazemeter and download them. Run the scripts in JMeter. Tada...


4

You have been developing this system for 2 years and have never performed any load testing at all? Preparing for expected load is usually part of the design. At this point, you are 2 years in, and may not have any evidence that your design is capable of supporting the required load. You do have performance and load requirements, right? With any ...


4

A few thoughts to add to the excellent suggestions others have made: Yes, full end-to-end load tests should ideally happen after code is stable. These will be used to find bottlenecks in the interfaces between different components of your code. This kind of load test is probably what your QA people are thinking of when you say "load test" - and I'd hazard ...


4

There's a lot of ways you can go here - if you want to stay with manual testing you can look to improve your ability to find and report problems, or you can add load testing or functional regression testing to your skill set. Depending on where you're based, the most effective method for you could vary - you'll want to balance between something you enjoy ...


4

'10000 users' using jmeter is pretty tough on single machine considering given configuration, the maximum number of threads supported by jmeter will be 300 to 400 depending on your script, eventually jmeter will get stuck with 'OutOfMemoryError' and it will create a very large dump file. it would be best to use distributed jmeter and simulate distributed ...



Only top voted, non community-wiki answers of a minimum length are eligible