Similar questions have been asked a number of times. The general consensus is that UI automation tools such as Selenium are not a good choice for load testing. There are a number of reasons why, but the main one is that it is a LOT slower. You can easily spin up thousands of concurrent requests in a second in a tool like JMeter which to reproduce in ...
According to Visual Studio: Load testing in the cloud, here are the steps to switch the test from using Visual Studio Online to running locally:
Simply open your existing project using Visual Studio 2013 first.
Within the Solution Explorer, expand the 'Solution Items' folder, then open Local.testsettings.
Edit the test settings file to configure your ...
There is never a general industry-wide answer to this sort of question. What might be "Key" for your company may very well not be "Key" for my company.
To answer this question for your specific context, find out what matters to the stakeholders you serve. Figure out what is important, what is not, and then determine what indicators can help you determine if ...
I have had some luck using Grinder. It is Java based but you can also write your scripts in Jython or Clojure.
You said that you would like to take advantage of your team's current Selenium scripts and expertise performance testing. You did not describe your regression test scripts, but you may want to reconsider whether they are appropriate. In ...
It is very likely you will get differing results. Especially in response times. This is mainly due to latency caused by the Geographical distribution. This will become important if response times is one of the metrics you are using to understand the website's ability to cope/scale under load.
A simple reflection of this is;
go to http://amazon.co.uk/
Answering the comments above,
Well I had Java and Jmeter installed correctly with ClassPath and everything. I also tried running Command Prompt as Administrator but got the same result. There was no problem in that. That is why the GUI version was working properly in first place.
After a lot of R&D and Googling I found out that the problem was with the ...
No. The items you mention scale very differently and there are far too many factors and resources that will get used.
For instance if I time requests on a local server using an application I'll find things like
1 user = 2 second response average (time per request)
10 users = 2.5 second average
100 users = 2.5 second average
1000 users = 20 second average
From Apache JMeter's website we get this,
Ability to load and performance test many different server/protocol
Web - HTTP, HTTPS
SOAP / REST
Database via JDBC
Message-oriented middleware (MOM) via JMS
Mail - SMTP(S), POP3(S) and IMAP(S)
Native commands or shell scripts
So I'm guessing you can't use it to load test Swing Application.
I'll assume you're just using JMeter as a type of Web Crawler with a fixed set of pages to consume, and you're only measuring response times and http status.
Create a thread group of 100 users
Place a CSV Data Set Config under the thread group
The CSV contains one URL per line
The CSV loads into a parameter named "URL"
A HTTP Sampler is used where the ...
Does this actually catch, before production, many of the "surprise"
problems we might anticipate? Or is there a more fundamental flaw in
the approach that will cause deleterious changes to pass testing and
You are wise to have a test system that you can use for catching performance issues, but your "scaling" approach is flawed.
I learn Gatling using the resources I found online and experimentation. Just google "Gatling tutorial", there are plenty of links to pages showing Gatling basics, many focus on different aspects so it's useful to read a number of them. I
always found official resource helpful, The Gatling documentation 3.0 is pretty good as well, they also have an Advanced ...
All right, So Tristann kindly revised his original question to include more details in terms of a scenario. So I'm adding a second answer to more directly address it.
Firstly you'd probably want to ask a few more questions about what the customer is most concerned with and what they want tested, here's a small sample:
what's the duration of the shoppers
You could also check out Gatling, which is an open source stress tool based on akka.
On the Gatling homepage, you'll find a direct comparison of how your test specification will look in Gatling vs. in JMeter. This might help you decide between these too.
If you want to create massive amounts of load, I think there's nothing better than akka as an engine.
You have 1 Master and 2 Slaves.
The total load being generated is 500 + 500 = 1000.
Since you mention that the server health is good during the test execution. Yet, you are getting an Internal Server Error (Error code 500).
Here's what you should try,
Run a test of maybe 500 user and while the test is running try to access the URL in a browser. If the ...
From my Experience I would Say Yes to VSTS Test Edition
I used VSTS Continiously ran load tests for 4 Hours, Simulated Maximum of 60 requests / sec with 4 Test machines, OLTP Application
Web Service / Biztalk / SQL Server / Replication / Service Broker based architecture
VSTS Tests - Easily bind data to tests. This is pretty straight forward ( http://blogs....
Randomizing the file BEFORE use is a much better approach.
It will much more efficiently (especially in the case of significant amount of entries) to randomize the order of entries in your CSV file before the test - and then read prepared data sequentially, in way you like (via CSV Data Set e.g.).
You can do the preparation using e.g. perl script or even ...
This is a pretty interesting problem, actually :)
You could try using a Markov Chain Monte Carlo approach. I haven't worked with these types of models myself, but the idea is that you start with a given long-run distribution and use Monte Carlo modelling to develop the actual Markov chain. I know this is a pretty well-used technique for simulation ...
Yes, there is a way possible to do this. You need to add a 'HTTP Request Default' configuration element from 'Configuration Elements' option i.e. Right click test plan and add this 'HTTP Request Default'.
In this 'HTTP Request Default' there is an option Connect in 'Timeouts (milliseconds)', you can specify your connection timeout value in this field and ...
If you have an average visit length of 5 minutes, a single user could create 60 min / 5 min = 12 visits per hour. To get to the target number of 1000 visits/visitors per hour, you need 1000 visits / 12 visits per user = 84 concurrent user.
In total you will have 1000 visits and 3000 page views.
Just as a side note: 5 min visit length and only 3 page views ...
Based on the answer and comments I had received what I wound up doing was:
Use a Once Only Controller with a BeanShell Sampler to read the file into an array, then when I need to pull that data I just use a BeanShell PreProccessor with:
Random random = new Random();
vars.put("MyChosenVariable", vars.get("AvailChoices_" + random....
you could use a plugin, although there is a simpler way. You can create a CSV file with the list of usernames and passwords you want to iterate through and then create a datasource. When you execute the web test, it will iterate through all of the items in the CSV file, one line for each test execution.
Based on what you outlined above I would probably ...
If the system needs to scale to many concurrent users or work with lots of data, I would say load testing should be started as early as possible. This way possible problems in the application architecture can be found in a phase where it is still possible to fix them.
Let's say that the database schema is such that some queries will be ...
I would recommend JMeter as:
JMeter: free and open source
MSVS: Test Professional costs around $2000 per developer.
MSVS: Windows only
JMeter: HTTP, FTP, JDBC, SOAP, TCP, JMS, SMTP, POP3, IMSP
However if your company has Microsoft products based development infrastructure, like Team ...
Option 1: You can keep the user count as 1 and Loop count as 100 in your thread group. This way your script will be executed 100 times, without concurrency of users and one user at a time. you can use CSV file if you want to change the user login credentials and other data at each loop starting i.e. different data for different user.
Option 2: Another ...
A web search of the terms resulted in,
Scalability Testing, is the testing of a software application to measure its capability to scale up or scale out in terms of any of its non-functional capability.
Performance, scalability and reliability testing are usually grouped together by software quality analysts.
The main goals of scalability testing are to ...
I wrote about concurrent users and numbers in a blog post: http://blog.xceptance.de/2011/06/07/get-the-right-load-mix-out-of-a-few-numbers/
Wait… where are my concurrent users?
This is simple: “concurrent users” is
an inaccurate way of describing
traffic, so we have not used that
number yet. Why is that?
To get to the bottom of that, we
There are a lot of performance testing tools:
Setup for the most part is very easy. You could run each of these packages on a local VM; however, if you want to simluate large loads of traffic you need to have a machine with a little more RAM and Processor.
Since this is an AJAX application you might have to simulate ...
I have no doubt that the IBM Rational tools could allow you to automate functional and performance testing. However, there are many free and open source tools that also provide the same (or often better) functionality. From my own experience working with proprietary test tools such as Rational (my experience was with Rational years ago, Silk and a bit of ...