JMeter threads vs users

Question

I want to know whats the difference between hard-coding user (1 user) credentials to a HTTP Request while threads is 20 and using a CSV Data Config that reads 20 user credentials while the thread count is 20.

To my understanding there are 20 threads generated regardless on the method, what I want to know now is the logic that is taking place. Are this methods similar?

Answer 1

Shorter answer in single line is "Yes, both types uses same method but CSV if preferred because this provides real world data & analysis."

Long answer, for JMeter or any other performance test tools (as far as I know about others), it doesn't matter if you are using any CSV/Excel or any other data input method or not for creating load, as long as the number of threads are same (exception is related to application, which I tell later in answer). Performance test tools are concerned about the Load not about the input methods.

But, yes using parameterization in Performance testing is a recommended, preferred and good practice, because of many reasons like:-

1. Want to test how the application react with different type of users
2. Want to test how the application react with different type of data sets being returned from a search query.
3. Want to achieve a more closer real world scenario

See, what a load test tool do? It only simulates 100s or 1000s of users and provide their results; which can't be generated manually. It means tool has to perform the work and task of 100s of users and every user is different in real world (with their test data, their permission set etc.). That's why we use CSV files to feed performance tests, so that it can provide results as per the different users load.

We do performance testing, to achieve benchmarks and to satisfy all users need, what if we satisfy only 1 user need and generalize it to rest of the N-1 users.

Another, reason is for making the performance tests Robust, for example (based on applications I have load tested)

1. If application is having a business logic that 1 user can login only 1 time and if same user login again, then he/she will be logged out from 1st session. In such a case you have to pass all required 20 or more users via CSV files.
2. Another is, lets say your application have some Business logic which accepts current date or some ID which is unique in application, now for the former part if you need to run your tests over a week or month, then either you need to change the script or just change the CSV file value and in the later case as the ID should be unique, so for creating 20 records you can't use the same hard coded value, hence you need to parameterize values
3. Your application has different role users and permission set of different role of users is different and so the logic behind getting permission set. Now, if you use same user all the time, then either because of caching or because of getting same permission set again and again, you will not get the generalize result, it will be specific only for that hard coded role user

In addition to it, you will see the benefit of using CSV files, when you need to do volume testing (part of performance), here you will need to first create data with different data sets and then do performance testing, here using CSV files you can easily create multiple sets of data and increase size of your database. Load testing a Facebook shared image with 1000 comments with 100 users and load testing a Facebook image with no comment with 100 users both are different scenarios and with CSV file you can easily create 1000 comments with different or same number of users.

Because, firing a select query over 10 record table and 10,000 record table will show performance change and for setting benchmark you will need to insert these 10,000 records first in your application and then display them through application.

Answer 2

It depends on your test scenario. Best practice is simulating different users each having its own session. By "session" I mean not only username/password combination but also the whole bunch of entities which are belong to user like:

• Cookies
• Headers
• Cache
• etc.

So when it comes to web application performance testing you need to make your test as realistic as possible and simulate a real user using a real browser. So make sure you follow the next steps:

• Real browsers download images, scripts and CSS files. To simulate this behavior configure your HTTP Request Sampler(s) to

  • Retrieve All Embedded Resources from HTML Files
  • Use concurrent pool of 2-5 threads for it

  The easiest and recommended way to do it is using HTTP Request Defaults

• Real browsers download images, scripts and CSS files but they do it only once. On subsequent requests these entities are being returned from browser cache. Add HTTP Cache Manager to replicate this behaviour.
• Add HTTP Cookie Manager which represents browser cookies and deals with cookie-based authentication.

See How to make JMeter behave more like a real browser for more detailed above recommendations explanation.

Answer 3

@ Puven Mannen Test users are simulated using threads in JMeter. ( You can check here more - http://jmeter-expert.blogspot.com/2010/03/how-apache-jmeter-simulates-multiple.html )

Answer 4

There is a good thread on this topic here that I have used for my reference. I think you will find it useful what is the concurrent users in load testing using Jmeter

I think an important not form that thread is 'Also JMeter concurrent threads are not "concurrent users" as real-life people do pauses between actions on web pages. If you don't set any pauses in your tests then 20 concurrent threads will stand for something like 200-250 concurrent users.'