What is the difference between testing a website on the same server acting as client running JMeter test and testing a website which is on a different server? Which is the best way to test? And why?
To avoid mutual interference it is much better to run JMeter and Application Under Test on different hosts. Theoretically you can have them on same machine but results won't be reliable.
Load Testing is not only about getting main metrics and KPIs like response time, transactions per second, throughput, etc. - a lot of other information needs to be considered during test results analysis to get to the bottom of the performance issue.
For instance, you are not happy with average response time. The reasons of application slowness could be:
running out of CPU/RAM/Network or Disk IO. That's why you
- shouldn't have JMeter running on the same host
- should consider monitoring application under test health
"bad" application/web/database server configuration. All these components (including JMeter by the way) come with very low default values which are fine for development and/or debugging but definitely not enough for load testing or production.
- network issues. If you observe high Connect or Latency values - network is a bottleneck. You will never see it having JMeter and application under test deployed at the same machine
- application under test poor code quality. In that case you'll need to use profiling tool to identify the most "hungry" application parts and flag them
Network is the highest cost resource in terms of adding to response time. When you run on the same server as the web server you are processing requests and responses across the system bus of the system, which is order of magnitude faster than the network where the clients will be collected. You are also shaping the performance curve of the server to handle the extra load imposed by the execution of your test tool as well as the responses. The execution of load would not be present in production on the host.
When you consider the finite resource model of CPU, DISK, Memory and Network, your staging of load on the same server as the target impacts negatively the finite resource pool of CPU, DISK and Memory on the service under test compared to production and minimizes the impact of the most expensive resource: Network.
So, from a staging perspective and a match to production target configuration, this would fail an audit. By the way, this is more of a process issue than it is a tool issue.
That depends on how you use JMeter. The tool was originally designed for performance testing. If you run JMeter on the same machine as your application, your application may run slower or encounter other resource contention issues compared to running JMeter on a different machine.
On the other hand, some people use JMeter for functional testing. For that use case, running JMeter and the application on the same machine is probably fine.
(As an aside, I do not recommend using JMeter for functional testing. That's not its design center and there are better tools for the job.)
The second one is the only way. It is strictly recommended that you do not run a JMeter test from a server where you already have the application installed which has to be tested. Jmeter test consumes a huge memory resource from server or machine. So if you have JMeter & the application to be tested on the same server, you can't expect to have accurate result from JMeter. It would be either bogus or false results. If you run JMeter test on that way it would be a worthless attempt. Why would you do that anyway!!! Try to make your test server have only that Application running which has to be tested. Make it as much isolated as possible. In this way, you would have a reliable test result. For more reliable tests using JMeter, follow the "Best Practices" from Apache JMeter documentation. http://jmeter.apache.org/usermanual/best-practices.html
There is no such restriction on the idea of running the JMeter test and hosting the application under test on same machine or on different machines.
It is always considered a good or rather smart practice to host the application under test on one server and execute JMeter with its test script from another system/machine.
Now, the question here is why you should choose to do this?
Well, performance testing is not only about sending HTTP requests and viewing the response times in some table or Graph. Lots of things need to be considered here;
The performance of the server hosting application under test. It's CPU, RAM, Disk I/O and Network I/O utilization.
Along with that JMeter when running also takes up a lot of CPU, RAM, Disk I/O and Network I/O of the system on which it is executing.
If you have the application under test and JMeter running on same system, you won't be getting proper results or rather no result at all. Your system make crash or hang up! That is why you should not run JMeter on the same system as application under test