Is it useful to do separate load testing on a mobile App? I mean a mobile App is just a light weight client/viewer (like a browser) which internally calls some services/web API's.
So, can we say load testing of a mobile App is actually load testing of the web API...?
If not which tools can we use to load-test a mobile App?
My scenario is 5000 users using the App simultaneously.
Is there a load testing of a mobile app?
Short answer: Yes there is.
Long answer: Well you can test the performance of your app on the device. You can test,
How much memory it occupies?
How much battery does it utilize?
What happens if you have several other apps open and then open your app?
Does it hang up the phone?
How much time does it take to process a request?
How does it work on low speed networks?
Well then you come to the server side. Here you can use tools like Jmeter. For this you will have to get the HTTP(s) requests your app makes and then you can generate a load for the same request. BUT That is for your server. If the server fails to handle the load, your app will receive ERROR 500 from the server.
So can we say load testing of a mobile app is Load testing of Web API...?
Answer: If your app makes HTTP requests to a remote server interacts with it using some sort of web service then yes you can say that.
If not which tools can we use to test load on an Mobile App?
Answer: Like I said above you can use tools like Jmeter to do load testing for your http(s) requests.
I would recommend using Apache JMeter for this, it comes with Proxy server so you will be able to record your test scenario and replay it with increased number of virtual users
You can also use Mobile Recorder service which is the simpler way of capturing mobile device traffic and converting it into a JMeter test.
Question 1: Is it useful to do separate load testing on a mobile App? So, can we say load testing of a mobile App is actually load testing of the web API...?
Answer 1: Depending on the nature of testing it is always advised to run at least a subset of tests on real devices. The resources available on mobile devices are hugely different in comparison to PCs/Macs. There are emulators which a available to help simulate various conditions and are hosted on separate systems i.e. not on mobile devices which help developers to unit test at an early test and minimise defects but that does not eradicate the need for running the tests on mobile devices. E.g. You are running load tests for streaming a video file for 1 hour for 5000 concurrent users. This may work on PC/Server where you did your unit tests but may fail on the mobile devices and may lead to crash of the device. Therefore I always recommend to run it. These factors must be kept in mind while formulating test approach and identifying tools and building a test environment.
Question 2: If not which tools can we use to load-test a mobile App? My scenario is 5000 users using the App simultaneously.
Answer 2: Looking at your scenario, you can use JMeter and HPLoadrunner as it is not the app which is under load, in this case it is login component or authentication server/token generation or streamining/drm servers which are under load, so you can trigger it from the APIs. But if you want to trigger it from real mobile devices then their are costly tools available to provide that kind of environment, however it will not be cost effective to ramp up to 5000 devices. Alternatively if you are testing for a Tier 1 company then they generally will have trial phases with friendly customers, in that case they will be the ideal people to trigger these tests and generate load, however may be late in the cycle.
5000 users using the App simultaneously is mostly load testing for the server.
But why do you care about names and definitions? Just think about what do you want to test your App for.
"Load" can be as simple as a monkey-user hitting buttons, or it could be heavy computations done on a loaded CPU, or trying to transfer a large amount of data over a poor network.
To answer your first question,
Is it useful to do separate load testing on a mobile App?
It is very useful to do a separate load testing for a mobile application because, the raise of several mobile network technologies, such as 2G, 3G, 4G, etc. has led to variations in response times with increasing load on the server. The application owner needs to be aware of these differences so that it will help in making the application more effective. Additionally the client being a mobile device has plenty of physical constraints such as limited CPU, RAM, internal storage etc.
These constraints will be a hindrance when, for example, an animation which runs smoothly on the desktop renders itself with lag and jitter on the mobile.
All of these need to be considered while conducting performance testing of a mobile application if one wants to simulate a scenario close to a real time condition.
Even if you are only trying to understand just the server performance, it would be advisable to emulate at least the network profiles so that there are no shocks while in production.
The rest of the answers have been quite elaborately stated above by @TESTasy
Below are few tools which can be used for load testing of Mobile app:
I work for IT Central Station, and the IT professionals in our community have contributed reviews for performance testing and load testing mobile apps. This is one of our most popular comparison pages for load testing mobile apps: https://www.itcentralstation.com/products/comparisons/apache-jmeter_vs_hpe-stormrunner-load (note that you need to register to read more than one review) I hope it is helpful.
This Senior Director of QA writes, "Valuable Features:
Flexibility and speed to provision and start large (high concurrency) performance testing cycles.
Improvements to My Organization:
This software solution has allowed us to run larger, high concurrency performance testing cycles in a very repetitive manner…a requirement for validating our application solutions and supporting infrastructure for customer facing web apps.
Room for Improvement:
Out of the box reporting could be improved.
Use of Solution:
We have been using the solution for 9 months." You can read his full review here: https://www.itcentralstation.com/product_reviews/hpe-stormrunner-load-review-32795-by-rich-harman
Performance testing for a mobile app is essential. If your mobile application takes more than 3 seconds to load a page, you lose trust. Your app loses it's professional seal of approval and people will go from not engaging on it, not buying the products it offers, to discarding the app completely. Companies lose billions of dollars each year from users loading their ecommerce shopping carts with stuff, then right before they click buy, the app takes too long to load -- and the users abandon the purchase right before completing it.
For performance testing, try SeeTestNetworkVirtualization:
