4

Do we have to test on emulators or a real devices for testing mobile applications?

Improve this question

7 Answers 7

Reset to default
6

Do we have to test on emulators or a real devices for testing mobile applications?

Why not both?

Some functionality of mobile applications can be well-tested on emulators. Often this mean UI, basic features, etc.

Other features may require real devices, particularly if you have device-specific functionality, or functionality that takes advantage of specific hardware features.

One approach would be to choose a device and its associated emulator, then test using both. Make note of what could have been tested equally well using just the emulator, and what would require the actual device. Then let that experience be your guide for other devices, and for future versions.

Improve this answer
1
  • Also, make sure that u test with different versions OS
    – Vivek
    Aug 25, 2015 at 3:33
5

Do we have to test on emulators or a real devices for testing mobile applications?

I agree with "Joe Strazzere on this point, that one should go for both Emulators and Device testing as per the process and that too first test on Emulators and then on actual devices. But I would like to add few more relevant points to the answer.

Whether you should go for emulators or for devices, it depends upon number of factors.

Budget of the project/application:

You should ask client and management that, Who will bear the cost of devices? Because at the e.o.d you have to show the ROI for the investment made. So don't go for too many devices in one go. See what devices are available within your organization and what devices client will provide. If you don't have sufficient budget then go for selected device only and for other level of testing use emulators. Which are generally free and cheaper than devices.

Timeline of the project:

You can't go for multiple devices and multi-platform testing, if you have to deliver early. Quality is time consuming, but if your client doesn't have time, then you have to skip the emulator testing and jump over the real devices. End users are not going to run the application on emulators, so device testing is must. Also, if you pull extra resources for testing then it will again cost you more.

Managing the Resources and real time behaviour:

With the use of real devices you can validate the real time scenarios like Phone call in between using mobile application, interaction with other applications etc. Also, with the use of real devices you can manage resources and time, it may not be possible for testers to access the emulators in off-office hours, but you can handover the device to them to play around (Exploratory testing and Adhoc testing) at home, during travel, during weekend etc. It may sound awkward but it will definitely add some value to your testing and bugs to your tracker.

Ease of access:

If you will directly go for devices then it may be more time consuming in beginning, as during the initial phase of testing, you will find lot of issues or you can say that the Rate at which issues are being encountered will be high. So, during that duration you will get yourself killed when you again need to rush to your machine for logging that issue, taking snapshot from the device, sending it to your machine (email or data cable or other means) and then logging the bug. This will cause a continuous interruption in your flow of testing and execution of test cases. This problem will get resolved if you use Emulators in beginning and once application is stable and Rate of issue encountered is decreased, you should move your testing to device, decreasing your hustle and saving your frustration and time.

After all the above analysis, one thing is sure, that you must test the application on atleast one not-outdated device of each OS e.g. if your application is going to support iOS and Android then, test it on one actual device of each type.

Which device should be targeted?

In addition to it, get more information about the end users of the application, where it is going to launch, study about that country and get the data which type of devices are more widely used in that geographical area. Then based on that study select the devices. You can go for the cloud based device testing too, using that you will be having access to multiple devices from single machine.

We had done this kind of study and analysis when we have to launch mobile application of one of our web-application for the European client.

Improve this answer
1

It's your choice actually. You can choose to only test your applications on emulators and not real devices- but that would be choosing a dangerous path.

We want our testing to closely resemble the real world scenario as much as possible. So testing should be done, as much as possible on real devices. Because real devices would provide your the real world scenarios like Interrupts- like calls, Network fluctuations etc.

Improve this answer
1

Short answer is: Real Devices.

But the point is if you can afford it. As it could be relative expensive if you want to test multiple devices.

So:

  • If you only have limited budget/resources, then use real devices for Android and simulator for iPhone (since iOS simulator is proved to be stable and trustable well).
  • If you don't have any budget, go with simulator/emulator.
Improve this answer
1
  • Could you add "the proof" for your "since iOS simulator is proved to be stable and trustable well"?
    – bish
    Aug 18, 2015 at 18:07
0

If you need a reason on why you need to test using devices as well as emulators you need to look no further than this article:

http://www.fastcodesign.com/1669877/infographic-heres-why-designing-android-apps-is-so-hard

Designing interfaces for 11000+ screen sizes is too much. Trying to shortlist that into a couple dozen is still more phones than most developers can afford. You will need to find other ways of testing your application.

You can test on real devices using the Xamarin Test Cloud:

http://xamarin.com/test-cloud

And you can test on emulators as well:

http://mobiletest.me/

Improve this answer
1
  • This doesn't answer the question. Asked question is not about the name of devices or emulators, it is about which should be considered and why?
    – Dhiman
    Aug 18, 2015 at 18:36
0

Well it depends on your project, if devices are readily available that's the best way to test. I have found many a times issues found on real devices aren't reproducible on emulators. So, the best bet will be to test on actual device and certify quality.

Improve this answer
0

As other users have said you should do it on both. It comes down to what your needs are and $$. Getting real devices to cover the entire spectrum is expensive, often unrealistic endeavor for a small company and is just not practical for testing.

Your lab should be a combination of both solutions. Cloud services can get expensive if you rely on them too much. Services like Testdroid provide a great array of devices but depending on your subscription plan and the length of the tests you run and how often, your bill can grow pretty quickly.

Emulators can only get you so far since you'd want to test some type of integration with your app or site and although close at times, emulators are not the real thing.

Chrome has a very decent solution on the developer console, good for checking layout and rendering. Genymotion is another tool for Android that does the job.

Real devices are great for business rules and exploratory testing.

Overall, the demographic of your user base will determine how much freedom and risks you can take while developing your mobile solution.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.