I have made a website. I visited this website using different mobile browsers and I am getting an error related to visual display and button click event.

Do I need to test with a large number of devices, or will using emulators be enough?

  • Re-write your question so it's taken off of 'hold'. I believe you're saying that you are using an emulator to test a website, and on every mobile browser you tried there was a UI error and a button was not working correctly. Are you then asking if you can trust the results of your browser test, or are you asking for developer support to fix the issue? If you do need developer support, then try Stack Overflow instead. – Edward_Haigh Aug 21 '17 at 13:57
  • Some error - which one? Same error? – Peter M. - stands for Monica Aug 21 '17 at 14:16
  • For the most part, emulators are fine since most of the errors for your application will probably be related to screen size or framework versions that emulators will handle. However, if a small country might blow up if it doesn't work properly, you'll probably want to put in more investment to make sure it works on real devices. In other words, you'll want to evaluate the cost vs. risk, and the risk is probably low unless you have very serious consequences in case of a failure. – mrfreester Aug 21 '17 at 14:29
  • 2
    Possible duplicate of Mobile application testing on emulators or devices – c32hedge Jan 8 '18 at 13:58

There are so many devices in the world that you will ruin yourself if you try to buy many devices. I recommend some emulator website like https://www.browserstack.com

You can select the operating software, the browser and its version. And without doing automation on it, it's not that expensive.

| improve this answer | |

Not in my experience, I've seen multiple examples of tests that pass on emulators but fail real world. Mostly these are non-functional or look and feel issues. If your automated tests pass in your emulator(i.e. everything is functionally correct) how will you know that it looks OK, is correctly styled, alignment is correct etc? These issues can be very important to clients.

I think you can 80% trust emulators but there's still no substitute for a physical device and a good QA. Emulators also limit mobile\app testing which should take into account bandwidth usage, battery drain and so on.

Given that over 95% of mobile sales are now Android\iOS you don't need much of a device farm to get decent coverage. We mostly test on desktop browsers, latest IPhone\Android and maybe a tablet or two. We also use Ghostlab to chain devices then run one complete pass across the device farm near the end of QA cycles.

TLDR - I'm not against emulators but I don't trust them 100% I still ask my teams to do some manual spot checks on real devices(belt and braces!).

| improve this answer | |
  • I agree but emulators are still a good solution if you don't have many devices and a budget to get some. – LeBrequin Aug 21 '17 at 14:41
  • This question is about automated testing, but I agree manual tests are still important especially for visual issues. However, products like applitools try to do automated visual testing as well, but that has always seemed like too much maintenance for automation to me. – mrfreester Aug 21 '17 at 14:46
  • But if we have 1000+ permutation and combination of browser and OS, then what will be preferable?....we can not buy 1000+ devices-it is not the solution.....Is Emulator efficient for same? – saif sadiq Aug 21 '17 at 14:48

Testing on mobile devices consider the following:

  • Do you have any user statistics about their preferred browsers (try to use data from your user base a previous workplace was significantly still using IE browsers, - go figure!)
  • Search online for sites that discuss mobile device compatibility. Research common failure areas
  • Ask developers to define guidelines about when not to implement bleeding edge features, this might help deployment time.
  • There are plenty of cloud based "real device" testing labs available, choose one and keep it cheap by only targeting devices that your users have.
  • buy one or two devices that represent a broad cross section of the weird and wonderful features (eg. Sony for instance love to be cutting edge).
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.