My company is considering offering an application for Android devices and iPhones. What are some of the challenges of testing a mobile application that are different from testing a PC application (or an application that runs in a browser)?
Off the top of my head:
- Memory footprint. Allocations that "rarely" fail on a desktop OS may fail on a mobile platform
- Battery impact - on a desktop app, it's common to have a thread polling, or a background task. On a mobile platform, activities like this can drain battery pretty quickly
- Desktop applications often assume reliable network access. Mobile platforms don't have consistent or reliable network access
- If you're using automation tools (or other tools) to help test your desktop applications, they likely won't work on a mobile OS. You'll need to look for alternate tools or mobile versions if these are critical to your testing.
- Location awareness -- if the app uses GPS or Cell-based location information, you need a mechanism to test related cases -- no location info available; unexpected locations, etc.
- Screen sizes and orientations -- In addition to different screen resolutions (especially on Android) and phone vs. tablet testing, most mobile platforms can dynamically switch from "portrait" to "landscape" mode without warning.
I'm sure there are big areas I'm missing, but the community should take care of that.
Some thing spring to mind:
- Cellular coverage affects not only your internet connectivity but also loads the CPU
- Mobile devices CPUs are usually slower
- Mobile OS's impose a lot of restrictions that doesn't exist in Desktops, for example IOS handling of multi tasking
- Compatibility is a pain, Android or IOS behaves slightly different on different devices, even screen resolution is different.
- Battery impact has more sides to it, mobile OS's have tricks to save power so you might find yourself in places you didn't expect. One example are data calls that can go dormant, taking seconds or more to resume.
- Cellular data introduce large delays and packet losses ratios, for example mobile TCP timers are much larger, making detection of connection loss more difficult.
In many ways, it's more like a PC application than a browser app -- you must test with the expectation that once a user downloads your app, they may never update it, or may not update it as quickly as you'd like. In addition to potentially buggy code in the field, you may also have to test back-end systems against various versions of the app.
Something that need to consider for mobile apps:
- External Interruptions like SMS, MMS, Phone calls: You are running your app in the device and suddenly receive a SMS, MMS, Phone call or any other notification.
- Different device manufacturers with different device configurations like Android and Window Phones have different manufacturers Samsung, HTC and so on with different processors, display resolutions etc.
- Supporting only mobiles phones or Tablets or both
- Different internet connectivity networks like WiFi, 2G, 3G
- Supporting NFC (Near Field Communication)
- Device Battery Factors: Please note that the different status of the Battery has impact on your app. If battery is drained, full or charging then it will have some impact on your app.
- Location based apps: These apps must be tested in real locations and with GPS On/Off status.
- Some platform specific testing like testing your app on iOS devices when some Restrictions are enabled like Disabling Camera, Disabling adding events in calendar, disabling In App purchase and then checking when you try to use these features in your app
In addition to the excellent points already mentioned, data synchronization to a mobile application is a rich source of bugs, and, even worse, of unwise specifications. A bug I’ve seen in many incarnations is that the mobile device can’t handle all of the data in a desktop application, so the programmer limits it in a way that seems reasonable to him, but not to the user, generally without documenting his decision or even letting the test team know.