Scope of testing in terms of cross browser testing and cross platform testing

Question

I would need support from QA experts to define the scope of testing for the following requirements

1. Cross browser testing - When we get a requirement to test a website developed in PHP and angular js. What are the main browsers which we need to test? What versions of it?
2. Cross platform testing - If the requirement is to test on mobiles, IPads and tablets. What are the versions we need to test. It's not possible to test all the devices but how do we ensure at least we tested few major devices? How to define the devices to be tested and its versions?

Answer 1

Preferably you look at the statistics of your current user-base, but if it is a new product I would do a form of risk-analysis using general usage statistics

There are some websites which have statistics on browser, device and resolution usages you can base your plan on (in random order):

1. https://netmarketshare.com/
2. https://www.w3schools.com/browsers/default.asp
3. https://www.w3counter.com/globalstats.php
4. https://gs.statcounter.com/browser-market-share/

Answer 2

Cross browser testing - When we get a requirement to test a website developed by PHP and angular js. What are the main browsers which we need to test? What versions of it?

We had the same issues in the past. Since we were testing with different versions for our frontend application. They should also run as webapps and of course they should also run on mobile devices...So our customer just said "should run it on all devices and on all browsers". But from testing perspective is is not possible to test on all devices & browsers. In our case we had lack of time and of real devices (because we tested on real devices also).

As Nils said, we also had a look at some statistics which browsers are used. In our case we used www.Statcounter.com and we made an overview about the devices with the help of the webpage:

• Different devices (Apple, HTC, Huawei...)
• Mobile versions (current ones, iOS 12.1, iOS 11.5...)
• Different usage of mobile devices in different countries, sometimes they are different (e.g. in China HTC and Huawei are more used e.g. than in UK)

But you should also discuss with your Business department, which devices or mobile versions are relevant (e.g focus on Apple Users because they are willing to pay for using the application/app). In our case we also discussed with our Product owner and created a browser matrix (this should also be documented in e.g. test concept or release policy).

So for my suggestion:

• Have a look at the lists e.g. from Nils (and make a report for yourself e.g. this month iOS Version 12.xx is used often e.g. 85% of Apple users are using it, Opera just 1,5% users are using it -> is this really worth to test it?)
• discuss with your business department / customer which devices /browsers are relevant (see browser Opera..)
• create somehow a browser matrix
• always have a look at the list which browsers and/or devices are runinng and hence are important. This has to be updated permanentely. We also did this in our project

Answer 3

We have numerous platforms available in the market and it is difficult to cover all the platforms during the QA tests. This is one of the common concerns faced by software testing services.

The best way to make a selection of the browsers/platforms/devices for testing is to analyze and gather details of highly used platforms on the production environment.

The analytics tool can easily provide these details. Besides that, we can also include few additional platforms in QA testing

Although, based on the platform usage details we are narrowing our scope of testing, still making sure no issues will be encountered to the major set of users.

Answer 4

First of all, before you jump on cross-browser or cross-platform testing, it is necessary for you to write your requirements considering the end-users. This will make things much more simplified in the long term and can even aid the development & QA process.

For cross-browser testing, consider the users you are aiming at. If the most part of users consists of Mac and Chrome users, you should aim for Safari, Google Chrome, and Firefox. If you have a wider user base and think have a budget and resources, consider taking it forward to other browsers as well.

For cross-platform testing, I would like you to have a broader approach and consider different operating systems for all their software versions for the past 5 years. This should include Windows, iOS, Android/Chrome, Linux, Ubuntu, etc.