How do we react as a real users while testing the web application

Question

Any ideas and suggestions to test web applications as a real user perspective. The reason while asking this question is while doing UI testing we are more concentrating on the functionality for the CRUD operation and missing the aspect of the user which is looking at the application.

Answer 1

Since having real users for testing each new version is not very feasible (at least in agile development) I would suggest creating personas (fictional characters).

I personally advocate for personas based on interactions with real end users. Even if the application is not yet in production you can recruit some potential users based on predicting who will buy / use the application when in production. Invite few end users (5-10) with some perks - free application / subscription, in-app tokens / money, company gifts (t-shirts etc.) as an appreciation for their time.

Then give them few scenarios (very generalized - e.g. Login, Make payment, Logout) and watch how they use the application. If budget is not tight you can also (with interviewee consent) film them using the application and/or record their journey on the device / application itself. Then you make notes about their usage of the application and their comments about it. See which usage scenarios were used by multiple users and which were not. Based on previous experience 10 end user interviews should suffice with material for creating at least 3 personas for testing.

Answer 2

On top of the great pre-release test ideas I would suggest dogfooding and using real usage data.

As someone that interacts with the product and use it mainly for developments purposes you are highly biased and can't see how it is actually being used in the wild.

Dog fooding, or using the product for its intended purposes by company employees, is not always possible or relevant depending on the product and company, but when it does it becomes a great tool for evaluating how the product is actually being used by real users while having a short and safe feedback loop.

Related to dogfooding are closed groups of beta testers, A/B testing of new or modified features for example using Canary testing. The former can be monitored using directly using feedback forms or systems but both would benefit from having a good telemetry system reporting about usage and problems.

Once you have a telemetry system in place (aka logging, data collection or diagnostics) you can start investigating how users really use your product, from simple number of uses to deeper insights about how much time is spent in each screen, loading times, how navigation is done in the software, file sizes etc.

Obviously data collection should follow privacy and other regulations and not intervene with proper operation of the product itself.

Answer 3

This looks like you are concentrating only on Functional Testing. Which is good, for start.

Then expand your testing for:

1. UI/UX testing. Is your website intuitive, is it nice, is it clear and understandable? Does it look good on different resolutions and browsers? Try mobile version, see if that looks allright. Check nagivation - is it easy. Can you go back easily to something you want to?
2. Performance testing. See how long you are waiting for search to be completed. Measure response times. See if waiting for something annoys you.
3. Security. Check for SQL Injections, XSS and basically OWASP top 10 ;)
4. Content. Check if there are no placeholders for text (or hints). Check if there aren't any content errors such as: wrong prices, non working links.
5. Browser compatibility testing. Check different browser: IE11, Edge, Chrome, Firefox, Safari. Try to mix that with different OS (ubuntu, linux, macos, android).
6. Beta testing. Think like a user. Or even better - hire a user (maybe your mum, gf/bf). Give them a seat, open this application and ask them for feedback.

There is so much more than just CRUD testing.

Answer 4

I can give you also an example about that what we did from business department side. We were also using a web based application for our solution which was car related. Means that the user has to log in the application in order to send rss-feeds or destinations to his vehicle. For new features we invited user for following purposes:

1. Testing before Go-Live Most of the users were working as consultants in other departments and had less/no experience within this product. Somewhow they are working as beta-testers (as already said above). But we prepared a spreadsheet with clear questions about the product, after the test (which took nearly around 30 minutes) we asked the user to fill in (anonymously) the spreadsheet. This was a good lesson for us since we also had detected new requirements and made suggestions afterwards to our Product Owner.

2. Check out whether new requirement is understandable This was the main point why we did the test with real life users. We weren't sure whether the new requirement was understandable. Hence you can prepare two tasks:

Open ended tasks: Imagine you have just implemented a shopping cart. We gave minimal explanation how perform a task. So you watch the user how he his doing this procedure. Make notes, how many time did the user spend with the action to get the item to the shopping cart. Is that action clear for the user? Important: Keep real life user talking so that you understand what he is doing.

Specific tasks Imagine you have just implemented a shopping cart but the shopping cart can only be done with a member card which you have to scan from your app (or whatever else). You give the participant special instructions "Please use the search bar in order to find the shopping cart for the xx product" etc. Then you can also check how fast the user (who has no idea about the newly implemented application) and when the user is spending (let's say more than couple of minutes) ...well maybe the requirement is not clear enough or to complicated for real life users. That is what we investigaged and this helped us a lot!

3. Content and language Content is already said by Michal, hence I don't want to repeat this. But we also invited user from real life perspective with different native speaking background! Because most of our text had been done by translation offices (e.g. dutch, swedish...). But we weren't quite sure if they were correct and understandable. Hence we invited users with different native languages also in hungarian and so on. And we were surprised that we found out that some text weren't writtten correctly. And beside that, they are also checking the content.

4. Usability testing is mostly done by UXler Well...this is a task from UX department. Because the guys from UX department are doing more stuff with eye-tracking. And dedicated to questions (is the system easy to learn? is the system useful? etc.). As business department we also covered these scenarios but we invited the UX team because we hadn't the tools for measure eye-tracking results and so on.

Answer 5

Assuming that you already have automated tests centered around application functionality, so your tests should be covering most of the application flow. One way to add user perspective to it would be to have your tests collect instrumentation data. This data can then be put against some user-friendliness guidelines you can compile, and an informative report can be generated based on it. For example, you can check whether the color contrast on a particular web-page is as per your guideline. Another example can be the count of steps/UI actions your automated test is taking to perform a particular feature/scenario, are these too many from a user-perspective. Also, as these extra checks might make your test runs slower, you may add property/flag to your framework so that these user-friendliness checks happen only when you desire. You may find the following links from software testing solutions helpful,

https://blog.qasource.com/how-to-test-your-app-for-compatibility-and-user-experience

https://blog.qasource.com/5-reasons-why-your-qa-team-needs-domain-knowledge/

Answer 6

In addition to Prome's excellent suggestions, I would add:

1. Start by specifically identifying who your users are. As an example, for the products I test, the users are order-fillers in a warehouse (godown), their supervisors, and the IT staff who will maintain the product.
2. Do whatever research is needed to find out some key facts about individuals in those roles. Key facts would be:
3. How much knowledge do they have about the product itself? Can you assume they would be trained to use it, or must anyone be able to use it without training? (This tells you how self-explanatory and intuitive the interface must be.)
4. How often are they using a particular feature? Is it once every 2 or 3 months, or is it hundreds of times each day? (This tells you how efficient and easy to use that feature must be.)
5. What are their priorities and goals, and how would they expect the product to help them? Are they more interested in working quickly, or are they more concerned with getting accurate results? Do they expect the product to provide them with information to help them accomplish the tasks they're performing? (This helps you evaluate things like the GUI layout, the organization of screens, the functions available from each screen, and so on.)

Once you have learned all these things you are now ready to "roleplay." You can more easily imagine yourself in the user's position and evaluate the product as an actual user would see it.