4

I have an Android app that allows to make video calls and I need to test the quality of the video and if both video streamings match correctly.

How would you test it, in an automatic proccess?

I thought about putting 2 devices, one in front eachother, and in the middle a tablet that records with one camera one device, and with the front camera the other device and then compare both videos, but this solution seems too much complicated.

1
  • 3
    A video call is intended for a human end user, it is best tested manually. – Yu Zhang Mar 12 '18 at 9:45
3

and then compare both videos

Simple comparison wouldn't work, the video is being compressed and decompressed and delay is being added in the middle so even in the best case you will need to align the streams.

Let's take it step by step-

Theory

You want to use some algorithm to compare a reference video stream to the received stream at the remote side.

Playback

Simply using the camera would make comparison problematic since you don't have the reference stream, also conditions might change from test to test making the results inconsistent.

Usually the best idea would be injecting a premade video stream, performance wise you would want to store it on the device on an SD card.

Record

There are multiple options here- You can use adb's record tool to simply record the screen

On some phones you can output the stream to hdmi and record it from there using traditional tools

If possible your software can output the decoded video stream to file instead of the stream, the main drawback here is that you don't test the software end to end.

Analysis

Sync

You will first need to align your video streams in time, one way to achieve that is to insert some sycn pattern into the video stream itself for example 3D bar codes that can be analysed by software. Other options could be having something identifiable in the stream like a moving element of somekind.

Assessing quality

The simplest and most reliable way is probably suing MOS and especially using PSNR since it is computationally supported on many platforms and environments

Challenges

storage

Video consumes a lot of space, you want your original and results to reside on an SD card or internal memory during the test while still keeping high quality of the video itself.

If your storage is limited you can compromise on synthetic or repetitive video streams.

performance

As explained above you want a lot to be done on the mobile itself (PSNR can be done offline) and this affects the performance of the software itself and the mobile's OS.

There is not a lot that can be done, but it's always a good idea to keep that in mind while testing.

Tips and Tricks

While not perfect I've solutions using still images out of a video stream being used to assess video quality.

The simplest way is using adb to get screen captures, a somewhat better way is using high speed camera to capture the screen.

Both solutions requires time synching, the external camera could be mounted with the mobile inside a box (a big cardboard box works just fine).

Both would compare to a reference image captured before with some margin for errors, for example using ImageMagick

Summary

My experience is not totally postie on this, it is doable but the tests tend to be be unstable.

This is especially true on Android and when using high performance video encoding/decoding since it is hardware dependent and changes from model to model and with changes of OS versions.

1
  • Thank you very much for your reply! I will work on it and let you know if I figure out something :) – Skavee Dev Mar 12 '18 at 10:23
4

There can be multiple test case of the application based on context. Some important Test case scenarios as follows

  1. Video calling icon should display when another users are online with webcam.
  2. Clicking on video calling icon should connect to the particular person with calling ring tone and callers name.
  3. Video resolution should good in all browsers and resolutions.
  4. video calls should start with sound enabled automatically.
  5. Video can be mute for temporary but Sound can be played. (And vice versa)
  6. Reconnect to the user is possible in the low network or quick disconnect/ connect of wifi connection.
  7. Chat/ text can be done along with the video calling.
  8. Joining/removing more people into same chat is possible.
  9. If one person is disconnecting the chart then the connection should not get fail with other persons.
  10. When the main caller will close the chat, then it should disconnect from all the persons.
  11. Video recording/sound recording functionality is working fine during video chat.
  12. Mute/ un-mute to sound functionality is working fine.
  13. Video calling window should be maximized/ minimized.
  14. The user should use other functionality of PC/ laptop/mobile, during the video call.
  15. The screen should divide and display all the person pics when the video call made for multiple persons. And disconnect with the person should remove the screen allocation for him/ her.
  16. If the person is not accepting the call request, then the call log should generate and should appear to the called person.
  17. Functionality should work fine with diff OS/devices & Networks.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.