9

When doing QA automation, during testing it is generally recommended to simulate the environment and all the conditions of an actual real user of the application under test.

But, when running end-to-end in browser tests on a CI machine, it is tempting and relatively easy-to-setup to run tests with a virtual display (xvfb).

What kind of specific issues can be possible on virtual display but impossible on a real one, and vice versa? Is it safe to ignore the "real display" requirement when running selenium browser automation tests?

Note that I saw this thread, but I would like to focus on specific potential issues here, not the general difference.

Improve this question

2 Answers 2

Reset to default
4
+50

The only difference I have found in practical terms is colour replication. It's not Bad, just not exact (in my experience).

On the whole, it's not a problem, but if you are making a site meet AAA standard for accessibility, it can be irritating at the times.

Improve this answer
-1

The reality is that different browsers and devices use different rendering engines, security processes and javascript interpreters to produce the page.

This is one of the reasons that occasionally you encounter a 'only in firefox' bug or 'only happens in Android <=v4' bug, etc.

These bugs reveal how no one system can perfectly emulate all the different systems that exist, at least not within one run of a given test.

The general factors that I would take into account when using a simulator are:

  • what rendering engine is used
  • what javascript engine is used
  • what screen size is used
  • network speed of virtual vs real devices

Because of all these factors you should be careful on using any given emulator (such as xvfb) as it will represent one approach.

Additional advice:

  • I tend to only run my cases in one browser - currently chromedriver - as the rare issues that arise are not worth the cost to my company because of the slowdown and extra time and effort that it takes with very little and rare reward. Also issues that do arise in only one device/browser are often not actually caught by the browser, e.g. 'too small to tap on' isn't detected my my automated test. This decision depend on your company.
  • I generally use a headless browser (phantom) for most tests that do not require javascript. xcfb is a nice option here as although virtual, you can still take screenshots.
  • Google have just come out with a headless chrome browser. You have to download chrome v59 to see it
  • When it is critical to run functionality against different devices I used the specific drivers, e.g. firefox, chromedriver, etc.
Improve this answer
4
  • 1
    xvfb isn't an emulator, it's a virtual frame buffer. Basically a virtual monitor. As such, the rendering engines and underlying systems wouldn't be affected.
    – ECiurleo
    Commented Jul 13, 2017 at 9:17
  • 2
    Thanks @ECiurleo Seems like I don't understand it well. Totally ok and I'd like to learn more. So which actual browser or rendering engine gets used by xvfb and how does that work? I'll google and read up more myself. Cheers!
    – Michael Durrant
    Commented Jul 13, 2017 at 10:05
  • 2
    No worries, browser and rendering engine work as they normally would. The xvfb is simply a mechanism to display the contents to you as a person. Think of it like running selenium without your monitor plugged in and accessing the box via a remote desktop. xvfb offers similar for X based systems
    – ECiurleo
    Commented Jul 13, 2017 at 12:08
  • Thanks. I'm still confused as to what I change or install to actually use the technology from my existing rspec tests.
    – Michael Durrant
    Commented Jul 13, 2017 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.