4

My current setup is using Jenkins as my CI environment, which has some ubuntu server nodes to execute the end to end browser tests on some amazon EC2 machines using selenium 2.

I am wondering if there exist limitations running tests using Xvfb, as oppose to using a real X server hooked up to a real display.

What are the pros and cons of doing headless browser testing on a Linux machine using X virtual frame buffer?

2

We are using Xvfb in our Continuous Integration environment and it is working great. It's no problem to take screenshots from Selenium for later analysis when errors are found. We have found no problems with this solution, rather a great advantage to be able to use virtual servers to spawn massively parallel tests against installations of the software.

1

It depends on what your tests are actually testing. "functionally" it should behave similarly, however alot of the variation is with browser/javascript/other component interactions. Headless would allow the functional test, but not guarantee the variations introduced with different platforms/browsers. When you add another third party like x virtual it goes even further from the raw browser OS testing.

For smoke tests it might work fine, but for true QA automation tests I would recommend duplicating as close to the final test environment as possible. I have seen false positives as well as missed bugs with variations which leads to a lack of trust in automated tests and simulated environments.

Usually cases for what you are suggesting and emulated mobile devices and such are for preliminary smoke tests on new functionality. Then a formal QA test run is done on the actual environment later. If that is the setup then the approach sounds just fine.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.