4

My team of testers and me have a big collection of functional tests (written in python and run with pytest). The thing is that all the tests are run on a single testing server, an ubuntu VPS with selenium,webdriver,firefox,chrome,xvfb,pytest etc.

What I would like us to do is create many different VMs (combinations of OSs and browser versions) that would be launched/run based on a configuration file, and would run the automated scripts that we have on our github repo.

Example: I want to run these X scripts on a Windows 7 VM with IE8.

I am currently familiarizing myself with Vagrant as a possible solution. Would it be possible to create some base boxes with specific configurations and then "somehow" trigger them from an interface?

I am referring to Puppet and Chef in the title because I have read many articles that combine them with Vagrant. I have no experience with them.

It would be helpful if I could understand the architecture that could be implemented... and maybe the limitations. Essentially i would like to make a small and local implementation of SauceLabs/Browserstack.

|improve this question|||||
  • How do you visualise "trigger from an interface" working - Do you mean some sort of web interface, or would a simple command line operation be sufficient? Would this machine(s) be running on a local workstation or remote? – Toby Jackson Jan 3 '14 at 15:31
  • What kind of virtualization software are you planning to use (Hyper-V, VMWare etc.)? If you can use Hyper-V then you may want to have a look at Sherlock, which I wrote for the test lab at my work. – Petrik Jan 5 '14 at 23:19
  • @Toby Jackson A web interface would be nice but not really necessary.For starters a command line operation would be great.The transition is fairly easy. – Kostas Demiris Jan 7 '14 at 9:51
  • Petrik: I am very familiar with VirtualBox but I dont know if it is suitable for this kind of operations.I am examining the possibility to use a Hypervisor, but as I understand it needs a little beastly hardware to run on(am I wrong?) . I am also studying VMWare's ESXI 5.5 as an alternative. My laptop has Core2Duo P8800 @2.66Ghz with 4G RAM Win764bit. Would it be possible to create a proof of concept on it with a couple of VMs ? – Kostas Demiris Jan 7 '14 at 9:52
  • @wehappyfew I don't know if virtual box has a way to automatically start, stop and reset virtual machines. If it does then it would work because that's really all you need for automatic testing on a VM. You could probably run some VMs on your laptop as long as they're not super big and you probably couldn't run multiple at the same time, but it should be powerful enough for a proof of concept. – Petrik Jan 8 '14 at 19:42
1

My team has created some Gradle tasks that spin up VM's using vagrant and then deploy stuff to the VM's using Chef (similar to puppet). It would be possible to do something like this for your test infrastructure, but it is a lot of work not only to build the initial infrastructure, but also to maintain those VM's over time, especially with the number of configurations you're talking about. I would suggest an alternative.

Sauce Labs is a service that already exists to do what you want to do and is not too expensive. It allows you to execute your WebDriver tests on their grid and their site has a lot of great documentation, making it relatively easy to get set up. In addition, you can watch your tests executing in real time, view videos of the tests after the fact, and get screenshots, all easily configurable. Their logging/reporting infrastructure is also pretty mature, you can easily use it out of the box to cover most of your reporting needs. If you haven't investigated it, I would highly recommend doing so.

|improve this answer|||||
1

After a lot of research,coding,trial and errors, I managed to build the infrastructure I envisioned. Essentially I wanted to build a small (very specific to my team's needs) scale of SauceLabs/Browserstack.

@Sam Woods , Sauce is a great service but overpriced for our needs. Also,Browserstack provides features that we don't really need in it's light plans. We are running all the tests every day.

What we really needed was to make sure that everything runs smoothly on IE8 at Win7 . I managed to set a Selenium Grid with two Nodes. The hub is a Xubuntu VM, and the nodes are a Win7 VM(with IE8) and Mavericks VM(with Safari). The VMs are built with VBox and started/killed via Python commands. I tried managing them with Vagrant but without success. All that were first built on my laptop (for proof of concept) but eventually we will buy a dedicated (with a lot of resources) machine .

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.