Testing Server/Workstation Hardware specs recommendations?

Question

I wanted to reach out and get some opinions of Testing hardware. Should I use VMs or a stand-alone tower? If I use VMs, what would be some recommended hardware specs for the hosting server? If a standalone physical workstation, what would you recommend in regards to processor power, ram, and gpu-wise?

Right now I have about 242 Selenium WebDriver tests spread across 11 test suites for which I run in parallel with each other. I am going to be adding at least another 360 tests in the near-future. My goal is to be able to run all the test suites simulatenously while not sacrificing the performance of each test.

Also let's say that my budget is $8,000.

Thank you all for your time and any advice you may give.

Regards, Travis

Answer 1

Travis,

I'm coming into this a bit late, but here's some thoughts for you:

• What is the total run-time of all your tests? For instance, if you can schedule your suites to run in sequence overnight and have all your results waiting for you the next morning, you may only need one system to manage testing.
• What levels of IO do your tests require? Tests with a lot of database or other external file reads are much heavier on IO than something that simply runs through a series of actions and assertions, although any automated test will have higher IO requirements than normal running due to the need to read in the test as well as the IO of the application in test.
  • What control do you have over your testing network? If you're able to add systems (physical or virtual) to a testing subdomain without needing to work with your employer's IT department, you have a lot more flexibility to store, spin up and rename virtual systems at need.
  • In any architecture, there will be choke points. The biggest one in virtualization is the server's IO capacity, at least when hosting automation VMs. I've seen high capacity servers overwhelmed by IO requirements because there wasn't enough disk IO throughput available.

Hopefully that will help clarify your thoughts.

Answer 2

A couple of workstations and VMs on each of them!!! Yes, I/O will be a problem but mostly if you install a SQL database on every VM.

You may eventually want to host a staging version of your website / service on some of the VMs and use others to run tests or to run load tests.

Commodity hardware is much cheaper than high-end servers (that's one of the reasons all the cloud providers keep their costs low). So a couple of workstations will sum up to much more than a single server.

GPU should have no impact, unless you plan to crack hashes or play 3D games while your tests are running; anyway I doubt you'll find a decent GPU in any server (unless you go for some fancy designer's workstation)

We use Dell Precision workstations, $8,000 will net you 3-4 of them, depending on how many accessories like monitors you want. That's a decent test lab :)