Looking for a HEADLESS C# Selenium automatic nightly regression test solution

Question

The WHY:

I am looking for options and possible full solutions if someone has already done something similar. I am testing web-based applications using C# Selenium with VS2015 by writing tests that automate the browser and make sure its doing what its supposed to be doing. I have written a whole test suite to test every inch of the website. It is tedious to manually run these tests during the day and then fix them as they come up, so I am looking for a solution that will help me automatically run this test suite every night at a specific time (like midnight) and receive a log/output file of it after that I can check in the morning to see what passed/failed.

Background Info:

• Using C# Selenium in VS2015
• Using TeamCity for CI
• Want to run these tests on its own server (Mac, MS, Linux, doesn't matter, as long as it works)
• Needs to run HEADLESS but emulate Firefox (no browser/display with Selenium - leaning toward Linux solution since its has Xvfb)
• Output/Log required to review next day

Solutions I have looked into:

Currently since we are using TeamCity, I had an idea that we could make a separate server for this nightly testing. But what else is required? Is there an easier way? TeamCity has a scheduler so I can do the build at specific time at night every night, but how does one go about automatically making it run the specific test suite in the form of a VS project file?

Anything will be helpful, even if distantly related or complicated as long as I have some information I can go off of and try and do this on my own!

Thanks :)

Answer 1

I actually saw this being done by Microsoft during STAR West. They showed me that there is a Nuget package for Selenium web driver. IIRC, they were using PhantomJS for headless browsing.

I had an idea that we could make a separate server for this nightly testing. But what else is required?

That depends on your application and environment. I wouldn't do nightly tests. If your tests run at night because your test take so long, go back and fix that. Maybe split up into separate BVT and full regression suites. However, if you don't gate code going into that environment (i.e. you do not merge changes to staging yourself) then you might want to have a nightly test.

Answer 2

If you do stick with TeamCity, I would probably recommend creating a separate build agent as opposed to a whole server.

You can find details of how to set up a test suite here

That build step can then be scheduled with various triggers

I would suggest you also have a look at TFS as an alternative to TeamCity (which does also offer IDE integration). As you would expect, being another Microsoft product the intergration with Visual studio is seamless. You are then able to schedule these to run regularly overnight.