2

The WHY:

I am looking for options and possible full solutions if someone has already done something similar. I am testing web-based applications using C# Selenium with VS2015 by writing tests that automate the browser and make sure its doing what its supposed to be doing. I have written a whole test suite to test every inch of the website. It is tedious to manually run these tests during the day and then fix them as they come up, so I am looking for a solution that will help me automatically run this test suite every night at a specific time (like midnight) and receive a log/output file of it after that I can check in the morning to see what passed/failed.

Background Info:

  • Using C# Selenium in VS2015
  • Using TeamCity for CI
  • Want to run these tests on its own server (Mac, MS, Linux, doesn't matter, as long as it works)
  • Needs to run HEADLESS but emulate Firefox (no browser/display with Selenium - leaning toward Linux solution since its has Xvfb)
  • Output/Log required to review next day

Solutions I have looked into:

Currently since we are using TeamCity, I had an idea that we could make a separate server for this nightly testing. But what else is required? Is there an easier way? TeamCity has a scheduler so I can do the build at specific time at night every night, but how does one go about automatically making it run the specific test suite in the form of a VS project file?

Anything will be helpful, even if distantly related or complicated as long as I have some information I can go off of and try and do this on my own!

Thanks :)

2 Answers 2

1

I actually saw this being done by Microsoft during STAR West. They showed me that there is a Nuget package for Selenium web driver. IIRC, they were using PhantomJS for headless browsing.

I had an idea that we could make a separate server for this nightly testing. But what else is required?

That depends on your application and environment. I wouldn't do nightly tests. If your tests run at night because your test take so long, go back and fix that. Maybe split up into separate BVT and full regression suites. However, if you don't gate code going into that environment (i.e. you do not merge changes to staging yourself) then you might want to have a nightly test.

1
  • Just an update for anyone else who is looking for something similar. This does in fact work with the Nuget Selenium Driver (works with firefox, chrome, etc just need to set the browser in the app.config or in your test class) AND the PhantomJS for headless browsing, though on the prompt if there is an error, the error log is very hard to decipher (imo) and it seems to have a ton of random errors that wont stop the test, but it will clutter the log. The test run 100% fine with VS itself without the PhantomJS, but through PhantomJS it behaves weird. Still working on this, will update later.
    – David Ko
    Jan 15, 2016 at 19:26
1

If you do stick with TeamCity, I would probably recommend creating a separate build agent as opposed to a whole server.

You can find details of how to set up a test suite here

That build step can then be scheduled with various triggers

I would suggest you also have a look at TFS as an alternative to TeamCity (which does also offer IDE integration). As you would expect, being another Microsoft product the intergration with Visual studio is seamless. You are then able to schedule these to run regularly overnight.

2
  • i will give TFS a look. Is TeamCity integration with VS not smooth or compatible as TFS? Most likely I will stick with TeamCity for now, so I will check out the documentation you linked and see if I can get something going soon!
    – David Ko
    Nov 2, 2015 at 14:41
  • TeamCity does integrate (added to original answer), but nothing is quite as seamless as TFS. Same company making all components has its advantages!
    – ECiurleo
    Nov 3, 2015 at 9:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.