TFS Selenium C# Visual Studio 2017

Question

I have recently joined a company and they use TFS. I am an automated tester so I have setup MS Unit framework, selenium and c# within VS2017. I am comfortable in doing this standalone because its breaks away the testing of the UI Website testing, gives me more control and flexibility. I have also integrated these test scripts with a package reporting tool called TRX Viewer and it works fine (nice automated report). So with all this in mind I feel I have a decent setup/model so I continue writing test scripts and produce reports(html). Having never used TFS before my boss asked me can I incorporate what I have done into TFS. Having looked and read the posts it seems I can but can you please give me your opinions on the pros and cons of doing this?

Answer 1

First up, your boss is going to want to see this information in a convenient form. TFS allows this without much extra work on your part.

Pro

• You can use TFS (if it's a recent enough version) to set up scheduled runs for your test automation. Reports of your runs then become an automatic thing - and you can arrange for them to be emailed to whoever needs to see them via TFS notifications.
• TFS continuous integration means that you can modify and enhance your automation on your machine, check the changes in to TFS, and set up a CI pipeline that will build then run your automation regularly. The first time you do this will be painful, but after that, you should have no issues.
  • I do recommend using a different system for the build server than you use for the TFS server. I also suggest that you have a separate machine to run the automation.
  • Your automation run machine will need to be configured to never go to screensaver and to automatically log on with a set of credentials that never expire. You may need to arrange a group policy exemption with your IT department if you don't have 100% control of the run machine.
• You can use TFS to create an automation test plan and test suite and give it test cases. Once you use Visual Studio to link each test method to a test case in TFS, you will have automatic integration with TFS.
• Alternatively, you can use the TFS API to post your HTML report to a TFS location - precisely where you choose to post it is up to you. Or post it to your test suite as an attachment.
• Once configured the whole thing is pretty seamless and low maintenance.

Con

• The build system will need a full installation of Visual Studio with the correct licensing. If you don't have the correct licensing the error messaging is not terribly informative.
• Getting everything set up and configured can be painful and take a long time. It took me weeks of work and frustration to get a CI build and run pipeline in place the first time. Now I can do it in less than a day.
• Many of the tasks in the TFS build/deploy system are less than helpful about their default settings. I needed a lot of trial and error before I sorted it out.
• If you want to dump your log to the file system, expect awkward addressing. At least one of the file copy tasks does not recognize absolute paths and assumes that you will be starting from deep within the file system.
• Unless you're running headless, you will need to configure the run system to never go to screen saver and to automatically log on with a set of credentials that does not expire. This could cause problems with your IT people.
• The TFS API is not the most friendly I've seen. It can be... challenging to work out what to do with it.

Overall, I think you would be best served in the long run by adding TFS integration to your automation tests, not least because this is something your boss wants you to do. I think the biggest pro for you is that once you get the integration with version control (either TFS Git or TFS version control) set up and build the continuous integration and deployment pipeline, you will have scheduled or per-change runs of your test automation without any further effort on your part, which will save you effort in the long run.