Applying non-MS skills in TFS environment

Question

I have to automate functional testing of an application with GUI, written in some .net.

My background experience in GUI apps testing has nothing about Visual Studio and other MS systems -- I used scripting in Ruby/Python with OS-specific accesibility APIs, such things as Jenkins hosted on POSIX servers and REST APIs to connect all these and other things together.
But here I have some Microsoft thing called TFS, which is already being used for building binaries and even manual testers do keep here test cases. And I suspect that I have to somehow record UI activity instead of full scripting, like I used to do previously.

So what is a possible application of my previous experience here, or do I have to forget about deploying new servers and inventing own ways to view and aggregate results and just fully dive into TFS, Visual Studio, learn C#, etc.? Which resources can help me to look around here?

Answer 1

The Microsoft stack is surprisingly still very popular, regardless of other cheaper alternatives.

TFS is basically the central repository. Call it git and Jenkins rolled into one. It manages version control as well as builds.

MTM is just Microsoft's test management tool. This is very useful for linking a manual test that has been created within MTM (its intended purpose) with an automated test that is part of a build on the TFS server. You can then specify test agents (basically jenkins slaves with a microsoft twist) to run your automated tests.

Automated Testing You basically have three main options for automated testing within the Microsoft domain with TFS: CodedUI, Selenium C#, and Telerik Test Studio (Art of Test).

CodedUI is Microsoft's integrated automation tool. Out of all of the three main options, it is undoubtedly the most expensive. That being said, it does VERY well at automating thick client applications that are not based on a web server, which the other two tools do not manage very well. Multi-browser support is hard to make work with anything other than Internet Explorer.

Selenium C# A free open source tool used for automating web applications through the use of WebDriver (Selenium 2.0). There are multiple language bindings for Selenium, which is why I specified C# -- but if you are using TFS, you will either be stuck with C# or Python (Python only if you take the time to set up TFS and Visual Studio to build Python projects...you may be able to use IronPython out of the box -- not sure). It is most definitely the most popular of the three main options within the Microsoft stack, due mostly to cost, but also due in part to the fact that Selenium has a very large online community with literally entire libraries of information. Built-in mutli-browser support for all common browsers, including Firefox, Chrome, IE, Safari, and even PhantomJS.

Telerik Test Studio A low-cost automation tool. Really, the only good reason to use this tool is if your application is built with Silverlight, as it is the one tool out of the three main options that handles this the best. It also boasts multi-browser support. There is also a free version of the tool, but there is very little documentation around it, causing most issues to turn into headaches very quickly unless you are okay with talking to the Telerik support team.

Shameless Plug I'm going to go ahead and put in a shameless plug for another free tool as well, which you may find to be useful as you start out automating within the .NET world: Sikuli4Net. Basically, it is a .NET implementation of the popular Java/Python image-based open source automation tool called Sikuli (www.sikuli.org).

Hope this helps you make your decision!

Answer 2

Here's the short rundown on TFS and the Microsoft application life cycle management ecosystem:

TFS - Team Foundation Server is the "glue" that pulls everything together in a central repository. TFS can be used to manage builds (using MSBuild.exe), issues, test cases, automation, source control, and so forth.

Visual Studio - As well as the code IDE, Visual Studio includes near-seamless integration with TFS allowing developers to manage source control, build definitions, issues, and so on. Testers can also use VS Premium and higher to create the MS flavor of GUI-based automated test (CodedUI). These are true automated tests, usually written in C# or VB, and compiled.

Microsoft Test Manager (MTM) - MTM has two main functions: the management and manual execution of test cases (along with the ability to record exploratory test sessions); and the management of automation environments (including execution of automated tests). MTM is included with the Premium and Ultimate VS licenses.

Some of the better references are:

• Matthew Aniyan's Index of CodedUI references (https://docs.microsoft.com/en-us/archive/blogs/mathew_aniyan/content-index-for-coded-ui-test)
• Raj Kamal's overview (which has a nice diagram showing how all the pieces fit together in the ecosystem) (http://msdn.microsoft.com/en-us/magazine/hh875174.aspx)
• Execute Automation has a decent set of "how to" posts in their VS testing category (http://executeautomation.com/blog/category/visualstudio/)

Your experience won't be wasted by you moving into the MS ecosystem - knowing how to code makes learning a new language reasonably easy, and if you work in the same language the programming team is using, you can ask them for assistance (or code reviews...), and you broaden your experience base.

You can do simple record-playback, you can do "straight" CodedUI, or you can extend the CodedUI platform in a number of ways. There are also extensions such as the CodeFirst API library on CodePlex (http://codeduicodefirst.codeplex.com/) that can reduce the dependence on CodedUI's object mapping.

(Disclaimer: I do not work for Microsoft. I work in an organization that uses the MS ecosystem for development and testing).

Answer 3

The Microsoft test tooling environment is quite complete and mature, so I'd advice you to dive into that first - at the very least to get an idea about what is and isn't possible. It includes record-and-playback functionality, but that's not the only option and depending on the project and your testing strategy you might need more coding/scripting (or on the other end of the spectrum, more manual actions).

Your previous test automation experience is still very useful, since you already now how to set up and maintain an automated test suite - the underlying principles remain the same, regardless of the specific tools you use.

Answer 4

The other answers cover some specifics, but let me just say this.

1. You need to understand and operate in a way that "best" suites your current company culture.
2. MS is very proprietary and therefore if you are doing stuff that requires TFS and it's more than just a source control then you probably will have to dig in and learn it to be effective at your job. Kate already gave a good summary on spinning up on MS stuff. Also might want to read here:Are there any detailed tutorials for TFS, Visual Studio and MTM?

On the other hand:

1. TFS is not the best tool for all jobs even though MS sells it as the ultimate to do everything. That being said...once you have item 1 above answered you can then figure out if you can utilize any non-MS stuff to script tests with. If you use Selenium any library works and not just .net (although java and .net seem to be more feature rich first before other languages - depends on who develops first in community)
2. Being MS and integrating everything they own together they do develop pretty robust API's for most of their stuff. Even if you do have to utilize their stuff you should be able to sync up any custom stuff into TFS through code.