Where should I create a coded UI project?

Question

Should Coded UI tests be within the solution that they are testing (similar to Unit Tests), or should they be contained in a separate, generic Coded UI solution that is built only once and executed whenever any solution needs to be tested? I should clarify that our Coded UI tests are not the regular recordings that can be created through Visual Studio, but rather hand-coded interaction classes developed by our QA engineers. Most of the common functions for interacting with elements on a page are packaged into a DLL that can be referenced in any test.

I'm basically just trying to find out what the best practice is for creating a Coded UI testing infrastructure. Any help would be greatly appreciated. Thanks!

Answer 1

If you have to support multiple versions for your software product I would keep the tests with in the same repository. Then the tests will version with the application itself. Which will make you able to run the tests for older versions with ease.

Also you will want your developers to run and maintain the tests. For example when they need to change the ID or the labels of a button. They should not need to checkout multiple repositories to verify their code changes did not break any old functionality.

The only reason I can come up with to not have the tests included in the main codebase is if the included test-data is so huge that a clean checkout of the code would take ages, but I would then put the test-data in a different repo by itself.

Answer 2

Like any other form of automated test, it depends on how you're invoking them and the environment you need to run. If you need a UI user session (which is the common setup) then you're probably not going to be packaging them with the unit tests.

Some of the factors to consider are:

• CodedUI tests are slower than unit tests because they interact with the application GUI. This is one reason to keep them separate from the unit test code.
• CodedUI tests can involve a much larger code-base due to the use of UI Maps or page maps (the part of the CodedUI project that stores the properties you need to reference specific components).
• They may also require a number of proprietary Visual Studio DLLs be present on the test/build system in order to compile (usually the DLLs used by the MS testing engine). Running the tests through Microsoft's test lab ecosystem doesn't require extra VS licenses. Running them elsewhere might - and the license required is VS Premium or Ultimate.
• Typically setup and teardown for an application-level automated test is more complex than for a unit test. It also usually takes longer. Depending on your application, you may need to perform long series of dependent action in order to complete a given test.
• You may also need to store substantial repositories of test data. This can slow down the entire checkout process.
• Whether you keep the coded UI code with the unit test code is not the same issue as whether you execute the coded UI code with the unit test code - ultimately the "best practice" is what works best for your environment.

Answer 3

I've never done that, but, from my understanding, they are tests so they should go to where the unit tests are.

Now, this might depend also how the tests can be run. My UI tests need to run on a PC/VM with a working UI user session, so there's no need/use in putting it together with the unit tests because these run on a CI server with no UI session initiated.