Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Should Coded UI tests be within the solution that they are testing (similar to Unit Tests), or should they be contained in a separate, generic Coded UI solution that is built only once and executed whenever any solution needs to be tested? I should clarify that our Coded UI tests are not the regular recordings that can be created through Visual Studio, but rather hand-coded interaction classes developed by our QA engineers. Most of the common functions for interacting with elements on a page are packaged into a DLL that can be referenced in any test.

I'm basically just trying to find out what the best practice is for creating a Coded UI testing infrastructure. Any help would be greatly appreciated. Thanks!

share|improve this question

3 Answers 3

up vote 3 down vote accepted

If you have to support multiple versions for your software product I would keep the tests with in the same repository. Then the tests will version with the application itself. Which will make you able to run the tests for older versions with ease.

Also you will want your developers to run and maintain the tests. For example when they need to change the ID or the labels of a button. They should not need to checkout multiple repositories to verify their code changes did not break any old functionality.

The only reason I can come up with to not have the tests included in the main codebase is if the included test-data is so huge that a clean checkout of the code would take ages, but I would then put the test-data in a different repo by itself.

share|improve this answer
    
This approach seems the best suited for our current structure. I definitely don't want our developers to have to jump around into different solutions just to update the id or classes of an element. As far as our test data goes, I was planning on implementing a data-driven approach to this project and keeping most of our test data in a separate database...with the addition of keeping some local test data if the connection to that database goes down. –  reece Mar 20 at 18:46

Like any other form of automated test, it depends on how you're invoking them and the environment you need to run. If you need a UI user session (which is the common setup) then you're probably not going to be packaging them with the unit tests.

Some of the factors to consider are:

  • CodedUI tests are slower than unit tests because they interact with the application GUI. This is one reason to keep them separate from the unit test code.
  • CodedUI tests can involve a much larger code-base due to the use of UI Maps or page maps (the part of the CodedUI project that stores the properties you need to reference specific components).
  • They may also require a number of proprietary Visual Studio DLLs be present on the test/build system in order to compile (usually the DLLs used by the MS testing engine). Running the tests through Microsoft's test lab ecosystem doesn't require extra VS licenses. Running them elsewhere might - and the license required is VS Premium or Ultimate.
  • Typically setup and teardown for an application-level automated test is more complex than for a unit test. It also usually takes longer. Depending on your application, you may need to perform long series of dependent action in order to complete a given test.
  • You may also need to store substantial repositories of test data. This can slow down the entire checkout process.
  • Whether you keep the coded UI code with the unit test code is not the same issue as whether you execute the coded UI code with the unit test code - ultimately the "best practice" is what works best for your environment.
share|improve this answer

I've never done that, but, from my understanding, they are tests so they should go to where the unit tests are.

Now, this might depend also how the tests can be run. My UI tests need to run on a PC/VM with a working UI user session, so there's no need/use in putting it together with the unit tests because these run on a CI server with no UI session initiated.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.