Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I'm working in a primarily Windows / .NET system. I've written a test framework in C# that includes a test harness that runs fixtures (anything that implements the ITestFixture interface). These fixtures are the interface between my framework and the SUT (System Under Test), and sometimes they have dependencies on SUT methods or objects. Nothing else in my framework depends on the code of the SUT. I am testing multiple projects, which have relatively little to do with each other, and will be testing more and more over time.

Right now, I have a test solution in .NET that has all of my test framework code . . . and all of the fixtures, and all of their dependencies. The result is a giant test framework solution with a ton of dependencies. This doesn't seem like a very good way to do things in the long-term.

Another alternative that we are discussing is having me create a "Common.Test" project that essentially includes the interface for the fixtures, and maybe a few utility functions. Developers would then include a "SUT.Test" project (where SUT is whatever the system under test is called) that includes their fixtures and depends on the Common.Test project. Fixtures would be compiled as part of their solutions, and the test harness would load their fixtures from their .dll files dynamically.

Do either of these options (fixtures in the test solution with tons of dependencies, or fixtures in the SUT code that are dynamically loaded by the harness) sound better than the other to experienced automation testers? Are there other options that I'm missing that should be considered? If you use fixtures, what does your team do?

share|improve this question
    
Changed the accepted answer to match what I actually did in the end . . apologies if this has led to any confusion –  Ethel Evans Jul 2 '12 at 18:28
add comment

2 Answers

up vote 3 down vote accepted

Personally I use a combination of approaches.

I have all my tests inherit from a base test class, and it has a lot of the common objects and contains any setup and teardown code.

I also have an “observer” class that all the layers of my stack can reference and that contains any utility classes that are necessary for execution. The observer also serves as an in-memory data store for holding test data objects instead of having to pass them up and down between my layers.

This approach allows me to enforce separation of concerns where my tests no nothing of how the actual application is implemented, not the underlying automation engine that actually drives the application.

If you want to have a look it is open sourced at testingstax.codeplex.com

if I had to call external dll's and I want to be loosley coupled I would look to use .net reflection to load them dynamically on the fly. That way I can have my cake and eat it too :-)

share|improve this answer
    
So far, what we are doing sounds similar - except it looks like all your tests can be executed without knowing anything about the SUT's actual code (e.g., by going through the browser). Am I right? What about testing a .dll that was going to be part of a larger system? Would you include every .dll you are testing in your test solution, or put your actual test code into the .dll solution? or would you create multiple solutions for collections of related .dlls that you are testing that include the .dll and your tests, and have your harness dynamically load their compiled code at runtime? –  Ethel Evans Jun 23 '11 at 18:06
    
And, thanks much for including your source code –  Ethel Evans Jun 23 '11 at 18:11
    
Ok if I had to call external dll's and I want to be loosley coupled I would look to use .net reflection to load them dynamically on the fly. That way I can have my cake and eat it too :-) –  Bruce McLeod Jun 23 '11 at 22:17
    
Thanks, that's what I was looking for. I think I'm trending in that direction right now. The dev has some code snippets already available that I can borrow (I'm pretty lousy at Reflection). Can you edit your answer to include your comment? I'd like to have it be a bit more visible for anyone else looking at this question in the future :) –  Ethel Evans Jun 23 '11 at 23:12
    
Went over this with the developer, and it sounds like in order to use the typed objects, etc., we'd need to have an interface we could include to get code to compile :( Instead I'll provide an interface for inclusion in SUT test projects. The SUT test project will include the only the fixture interface and fixture implementations. Then I'll use Reflection in my test harness code to load the implementations of the interface and execute them. I have access to their solutions in source control, so can set up the test projects myself. Getting interfaces from devs, not so easy. –  Ethel Evans Jun 23 '11 at 23:24
add comment

In my company the test fixtures reside in a seperate folder called component_test_tools and it is created and maintained by the DEV. Our test case call these fixtures. When the application code/object changes and the DEV forgets to update the respective fixture, all the test cases calling those fixtures fail. The DEV needs to ensure that whenever he changes the code and objects, the fixture is also updated.

These dependencies caused some invalid failures in the test run. The testers depend on DEV to fix the fixtures which was a real issue that I faced. The DEV considered automation fixture bugs to be of low priority compared to application code bugs and it was challenging to get them attend to these issues.

share|improve this answer
    
Yeah, I strongly suspect that I will be maintaining fixtures partly to avoid this sort of dependency on the developers. So, it sounds like the fixtures in your system dynamically load the production code, but are all in their own solution? Thanks, this is helpful. –  Ethel Evans Jun 23 '11 at 18:13
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.