Implementation of specialized testing component

Question

I have to design and implement tests for a project. In order to be able to re-use and automate the tests within several frameworks and tools (mostly for integration, functional and acceptance testing), I implemented specialized tests components, each used to perform actions and checks on specific functionalities.

For example, supposing an application working as follow:

1. Receive JMS messsage
2. Perform some business processing on these data
3. Write data on external system
4. Insert processed data in database
5. Finally, Write a file on an archive server

I'll have one or more 'helpers', configured according to the environment, performing test actions with functions like:

• feedMessage() : send the JMS message to be consumed by our application
• checkDataWritten() : check data are correctly written on our external system
• checkDataInserted() : check the processed data has been inserted in database
• checkFileArchived() : check the archive file has been created properly
• And other functions to prepare and finalize my test

My test is then usable with various tools and context (using TestNG, JMeter, JUnit, etc.) without rewriting logic twice.

Is it viable/maintainable on the long term to implement a specialized test component for each functionalities? Isn't my test code/framework a parallel application in the end, which should be tested as well?

Thanks.

Answer 1

Is it viable/maintainable on the long term to implement a specialized test component for each functionalities?

Great question. Automated tests work best when the implementation of the SUT (system under test) is subject to change but the interfaces that your test relies on remain the same. None of us can tell you whether you chose the right interfaces. If you remain on this project long enough, you will be able to answer that question for yourself.

Isn't it a parallel application in the end, which should be tested as well?

I assume by it you mean the system you are testing, not the test code itself. Yes, you may also want to test multiple operations taking place in parallel.

If your parallel test fails, the first thing you will ask yourself is, "Does the single-threaded test pass?" So even if you write a parallel test, you still want the simpler, single-threaded version.

If you are contemplating a multi-threaded test, think about your goals, because they may lead your toward a different test design. For example, in my experience a performance test tends to be implemented differently from a multi-threaded functional test. Another SQA question discusses this idea in more details.

Edit: OP says it meant the test code. Yes, once your test framework is complicated enough, you will want tests for it too. Some other SQA questions on that theme are this and this

Answer 2

Yes, it's a parallel application, but the thing that tests it is the code under test. Assuming you have a spec that you're writing for (Which you do, even if it's not written down), then the code under test is the implementation of that spec. The test framework is there to validate that the code under test is a correct implementation of the spec.

And if the two match up (i.e., test cases don't fail), then, at least for what you've written test cases for, you can be reasonably sure that the code is correct, and that the test framework is correct as well. If they don't match up, and you have test case failures, then either the test framework is correct, and there's a defect in the code under test, or the code under test is correct, and there's a defect in the test case. (Or they're both incorrect, which is possible).

Answer 3

Building a testing framework means coding and if the framework is a complex one for sure you will be required to spend sometime to maintain it. Try to build your framework so that there is no dependency between components, or at least the dependence is really limited, and the final result will be a reliable system.

I worked a lot on testing framework for several applications and I can clearly state that whenever you run a battery of tests and some tests fail you have to check both if the application fails or the testing fx has to be updated.