This idea was presented to me recently:
You have a complex legacy ETL system in Java in a TEST environment that no one understands completely. You write some tests that run it and compare the output with your expected outputs hardcoded into your tests.
But you also run your tests against a new rewrite of the system in a DEV environment and you compare the output of the legacy system in TEST with the output of the new system in DEV. If the rewrite was successful, the outputs should match.
So the question is, is this 2nd technique a good idea long-term? Short-term, it's convenient because you don't have to hardcode all your application's logic into your tests. You kindof skip understanding the business logic and you get a black-box comparison for free. But long-term, what's in DEV will be migrated to TEST and what's in TEST will be gone. New development will be in DEV. In the future running the suite will just compare this release's development with last release's development.
Someone proposed to me that this was still useful, but I'm not so sure. Something doesn't seem right to me. Some problems I see are:
- You have to keep both TEST and DEV pointed at the same input data all the time.
- You are losing the value of readable test specifications of behavior.
- As soon as the legacy system is gone, you lose a lot of implicit behavior comparison from the old "source of truth" legacy system. You're now just comparing mostly against yourself.
Has anyone heard of examples of this or articles on it? I'd be interested if it has a name or hearing about the pros and cons. I couldn't find anything on Martin Fowler's site.