Given the following example MongoDB document in a collection person, which goes through different processing steps:
{
'status': 'Step 2 is finished'
'name': 'Some Name'
'affiliation': 'Gang of Four'
}
Old software release: Documents were created in process step 1 without the field affiliation. The missing field was added later in process step 2.
After step 1:
{
'status': 'Step 1 is finished'
'name': 'Some Name'
}
After step 2:
{
'status': 'Step 2 is finished'
'name': 'Some Name'
'affiliation': 'Gang of Four'
}
New software release: The entire document is created in step 1. Step 2 relies on the existence of the field affiliation and fails if it is not present.
The gap: A diff of the document schema between old and new release does not indicate a change, so no migration is built. When the new release is deployed, there are documents in the database which were produced by the old processing step 1, and are thus missing the affiliation field. Step 2 fails on these documents. The existing automated tests create test data dynamically though API calls, and so it is not possible to run them on 'old' data.
My ideas so far:
- Generic tests, which search for documents in 'Step 1 finished' status and run the 'step 2 code' on them. Given a larger number of possible status and transitions, the number of necessary test scenarios would soon grow.
- Base existing automated tests on static, raw database test data, which is versioned with the releases. This adds the burden of manual test data management which I would like to avoid.
What are good strategies to find these kind of gaps with automated tests, or to avoid them in advance?
