Should one test rely on the output of previous test?

Question

First of all I have read some great answers in the following link: Are your UI automated tests dependent or ordered?. But would like a clearer understanding in terms of dependencies within test data.

I'll be straight forward here and give an example, we have two tests A and B:

• A tests for if an object can have a unique attribute
• B test for if an object can have another unique attribute
• The object cannot have two attributes at the same time

Right now I am trying to decide on whether I should use two separate or one single object to test these cases.

If I were to use one, then test A (or test B) would have to make sure that it removes the attribute from the object after the test is finished, so that other tests can add other attributes to the same object. This seems like dangerous dependency to me because if test A fails to remove the attribute for some reason, it would have an impact on all following tests.

This can be resolved by using two separate objects, but then the problem is that this will likely require one object per test, which introduces a huge amount of objects that I might not be able to offer for my current test suits.

I would like to know what is the normally preferred approach in the software testing field, and the best practice I could implement given my situation. Thanks!

Answer 1

As I understand your question, the answer is to use different instances of your object, since they will have a number of common attributes but the attributes your two tests are checking are mutually exclusive.

For instance, in an application that manages student enrollment and teacher allocation of courses for a school, there might be a Person object. A Person may have the Student attribute or the Teacher attribute but not both.

In this example, TestA would create an instance of Person as part of the Arrange portion of the test, perform whatever test actions are needed, then Assert that the instance's Student attribute is set and its Teacher attribute is not set. The teardown would then destroy that instance.

TestB creates a second instance, performs its actions, then Asserts that the instance's Student attribute is not set and its Teacher attribute is set.

You can set this kind of framework up in multiple different ways (and combinations of these):

• Loading object instance data from a file or database where the data is flagged with which test it applies to
• Helper routines which create the instances based on the rules you give in your test code.
• Mocking or stubbing your test objects to minimize the amount of setup each test requires.

There are other ways as well - this is far from a comprehensive list. As a general rule unless you have no other choice you should be aiming to keep your tests as independent as possible, your test data as independent as possible, and your code as DRY as possible.

Answer 2

The most classic way is to test and assert them in 2 separate test (lets say at unit test level) and then have another test which test every possible combinations ("Integration test" though probably not the best name for it).

However, I think testing and asserting them only in 2 separate test cases and execute them in sequence while the assertion of the 2nd test depends on the result of the 1st one, makes sense too.

The best example which comes to my mind is testing a "Buffer". Lets assume we have a very simple buffering service (capacity=2) with the following methods:

 1. Read() // Return last item from Buffer
 2. Write(item) // Add "item" to the buffer
 3. Remove() // Remove the last item in buffer

You can have 1 test suit (class) which contains 4 different test case in it as follow:

• buffer = new Buffer() // static class variable shared among all of the test cases
• Test case 1: buffer.remove should fail with msg "Buffer is empty."
• Test case 2: buffer.write(A) should pass && Buffer.size()==1
• Test case 3: buffer.write(B) should pass && Buffer.size()==2
• Test case 4: buffer.write(C) should fail with msg "Buffer is full."

As you can see, each test case is testing 1 method, though the result of each of them are depending on the previous test case.
In Scala(test) you can share your test by different fixture objects. I am sure other testing frameworks support it too. Have a look at here.