Use tested class during test setup and assertion? (Errors cancel out each other...)

Question

This is a general question that affects lots of my unit tests as I just recognized:

Can I use the class I'm unit testing to perform setups in my test cases?

Here's a simplified version of the example that's bothering me:

I'm implementing an abstraction layer over my database to simplify accessing it. I have

• an int create(String name) to create a database entry
• and a String read(int id) to retrieve it afterwards.

Those methods could fire SQL statements against a DBMS or use an ORM framework. Maybe they run on a different thread etc.

Of course, I want to unit test my abstraction layer. Let's call it database service, or DBS for short.

@Test
public void testCreate() {
    // GIVEN an empty database

    // WHEN creating an entry
    int id = dbs.create("John");

    // THEN the entry should be in the database
    assertEquals("John", dbs.read(id));
}

@Test
public void testRead() {
    // GIVEN a database with one entry
    int id = dbs.create("Julia");

    // WHEN searching for that entry
    String name = dbs.read(id);

    // THEN the found entry should match the entry in the database
    assertEquals("Julia", name);
}

The problem is that the first case tests create() and relies on the correctness of read() while the second case tests read() and assumes that create() works correctly.

Then, errors in both implementations can cancel out each other and both tests pass like with the following implementation:

private String cache;

int create(String name) {
    cache = name;
    // never write to database
    return (int)(Math.random() * 1000);
}

String read(int id) {
    // never read from database
    return cache;
}

So, my idea was to make one of the tests "atomic" by removing the usage of the tested classes' methods. For example, by replacing the call to create() in the second test case with a direct SQL query.

Then, the first test case would still succeed. But the second one fails. And after fixing the second one, the first one will fail, therewith, revealing the erroneous implementation of create().

The problem with that approach is that the "trusting" test cases rely on the existence of appropriate "atomic" test cases. That violates the principle of independent test cases.

The logical consequence seems to be removing all usages of my class from my test cases (except for the method under test of course). But then I would have to implement all of my methods (in this case both create() and read() to test my actual implementation that might be the same in the end...

Is that the way to go? Or am I too serious about this as erronous implementations that cancel out each other won't happen too often anyway?

Answer 1

You can but it's not a good idea

You've identified one of the big issues: if the implementation of the database access layer is faulty in the right way, your tests will never detect a problem.

You've also partially identified the solution: for your first test, you query the database directly to verify that the entry was created.

For the second test, the answer is to either create the database and target entry via SQL, or to use a known database with your target entry.

You would also need to unit test your implementation of the database access layer. They may or may not be very similar to your abstraction layer unit tests, but this is not a big issue.

The real benefit would come if/when it becomes necessary to rewrite or refactor your database access layer, or add support for a different database. If your abstraction level unit tests fail, you know you need to investigate your database access layer and see whether you need to change the unit tests (possibly necessary for example if changing database engines) or there's an error in the database access layer.

The highly simplified example really doesn't show the need for this, but if you're abstracting away complex database operations the value of keeping your tests atomic becomes more obvious.