Does cascading tests violate the "one assertion per Test" principle?

Question

Since I have been writing tests, I always stuck to the principle of one assertion per test. Now assume I have a simple application where I want to perform a system test with the following three test cases:

1. Test if a user can register a new account
2. Test if a user can log in
3. Test if a logged in user can add a post

I understand that in a unit test all three tests can be performed independently by mocking the other components. However, in a system test, 2. depends on 1., and 3. depends on 2.

I always pondered how to model such dependencies in tests while still maintaining the one assertion per test principle. Up until recently I simply would write three independent test cases, where each necessary step was repeated within the test code. However, this often led to test code duplication and in consequence to hard to maintain test code. So I began to cascade my tests like in this pseudo code example:

def test_register_user():
   new_user = app.register new user()
   assert(new_user in app.list_of_users())
   return new_user

def test_login():
   user = test_register_user()
   logged_in_user = app.log_in_user(user)
   assert(logged_in_user in app.list_of_logged_in_users())
   return logged_in_user

def test_write_post():
   app.write_post(text="text", user=test_login())
   assert("text" in app.list_posts())

So if test_write_post() is executed, it will call test_login() and test_register_user(). I like this approach because it does reduce test code duplication. However, I am not sure whether it violates the "one assertion per test" principle. On the one hand there is only one assertion that directly belongs to the tested method. On the other hand if you follow the execution path of the test case, several assertion will be checked.

My question is: Does the presented style of cascading tests violate the "one assertion per test" principle? If so, are there any better practices how one can deal with dependencies in system test while keeping the test code maintainable?

Answer 1

I prefer to test in isolation, test should not be depended of each other in order to be able to run them on their own. This makes testing a smaller part of the application easier for developers. Also if each test has its own environment to run in you can run tests in parallel without tests conflicting each others runs.

There will be some extra overhead in setup for each test, this makes a single test run of the suite slower, but since you can run tests in parallel it will scale easier in the long run.

Create a function for each duplicate behaviour in your tests. This is handy since if the signup changes you do not have to update multiple tests. Keep your code DRY. Also have a look at the Page Object Pattern.

Page Object is a Design Pattern which has become popular in test automation for enhancing test maintenance and reducing code duplication. A page object is an object-oriented class that serves as an interface to a page of your AUT. The tests then use the methods of this page object class whenever they need to interact with that page of the UI. The benefit is that if the UI changes for the page, the tests themselves don’t need to change, only the code within the page object needs to change. Subsequently all changes to support that new UI are located in one place.

I would setup your example something like this:

def setup_new_user():
    new_user = app.register new user()
    return new_user

def login(user):
    logged_in_user = app.log_in_user(user)
    return logged_in_user

def test_register_user():
    new_user = setup_new_user()
    assert(new_user in app.list_of_users())

def test_login():
    logged_in_user = login(setup_new_user())
    assert(logged_in_user in app.list_of_logged_in_users())

def test_write_post():
    logged_in_user = login(setup_new_user())
    app.write_post(text="text", user=test_login())
    assert("text" in app.list_posts())

Answer 2

Does the presented style of cascading tests violate the "one assertion per test" principle?

Yes, it does. While the style is quite clean, a disadvantage to your solution is that it becomes harder to run test_login and test_write_post with a different user, for example with one that is already registered. I personally find a better solution is to have cascading test suites which can define the order in which to run their tests.

def test_register_user(username):
   new_user = app.register new user(username)
   assert(new_user in app.list_of_users())

def test_login(username):
   logged_in_user = app.log_in_user(username)
   assert(logged_in_user in app.list_of_logged_in_users())

def test_write_post():
   app.write_post(text="text")
   assert("text" in app.list_posts())

def suite_new_user_writes_text
   return new suite()
           .add(test(test_register_user))
           .add(test(test_login,username))
           .add(test(test_write_post, username))

def suite_new_user_writes_text_and_logs_out
   return new suite()
           .add(suite(suite_new_user_writes_text))
           .add(test(logout,username))

def suite_new_user_writes_text_and_gets_banned_for_writing_profanities
   return new suite()
           .add(suite(suite_new_user_writes_text))
           .add(test(test_ban_user,username))

def run_tests
    suites = meta_search_for_functions_starting_with("suite_")
    foreach suite in suites test_and_log(suite)