How should I handle tests failing prior to a logout and causing other tests to fail?

Question

1. I have tests which require user to login
2. For such tests I register new user
3. Such tests also have user_logout function (right before assert)

The problem:
If tests fails somewhere prior to calling user_logout and assertion, it breaks some of the following tests which require registration - the signup form by design is not available for logged in users, so the user registration flow fails because selenium can not find registration forms (no logout - kept user - session - no registration form).

I have two ideas how to prevent tests failing not because of assertion to affect the following tests:

1. Add to the tests with login kind of flag 'requires_logout' and add some handler to the conftest.py with pytest_exception_interact
   
   • pros: solution looks solidly and impressively
     
   • cons: conftest will require import of UI tests handle lib to call user_logout there. Don't know why but I don't like it
     
2. Wrap test itself in try-except until user_logout and assert and do an 'emergency' user_logout on any exception.
   
   • pros: seems easy and reliable
     
   • cons: doubts that it is proper usage of exceptions and tests framework; double user_logout call in test file (within try-except and 'regular' prior assert.
     
3. Within registration flow check if we're logged in and logout but someday I'll write tests to check that registration is not available for logged users.
   

What is the best way to handle this situation?

Answer 1

Each test should be independent from any other test.

If you want to use logged-in/logged out states, don't set login/logout from other tests. That is bad, and causes exactly the problem you have.

Use test fixtures if you want to set up some state before tests. So if you have bunch of tests that require you to be logged in, you would have test setup that logs you in, then tests run, then test teardown logs you out(or better - quits driver).

Answer 2

Your problem here is that your tests are not independent: the tests for user registration depend on the previous tests succeeding and the user in those tests having logged off.

There are different ways you can make your tests independent - the suggestions I'm giving are a few of the simpler ways.

• Recreate the webdriver at the start of each test. This is the simplest method of making your tests independent. If you use a before-test method to do your instantiation and an after-test method to clean up any leftover activity you won't need to have a lot of copy-paste code.
  • Advantages - simple, easy to implement
  • Disadvantages - adds a significant cost in time and processing if there are a lot of tests. Recreating the webdriver also forces a new browser instance, which can make test runs unacceptably slow in a rapid development environment.
• Kill the browser/session at the end of the test. This is a slight variation on the previous suggestion but doesn't require recreating the webdriver instance. Instead, use the before-test method as your starting point, and the after-test to either kill the session or close the browser. That will give you a clean session whether the test succeeded or failed.

• Add a logged_on check to the after-test method. If you're running a lot of tests and need to keep the browser and webdriver in memory for performance reasons, this is your best option. The main disadvantage for you with this option is that you'd need to restructure your assertions since you're logging off prior to asserting. On the plus side, if you've structured your code properly and you're using a Page Object Model or similar, you won't need much more than something like this (pseudo-code):

  [after-test]
method testCleanup()
if currentPage.isLoggedOn() then currentPage.Logoff()