1
  1. I have tests which require user to login
  2. For such tests I register new user
  3. Such tests also have user_logout function (right before assert)

The problem:
If tests fails somewhere prior to calling user_logout and assertion, it breaks some of the following tests which require registration - the signup form by design is not available for logged in users, so the user registration flow fails because selenium can not find registration forms (no logout - kept user - session - no registration form).

I have two ideas how to prevent tests failing not because of assertion to affect the following tests:

  1. Add to the tests with login kind of flag 'requires_logout' and add some handler to the conftest.py with pytest_exception_interact
    • pros: solution looks solidly and impressively
    • cons: conftest will require import of UI tests handle lib to call user_logout there. Don't know why but I don't like it
  2. Wrap test itself in try-except until user_logout and assert and do an 'emergency' user_logout on any exception.
    • pros: seems easy and reliable
    • cons: doubts that it is proper usage of exceptions and tests framework; double user_logout call in test file (within try-except and 'regular' prior assert.
  3. Within registration flow check if we're logged in and logout but someday I'll write tests to check that registration is not available for logged users.

What is the best way to handle this situation?

  • 2
    You should start each your new test from a clean profile with no cookies so that server will not be able to detect which a concrete user you are. As far as I know recreating webdriver is enough to clean up the profile. – Alexey R. Dec 19 '17 at 12:01
  • 1
    Log out in test teardown. – Embedded Dec 19 '17 at 12:52
1

Each test should be independent from any other test.

If you want to use logged-in/logged out states, don't set login/logout from other tests. That is bad, and causes exactly the problem you have.

Use test fixtures if you want to set up some state before tests. So if you have bunch of tests that require you to be logged in, you would have test setup that logs you in, then tests run, then test teardown logs you out(or better - quits driver).

  • I prefer to not restart webdriver for each test because, as mentioned, there will be huge increase of time required for testing (as already was mentioned). – Vladimir Kolenov Dec 20 '17 at 2:32
  • That's why i mention fixtures. You would generally want some test setup to run once before all the tests which assume user is logged in, so you would use one webdriver instance per fixture. – George Dec 20 '17 at 12:53
  • Thanks, I went the way of registration/logout fixtures but at this point can not get how to pass webdriver instance existing in one fixture to another fixture. Because I: 1. Create fixture 'main' fixture where I did setup webdriver session and testrun settings 2. Create fixture 'user_logout' in conftest 3. Pass both fixtures to test_something() - now for 'user_logout' required instance of webdriver creted in the 'main' fixture. How to pass one fixture to another(???) – Vladimir Kolenov Dec 26 '17 at 9:01
  • nvm, got it - just pass fixture name as parameter to another fixture – Vladimir Kolenov Dec 26 '17 at 9:11
1

Your problem here is that your tests are not independent: the tests for user registration depend on the previous tests succeeding and the user in those tests having logged off.

There are different ways you can make your tests independent - the suggestions I'm giving are a few of the simpler ways.

  • Recreate the webdriver at the start of each test. This is the simplest method of making your tests independent. If you use a before-test method to do your instantiation and an after-test method to clean up any leftover activity you won't need to have a lot of copy-paste code.
    • Advantages - simple, easy to implement
    • Disadvantages - adds a significant cost in time and processing if there are a lot of tests. Recreating the webdriver also forces a new browser instance, which can make test runs unacceptably slow in a rapid development environment.
  • Kill the browser/session at the end of the test. This is a slight variation on the previous suggestion but doesn't require recreating the webdriver instance. Instead, use the before-test method as your starting point, and the after-test to either kill the session or close the browser. That will give you a clean session whether the test succeeded or failed.
  • Add a logged_on check to the after-test method. If you're running a lot of tests and need to keep the browser and webdriver in memory for performance reasons, this is your best option. The main disadvantage for you with this option is that you'd need to restructure your assertions since you're logging off prior to asserting. On the plus side, if you've structured your code properly and you're using a Page Object Model or similar, you won't need much more than something like this (pseudo-code):

    [after-test]
    method testCleanup()
    if currentPage.isLoggedOn() then currentPage.Logoff()

  • I would exclude the last item from the suggested ways since it is not reliable one. Logging out using user interface might fail due to a number of reasons so that it does not guarantee your next test will start from scratch. – Alexey R. Dec 19 '17 at 13:36
  • 1
    @AlexeyR. Yes, it could. That's why I listed it last - it's less certain, but is the least time-consuming method when there are a lot of tests to deal with. – Kate Paulk Dec 19 '17 at 16:33
  • So you guys think that try-except not a good decision also? – Vladimir Kolenov Dec 20 '17 at 2:33
  • @VladimirKolenov - I would put try-except in the before test and after test routines. I'd also suggest that if you are having this many problems with test failing to reach the asserts, you need to reconsider how you are building your tests - flaky tests are bad news all around. – Kate Paulk Dec 20 '17 at 12:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.