2

I'm using Python+pytest and Helium to create my tests. Each time I run a test individually, it works 100% of times. However, if I try to run more than 5-6 tests together, some are failing, some are passing. The errors are random and never the same(from "LookUp" errors to "NoSuchWindowException" and other). I already checked the answers from similar questions asked:

  • no preconditions are altered
  • no dependencies between my tests
  • no alteration of data that is used in another test
  • i'm not using xdist, I am only trying to run them sequentially
  • I created different structures and arranged the tests in multiple ways trying to narrow down if a common element exists - nothing showed up. No matter what I include or exclude, just after I try to run 5+ tests, this behaviour is reproduced. I'm always starting the tests with start_chrome(URL) and end them with kill_browser(). My best guess is that somehow, in the background, the instances are overlapping or something similar to this, but I have no idea where to start the debug process.

I'm pretty new to this, any help is welcomed. I'm running my tests using python3 -m pytest. Here is my pip list output if it helps:

attrs        21.4.0
chromedriver 2.24.1
distlib      0.3.4
filelock     3.6.0
helium       3.0.8
iniconfig    1.1.1
packaging    21.3
pip          21.2.4
platformdirs 2.5.1
pluggy       1.0.0
py           1.11.0
pyparsing    3.0.7
pytest       7.1.1
selenium     3.141.0
setuptools   58.1.0
six          1.16.0
tomli        2.0.1
urllib3      1.26.9
virtualenv   20.13.4

Edit per comment: here are some of the errors:

- AttributeError: module 'locators' has no attribute 'checkbox_terms_and_conditions'
- AttributeError: module 'locators' has no attribute ‘set_date’
- action = <function APIImpl._perform_mouse_action.<locals>.<lambda> at 0x7f80dcba6ee0>
- 
-     def perform(self, action):
-       from helium import Config
-       end_time = time() + Config.implicit_wait_secs
-       # Try to perform `action` at least once:
-       result = self._perform_no_wait(action)
-       while result is None and time() < end_time:
-           result = self._perform_no_wait(action)
-       if result is not None:
-           return result
- >     raise LookupError()
- E    LookupError

Please note that each 2 runs have different errors, mostly LookUpError, NoSuchWindow,or AttributeError for locators file.

in my tests, I use an import statement for the file that contains my web elements locators, like ids, css class, xpath etc. each folder of tests has its own “locators” file. I’m not sure if this is the best method, any help is welcomed; thank you.

4
  • Please post the error message or the exception you are getting.
    – ChathuD
    Commented Apr 5, 2022 at 16:04
  • thank you, I added in the original post, since it was too big for the comment section.
    – Johnny
    Commented Apr 6, 2022 at 8:15
  • My first guess is that the setup/teardown between the tests is causing issues. As a stupid debug step, I'd try adding a sleep after the start_chrome().
    – ernie
    Commented Apr 6, 2022 at 17:44
  • Thank you. I tried with a time.sleep(5) after each start_chrome() but I get the same results. I tried decreasing to 1 second sleep and increasing to 8 seconds sleep, but it behaves the same. A lot of attribute errors for my locators file, like `AttributeError: module 'locators' has no attribute ‘set_date’ . But my file does contain that attribute, and the test runs OK if I run it alone. I am also running the test solo with headless=False and I can see that it does what it's supposed to do.
    – Johnny
    Commented Apr 7, 2022 at 10:35

1 Answer 1

0

Since you have shared that there are no lookup errors or no such window exception, I think it might be an issue related to waiting time. This is why when you are running an individual test, it works fine. However, when you must be running multiple tests together, no wait time between the tests restricts the elements in the script to load.

Let me know if you have added the wait time in-between the tests. And if adding wait time resolves your issues, I would like to hear it too.

Also, here I am sharing a link for a similar query that I think might help you solve the problem https://www.typemock.com/answers/9694/running-all-tests-fails-some-tests-that-pass-individually

1
  • thank you. As ernie suggested, I tried with multiple sleep()s, before and after each test. Unfortunately, it seems to work only for the first 4-5 tests, and after that pytest just ignores my wait times. For example, I added 10s after each test, and the first 4-5 tests are starting with a delay of 10s and PASS, after that the wait times are no longer taken into consideration. Which results in the same issues - no such window, lookup errors or moduleXhas no attribute Y. As I found on a similar issue, I also tried using a fixture for delaying my tests, but only works for my first 4-5 tests.
    – Johnny
    Commented Apr 11, 2022 at 13:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.