Should I open the same page in every test?

Question

I'm new to automated testing and asking for your help.

As I know every test should have its own steps and if I define several tests in one class, should I open the same page several times in each test or I can open it one time somewhere (in first test maybe) and then it will be kept open all time.

Which approach is more correct?

Answer 1

For manual testing it would be smartest to re-use the page (or test) state, because it will be way faster. It is possible to argue the same for automated tests, but if you do not run tests in isolation from each other there is a high risk that you will loose confidence in your test suite. This because failing tests have possible side-effects on each other that cost a lot of time to investigate if you can reproduce it at all. Certainly when one test fails there is a high chance multiple will fail as they are depended on the state of the page. Which test will you research first? How do you know you have the correct one?

Automated end-to-end tests are there to test the product integration and that common workflows still work. It is not a replacement for duration tests or how feature work together over time. Create different tests for this.

I would advice to not re-use pages or state if possible. I have had a situation where workflows where very time consuming. In the end I created a lot of small tests which at the end of the test would save their state to a database in order for it to be quickly re-used by depended tests. This way we could run each test in isolation, but still re-use the application state if the test was successful, speeding up the whole regressions suite greatly. We documented all the tests dependencies in code so it was pretty clear how to research fails.

Maybe now test execution time is not an issue, but it will be as your product grows and market cadence expects faster releases. Being able to run your tests in parallel is a must. To reduce test execution time now and in the future it is better to start with writing tests that can be run in parallel from the start. If you do not do that it is so much harder to scale in the future. State-reuse therefor is an anti-pattern, because it will limit running tests in parallel in a later stage.

In the end it depends. Preferable you run tests in parallel instead of re-using state to make them faster. Sometimes this is hard, but do try to find a way that tests do not influence each other to minimise research into failing tests.

Answer 2

re-using a browser between tests is generally an anti-pattern and should be avoided. Tests should always start from a known state so the results are deterministic. You also want isolation between tests. Whatever you do in test A should not affect what happens in Test B.

Answer 3

Which approach is more correct?

Both approaches are ok, but the best one is the one that the typical end user would follow.

So if the user opens the page only once, and uses all the features, this would be the best way to test.

Answer 4

The short answer - Probably not.

The long answer - There are some principles of test automation that mean you should not treat automated tests the way you would treat manual tests. It's fine, and common, to keep the same page open and run all the manual tests needed against it. Doing this with automation is dangerous for these reasons:

• Computers are very fast idiots - they do precisely what they are told to do and nothing else. This means that if something is wrong after Test 1 and it doesn't stop the automated run, your results from Test 2 to the end of the run are compromised. You don't want that.
• Parallel running is good - you should be able to have your tests run completely independently of each other so that you can break them up and run them in parallel. Testing through the browser is the slowest form of automation, so you don't want to tie up machines for a long time. At the same time, independent tests mean that the results of one test don't affect the results of a later test run on the same system.
• Each test does one thing - Use something like the page object pattern to do the heavy lifting of managing the page functions. That way you have one routine to perform any action, and your tests just call that routine. Building your automation this way means you can have a pre-test setup that starts the browser, logs in, and navigates you to the correct page. Your test verifies that your target action works as expected. Then your post-test teardown logs off and closes the browser.
• Early failures can hide later issues - a problem with an early test in a run of dependent tests can hide problems with the later test. If each test depends on the one before it completing successfully, your last test may not be run because of failures. You can't tell if it has a problem unless it can run.

That's just a starting set of reasons not to re-use your sessions. There are many others. It's very rare that the run-time improvement you get from making your tests dependent isn't completely lost in the extra time you need to spend dealing with the consequences of dependent tests failing.

Answer 5

Simple answer: yes, open the page for each test.

Why: tests should be independent, so you should be able to run any test without the others.