2

I know good test cases are isolated (that is, they don't depend on other tests or objects created by other tests). However, how should one proceed when a test becomes the precondition for another test?

As an example, let's say test 1 validates a new user account can be created in the application under test (so test 1 will focus mainly on following the needed steps to create a user account). Then, another test, let's call it test 2, verifies a user can log in with valid credentials. In order to do so, valid credentials should exist, i.e., the user account needs to be created. So test 1 becomes a precondition for test 2. If I want to run test 2 without having run test 1 first, it will fail. This doesn't sound like a desirable situation.

So how should I handle it? I have a suite written in Cucumber using Gherkin, where "Given" steps should be used to place test preconditions. So how can I use test 1 as a precondition for test 2? Is this even right?

Improve this question
1
  • Tests depend on data , not on other tests.Data can be created separately in various ways, depending on situation.
    – Vishal Aggarwal
    Commented Feb 26, 2018 at 1:38

4 Answers 4

Reset to default
4

Tests should be independent and create whatever they need to work. That said, there can be difficult cases.

The situation you present is very common. It's a special case that presents itself to most automation engineers as some point. It's not so much an example of dependencies that cross tests and more a specific issue - creating a user account.

I have observed 'but first I need to create the account' as one of the most common problem areas for testing. Sometimes it includes "and then respond to the account verification email, using the code it provides" and steps like that.

This is a hard problem to solve. Their is no 'perfect' solution and many of the factors will be specific to each situation. Some of the solutions will not work in some situations for a variety of reasons.

  1. Use automation to create the account and then use it. This often doesn't work as there will be a verification code or email to respond to.
  2. Have a special test login that only works against the test database so credentials can be stored in plain text as not a production security issue. The test server may need to be behind a firewall depending on the security requirements. The fact that ex-employees may know the login is one reason why this may be needed.
  3. Have an ability to avoid the login for a specific test user (that doesn't exist in prod) in the test system only.
  4. Use the existing ability to create a user through an api and then use that user in the testing
  5. Create a new ability to create a test user (only) through an api.
  6. Create multi-level testing of all the components involved so that the functionality under test does not also test the 'new user creation' and so an existing user can be used (so this UI test would be in conjunction with other unit and integrated tests).
  7. Store the credentials needed but obfuscate them. It is often a simple but effective approach to have to do a little bit of work to decode them to prevent casual and immediate use. For instance obscuring the names 'username' and 'password' (and variations like pw, pwd, etc) and using simple things like 'use next letter' as tribal knowledge that isn't recorded can be surprisingly effective at preventing casual hack break-ins and exposure of information. Just be aware that obfuscation is not secure authentication. This is also an approach that could be combined with others. The best security is multi-layered.

One of the most effective ways to help address the issue is to schedule a meeting with app dev and other team members and go over the issue and the obvious need and get other folks to answer the question 'how do we solve this problem at our company'. There is often a fair bit of 'not my issue' around this area... but we must have automated tests. So automation engineers often get stuck in this trap and have to do strange hacks or simply get stuck. Make this a whole team problem and have application engineers take the lead on solving it with their automation engineer team members.

Improve this answer
2

In your example test 2 is only dependent upon test 1 if you are creating a new account on every test run. This is slow and possibly redundant.

You can validate account login separately from account creation, provided you have a preexisting account on the system that you know the credentials for.

It's common to have tests that are dependent upon other features. Try to keep the dependencies in the setup and teardown steps, so that it's clear what are test failures vs script failures.

Improve this answer
2
  • I see that, but I don't have an API to create users programmatically so I have to go through the "account creation" test. And it's not always that I need to create a user. Some tests don't require it (although most do). So if I add my "test 1" (user creation) as a background it will run before each test and that's not exactly what I want, I'm ok with jut one user account.
    – Floella
    Commented Feb 21, 2018 at 11:50
  • If you aren't testing user creation you don't need to create a user. You just need a user. One way to speed up the process is to have some users that can be used for testing sitting on the system ready for use.
    – sphennings
    Commented Feb 21, 2018 at 14:40
2

There are two main approaches you could use. Both make use of having a setup routine that runs once at the beginning of your test run - usually it will be a ClassStart or ApplicationStart type of event. You should also run a cleanup routine at the end of the test run to roll back any changes you have made.

  1. If you have a static known starting point - If you have the ability to use something like a pre-configured test database that you control, you can restore the database before starting your tests, and use the known user accounts that are in the database.
  2. If you have no control over application data - In a situation where you can't reset the application data to a known starting point, your best option is to use the setup routine to create the user account you will use for tests that require a logged in user. This account will not be the same as the account you create as part of the user creation test - you're just going to create it because you need a known user to perform a number of other tests.

Both these approaches reduce the dependencies between tests, but make the test run dependent on a successful setup - and should abort the run entirely if setup fails. As long as your setup is stable, this is an advantage because it gives you earlier warning of any problems.

Improve this answer
2
  • Thanks :) In this case I have no control over application data. But since there's no API I can use, the only way to create a user account would be by running a test that creates it (or, if we don't want to call it a "test", it would still execute the same behavior the account creation test steps perform). Would it still be a good practise to "execute a test" in a suite setup?
    – Floella
    Commented Feb 21, 2018 at 13:55
  • 1
    @Floella You should separate your actions from your validation. If you have a method that creates an account. That should be used in the test validating account creation but it can also be used in setting up other tests if you need to create a new account for that test.
    – sphennings
    Commented Feb 21, 2018 at 16:25
1

Componentization is key in BDD.

Let's say test 1 is a behavior of account creation and test 2 is a behavior of login.

Create two separate components one for creation and other login and make of use of them wherever required.

According to the feature your testing you have to choose which behavior to be used in GIVEN acting as a precondition.

Improve this answer
2
  • What do you mean by "components"? So far I have added my account creation test as a background for all tests, but this is not ideal, as it will run before each test (I only need an account created once) and I'll need to copy and paste the same account creation test in each feature file where I need an account created.
    – Floella
    Commented Feb 21, 2018 at 11:52
  • I referred to step definition or methods as a component. In this case you could remove Background keyword and paste the same statement or you could creating two features one for userRegistration and another for userLogin which will contain two different backgrounds.
    – Bharadwaj Pendyala
    Commented Feb 21, 2018 at 12:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.