2

Hypothetically, if an automation engineer had to write a test for commenting on a post in a group on Facebook and did not have direct access to the db, but instead to the rest api, the automated test would need:

  1. A user
  2. A group
  3. A post in the group

Since a test should not depend on data from previous tests, should this test create a group, then create a post and then create a comment on the post? Should the same test then delete the comment, delete the post, and delete the group? There are lots of points for potential failures here. What are some best practices for getting an application into the state necessary to perform an atomic automated test? Create and delete the data via the API as part of the test setup and teardown? Something else?

5
  • Hi why you think it's not good to delete the state?
    – PDHide
    Commented May 30, 2020 at 6:57
  • The only thing that we could miss out in such cases ( that I can think of ) is to find out how the system behaves when the comments grows with time , I don't think that's a good test use case to cover on a production system through test data . Instead you can cover this scenario in your dev machine , keep everything other than a dummy comment for example ; " this comment shouldnt be deleted , this was commented at {{current timestamp }}"
    – PDHide
    Commented May 30, 2020 at 7:00
  • A clean state system may work as expected but not a system thats being used in real conditions , but both are different test scope . Your team should decide on how to cover it , have a dedicated system to cover this , or add permanent comments like I mentioned that will never be cleaned up . And so on .
    – PDHide
    Commented May 30, 2020 at 7:03
  • Agree. Just wondering about all the overhead of having the UI test do all those steps before it actually tests the thing. Is it better to use the API to do all the setup and teardown?
    – pgtips
    Commented May 30, 2020 at 14:12
  • 1
    unless you are validating delete feature, it doesn't make sense to delete using UI as it will time consuming. You should delete or clean the setup in one go using DB or API, API is more preferred as it ensures more access controls and avoids unintended changes
    – PDHide
    Commented May 30, 2020 at 14:18

1 Answer 1

1

So, here is what we do in such scenarios:

You are right. Test should not depend on data from previous test. But still, you can have test data created for tests and that data can be used repeatedly.

For example, in this case, we will create dependent data manually. It's a one time activity. So, we have an existing user and an existing group and an existing post.

As part of the test:

  • User will log in

  • Add a comment to the post in the group

  • Validate the user was able to comment successfully

As part of the teardown, delete the comment (from UI or api, whatever works). Your test can execute any number of times, with the pre-requisite data you created once.

That's it.

Why we take this approach?

  • We don't want to have too many pre-requisites.

  • It will increase the test execution time significantly.

  • Moreover, if the api endpoint to create user/group is down, your test won't be executed.

We have many scenarios where we do things as part of pre-requisites, for example, creating users etc..but we try to keep it to a minimum.

We have tried different things; as you said, creating all the dependent data before the test execution and then cleaning everything after the test execution completes. But, we found the above approach as the simplest with practically no overheads.

2
  • 1
    Not saying this is a bad approach, but how do you handle leftovers from previous tests ? let's say the comment deletion was successful but corrupted the DB for the next test, or even worst corrupted the DB in such a way that will be felt only in few tests ahead ?
    – Rsf
    Commented Jun 1, 2020 at 7:14
  • Well. We take this approach where the risk for such mess up (like corrupting a DB etc. is very unlikely). If the tear down action has such a huge impact that it can fail 10 subsequent tests, then the approach needs to be revisited. But from my experience, the likelihood of a disaster, by an action like deleting a comment as part of the teardown is very low.
    – Aalok
    Commented Jun 1, 2020 at 7:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.