Is seeding test data in e2e tests a correct approach?

Question

We have this app where creating an account takes about 10-15 min and the process is really complicated. Thus we started using seed accounts which are basically already created accounts that are implemented after each db clean. Edit: I meant we started using 'seed accounts' which is some kind of setup, so whenever we clean our db, the db itself is populated with that predefined setup (already created accounts)

We have plenty of test cases, that tests features on these seed accounts. However Im wondering how many e2e tests (create account + test these features) should we include? Because seed accounts are mocked accounts which may be different than the real ones created manually by user.

Going for a testing these features with e2e testing (creating account + features testing) have a lot of cons:

1. a lot of time wasted
2. tests might be interrupted with problems during account creation

Another thing that struggles me is that tests made on seed accounts will be also covered with e2e tests (if we go with testing these features on created account) so its tests coverage duplication.

So my question is:

Should we go with approach whether we tests features on seed accounts and then perform e2e test only to check if account has been correctly created (by asserting successful login)?

Answer 1

No, Create a single E2E test without seed accounts and test individual features in integration tests using seed accounts.

In this scenario, I would create a single E2E test covering primary user flow( how end user typically will use the application 80-90% of the time in production).In this, I will not use any mocking/ DB insertions for test data creation(seed accounts).End to end flow in a single shot just like the real user.

For other feature level integration tests, I will use the seed accounts and each test will focus on feature/ sub feature it is primarily testing, not on the entire flow.

In general, I will make sure there is no redundant test cases/ test verification steps throughout the suite which is being executed as part of a single run.

In a recent code review of a test team, I found 20-30% test assertions were redundant throughout the suite and removing them significantly reduced the execution time at the same test coverage.

Answer 2

Yeah what you're describing for testcase setup is definitely blowing up the duration and size of your tests. I understand why this could cause you some speed and stability issues. You're going down the right path thinking about seed data but I would also suggest thinking about how you get the data into the application under test in a way that is quick and reliable.

For my own seed data, instead of creating the data that the test depends on as part of my automated test, I create the data before the environment has even been deployed as a bootstrap for the database.

1. If you don't have a minimal database to start from, work with your developers to get it created. Ideally your starting point is only the data you need to get up and running "as a user" from there.
2. Figure out "the most realistic way" to create the data I want to use for testing. I usually try to mimic how consumers of our app do things. For me this is usually a script using one of our REST apis, a script running selenium or similar against our app, or manual data entry.
3. Pick which approach you want to try or that makes sense for the specific case
4. Run the script or do manual data entry
5. Update the minimal database data you started from to persist your changes, use this updated data to bootstrap your database for applications under test

I typically also keep a running list of actions I've done as I'm creating data. This makes it easier to confirm that it still exists and works as I intended it to when you test the new bootstrap. This dataset should get all of the same migrations you'd run against other environments as periodic updates.