0

I am working on a functional test automation project for a web application, and I want to integrate my tests with Jenkins.

The problem is that I cannot make a decision on the initialization technique of my test data to be able to replay my test several times.

The techniques I know are:

  • Backing up and restoring the database
  • Use of web services to update test data
  • Using automated test cases (Selenium) to recreate test data

Are there other techniques for resetting the test data? Knowing that I have a constraint, I cannot restore the database because I do not have access for security reasons.

Improve this question
2
  • 2
    "I do not have access for security reasons" Could you detail more of it? Are you not from the development team?
    – João Farias
    Apr 10, 2020 at 18:06
  • The application is deployed in a secure tenant : it is a tenant set up for testing but in a production environment.
    – Aya Ali
    Apr 10, 2020 at 18:28

2 Answers 2

Reset to default
2

The best technique is to have a separate, empty database that is reset after every test.

This is the technique used by many frameworks.

A lot of the change is also in mindset. In manually testing - which today is often done in an 'exploratory mode'- of a zip code you will randomly try several real ones and expect them to work. So you need all 50,000+ US zip codes loaded. For an automation test you could have just one valid zip code, one invalid, etc. I've seen this done on a large scale. The mindset change is the hardest part.

As many situations make the above very hard if not possible to do, other mitigation routes include:

  • restore db or simply delete and copy if possible
  • reverse the transactions recorded
  • delete the new records that were created

A key question I think you need to ask the project owner - testing requires decisions on database approach, what options would they like - and be sure to outline pro, cons and costs with each alternative. If the response is 'no db access to change records', then refine the details for that options showing the consequences and costs. Be sure to focus on costs over the coming year as db issue quickly become insidious in my experience.

Improve this answer
3
  • Thank you for your response. In my case, the approach to initializing the database is not possible at the moment I will try to convince the team. But, are there any other method of resetting test data?
    – Aya Ali
    Apr 11, 2020 at 17:29
  • @a.kh: I think you're focusing on the wrong question. There're at least 4 different approaches (only in Michael's answer) you can take, how many else do you expect to be given? Sooner or later you'll have to have that access into the database (you, somebody else who resets/deletes/copies the data, via some API you could use, ...), so why not try to get that access in the first place? Your approach right now is like trying to figure out ways to turn on a light when the lightbulb is broken.
    – pavelsaman
    Apr 11, 2020 at 19:45
  • 1
    @pavelsaman : Thanks for your feedback. In fact, i'm just trying to ask if there are other methods, other than mentioned in my question or in the responses of people who gave me their feedback in order to be objective in my study. Knowing that I always prefer the restoration of the database.
    – Aya Ali
    Apr 13, 2020 at 10:20
1

Let me start my answer with a Highlight.

Best technique is always context-driven. No technique will be best in all circumstances.

As @Micahel Durrant as pointed out, you may need to do pros, cons, time and cost analysis for all available options. IMHO I'm not a big fan of restoring the DB approach because of the following reasons and I may be wrong here.

  1. Having empty schema is not a great strategy for edit scenarios. Not sure your test cases have this constraint but let me elaborate with an example for you.

Suppose you created a new user account with code that was deployed yesterday and account creation and updation feature was working perfectly fine yesterday. Today, you have deployed new code, account creation is working fine, even updation is working fine for users created today but what if updation is not working anymore for the accounts that got created with old code? You won't be able to test this kind of scenarios if you are flushing the data from Database.

  1. Another shortcoming with CI/CD and data restoration approach is for every deploy before you trigger the scripts, you will have pre-step to restore 'standard DB schema' on the database server and post-step after the execution of scripts, you will delete the data again. What if some issue is found but the team is not able to see it in action or reproduce it as you already deleted the data. Of course, this can be handled if you delete and install new schema in pre-step but then you expect this issue reproduction or exploration by developer team should happen before you trigger next execution (or ideally by next deployment) which won't happen most of the time.

If you are executing the end-to-end checks, please think about if you can set up data in the script itself, using services or even UI. It may cost you some execution time but with parallel execution, it shouldn't be a problem. Also, you can create scripts in such way that data created in one script can be used in multiple scripts. Test Runner like TestNG support dependsOnGroups and dependsOnMethods annotations for such a need,

Improve this answer
8
  • Thanks for your feedback. You're right, the database restoration technique does not reproduce the bugs detected by the automated tests. But the initialization of the data by the user interface scripts cause the dependency of the scripts : for example if the setup script fails the main test case is not going to pass?
    – Aya Ali
    Apr 11, 2020 at 17:11
  • 1
    Yes, creating all the setup data needed for even the simplest case can be daunting and indeed too much for most organizations most of the time. Existing developers may laugh out loud at the mere mention of it and that mention is probably not the first one. However the payoff is really huge, really one of the biggest automation requirements for having all that modern CI/CD etc stuff actually working. It's a considerable investment that is required at some point. The cost of doing otherwise is severe over time. This is a leadership - long term view. Like just a few months ahead really.
    – Michael Durrant
    Apr 11, 2020 at 20:11
  • 1
    so expect it to take months to years and require a big effort. Last time I had to deal with this we had to navigate 23 APIs to get data needed for automation!
    – Michael Durrant
    Apr 11, 2020 at 20:14
  • 1
    Don't even think of creating data through the UI unless you want the guarantee of constant pain in dealing with issues in the years ahead. It will be increasingly hard to meet the business needs when you have to maintain UI data creation script. Although they are 'scripts' they are not the automation you seek because UI scripts take a LOT of maintenance and fail intermittently due to browser async, etc.
    – Michael Durrant
    Apr 13, 2020 at 10:29
  • 1
    @Amol: I will try the use of web services in my study. Thanks for your a lot for all these explanations. I come back to you with good news :).
    – Aya Ali
    Apr 13, 2020 at 18:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.