7

We have picked multiple existing end-to-end tests to be executed against our production instances, triggered after a new release and daily.

Some of the concerns we have are potential side effects of the smoke test run and interference with the active users during the run.

What are the general guidelines for running smoke tests against production and what issues can we expect to encounter?

Additional Info:

The application under test is a web application (AngularJS-based) in the healthcare insurance space. Tests are written in Protractor/WebDriverJS using Jasmine test framework.

Improve this question
5
  • Is there any reason to execute tests on production? What is the nature of this software you are testing? E.g. e-commerce, education or etc?
    – Yu Zhang
    Commented Jun 9, 2017 at 3:27
  • @YuZhang sure, added some information into the answer. These smoke tests should contribute to helping us to ensure correct release and release configuration installation, serving as an indication that the most critical parts of the app are behaving in the way we expect them to behave. At the moment, these are basically high-level "view-only" tests - going through the major screens and ensuring they are displaying the data and have all the required "controls". Thanks, good question.
    – alecxe
    Commented Jun 9, 2017 at 3:33
  • 1
    Closing mafia strikes again? And they are cowards enough to vote for close without any comments, so not to left any traces of a name? Shame on you guys!
    – Peter M. - stands for Monica
    Commented Jun 9, 2017 at 14:12
  • @PeterMasiar well, I was not too happy with the form I've created the question in..it feels a bit too broad, but I think it may be useful for the community in the long run. If you have ideas how to improve the question to make it less broad, please edit. Thank you!
    – alecxe
    Commented Jun 9, 2017 at 14:18
  • 1
    @alecxe - yes the question MIGHT a bit broad, but IMHO it IS perfectly valid for this forum. Compared to most other questions asked this week (trivial webdriver handling issues which can be overcome with just few weeks of coding experience), it is very much on topic and adds value to this forum as resource for QA and DevOps. My beef is NOT with you asking the question, but with the Closing Mafia cowards who voted to close the question (without comments) because it was not trivial enough for them to understand.
    – Peter M. - stands for Monica
    Commented Jun 9, 2017 at 15:46

2 Answers 2

Reset to default
4

Just design these after production deployment tests to not interfere with the users. It is not a load-tests and if they are view-only then everything should be alright. If this is the case I cannot come-up with anything real users will have trouble with, unless you system is only build to have 1-2 users at a time, lol.

If you have a continuous deployment model you need to have some automated tests and or checks to verify the application was deployed correctly. These are called post-deploy tests.

Applications with large users bases monitor user behaviours. Checking if users are still loggin in and finishing the major workflows in the application. This could be an alternative to running tests against the environment.

Improve this answer
3

It is very important that our application`s smoke scenario should be run in Production environment.

These are possible ways :

  1. Create your dummy user name , password and healthcare data (which should be test data on client side) So ask for permission to create minimal of such data. After that use it.

  2. Create another database (replica of product env config in another server or domain and check with same build files and with same scripts).

  3. Last which I think is good option, Do not create/add/update/delete any data. Just do fetch operation and verify it is showing perfectly. Like patient details, hospital details etc.

Note : Please take care while using production environment. And ask permission before take any action. Also use only possible scenario of given list.

Hope you get some solutions.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.