3

I want to know, which is the best way to run automated and acceptance tests?

Because now I have two environments, which are develop and qa, but I don't know if the better way to run these tests is during the build of project in develop environment or after deploy in qa environment, Could you give me your opinion about this?

  • Hi Pedro; there are good reasons to run the tests on both environments, but it depends on what those environments are used for. Is your qa site only for the use of the software testing team? – Vince Bowdren Apr 16 '15 at 16:50
  • @vincebowdren Yes Vince, QA environment is only currently to manual tests and the developers run unit and integration tests in develop environment, we don't have automated UI tests yet – Pedro Henrique Apr 16 '15 at 16:55
3

If QA and DEV environments are equal (have the same dependencies installed, the same build, etc.), there is no much sense to run acceptance tests on both of them, since it will be just duplicated execution and additional overhead in terms of analysing test results on two environments instead of one.

Acceptance tests are the tests which provide you additional confidence in what you're doing. You'll get maximal value from using them in case if you run those test on the environment which is as close to production on as possible.

You haven't mentioned what's your deployment process so that it's really hard to say exactly where to execute acceptance tests.

But nevertheless consider following issues when dealing with acceptance testing:

  • Acceptance tests should test mostly application functionality on the system level and not the components in isolation;
  • Acceptance tests should be run prior to every production deployment, regardless on how you deploying (with "big" builds containing several commits or using "continuous deployment" approach);

Hope that will help you!

2

Run all the tests just right after the build as soon as possible, before deploying to the test environment. Developers should have feedback about their check-ins as soon as possible. Also you say the test-env is used for manual testing, why would you want to risk deploying a non working build. Certainly if you could have run the tests first.

In short the ideal flow looks like:

  • Check-in new code
  • Build
  • Run all tests
    • First the unit-tests
    • Second the slower acceptance tests
  • Deploy to test-env
  • Run small test to see deploy was successful

Unless both environments are really different I see no need to run al the tests again in the test-env.

To save power I would not run the acceptance tests and deployment if the unit-tests are already failing. Make it a priority to always keep all the test passing.

This makes sure the test-env always works and manual testers will not be wasting time reporting broken builds which are already covered by automated tests.

1

to my experience in automation test, it is more valuable to run in qa environment but you should communicate with developer frequently on changing part which may affect your automated test cases so you can prepare your automated test cases during development phase and give priority to more static features of the product to prevent updating automated cases more frequently. Also running acceptance cases may run in both environment but my suggestion is share key test cases of accaptance test cases to developer team and let them run before they give package to qa environment, also you can guide and help them to run these cases so both of you can agree on the test result and it prevents blocking test the package in qa environment. In qa environment you can run all acceptance test cases and automated test cases.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.