Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It only takes a minute to sign up.
Sign up to join this community
I want to know, which is the best way to run automated and acceptance tests?
Because now I have two environments, which are develop and qa, but I don't know if the better way to run these tests is during the build of project in develop environment or after deploy in qa environment, Could you give me your opinion about this?
If QA and DEV environments are equal (have the same dependencies installed, the same build, etc.), there is no much sense to run acceptance tests on both of them, since it will be just duplicated execution and additional overhead in terms of analysing test results on two environments instead of one.
Acceptance tests are the tests which provide you additional confidence in what you're doing. You'll get maximal value from using them in case if you run those test on the environment which is as close to production on as possible.
You haven't mentioned what's your deployment process so that it's really hard to say exactly where to execute acceptance tests.
But nevertheless consider following issues when dealing with acceptance testing:
Hope that will help you!
Run all the tests just right after the build as soon as possible, before deploying to the test environment. Developers should have feedback about their check-ins as soon as possible. Also you say the test-env is used for manual testing, why would you want to risk deploying a non working build. Certainly if you could have run the tests first.
In short the ideal flow looks like:
Unless both environments are really different I see no need to run al the tests again in the test-env.
To save power I would not run the acceptance tests and deployment if the unit-tests are already failing. Make it a priority to always keep all the test passing.
This makes sure the test-env always works and manual testers will not be wasting time reporting broken builds which are already covered by automated tests.
to my experience in automation test, it is more valuable to run in qa environment but you should communicate with developer frequently on changing part which may affect your automated test cases so you can prepare your automated test cases during development phase and give priority to more static features of the product to prevent updating automated cases more frequently. Also running acceptance cases may run in both environment but my suggestion is share key test cases of accaptance test cases to developer team and let them run before they give package to qa environment, also you can guide and help them to run these cases so both of you can agree on the test result and it prevents blocking test the package in qa environment. In qa environment you can run all acceptance test cases and automated test cases.
