0

Background:

3-week sprint, 2-days testing against staging and 1-day testing against qa. qa environment is only up for the duration of 1 day as it costs a bit money to have it running on AWS. We have established automated tests but it is not uncommon for them to fail against qa.

The solution I can think of is to establish a suite of manual tests that we can execute in case of automated test suite against qa fails, given there is not enough time to fix broken automated test in such a short time frame. (we can fix them later, but different ones break from time to time)

Before I present this approach to the management, I think I should have a proper name for this kind of manual test suite. Do we have a terminology for this kind of tests already?

5

I don't think there's a special name for this particular situation. Nor do I think we should create one. I think we should try to keep things simple, rather than layer more artificial names of top of what we currently use.

We have established automated tests but it is not uncommon for them to fail against qa.

(we can fix them later, but different ones break from time to time)

I'd try to figure out why they fail and what I can do to repair them. This is key here in the whole situation. Having flaky tests (it seems so from the description, although it might be that the app is broken from time to time) not only because they don't really bring any value, but also because you keep spending a lot of time on them instead of on testing.

All in all, I'd keep stable tests in the suite and retire those flaky ones. Then I'd start working from this there.

2
  • +1, for figuring out the root cause of why tests are failing in the first place. – Vishal Aggarwal Mar 11 at 1:03
  • 2
    IME, I have seen people spend more time in placing band- aid solutions to automation than actually required to address the root cause in long term. – Vishal Aggarwal Mar 11 at 1:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.