Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Our long term stability test cycle takes about two weeks to complete, during this time another release (sometimes two) is being released. What should I do ?

The test cycle can't be shortened, and we can't run two releases concurrently.

share|improve this question
    
Can you say more about why the test cycle can't be shortened, and why you can't run two releases concurrently? –  Dale Emery Jun 1 '11 at 18:56
    
We can't do concurrent testing due to limited resources (isn't it always ?) of all kinds- testers, device, test equipment. The cycle can't be shortened much without loosing its significancy, it consists of several common use cases, each taking a day to complete. The problem is not efficiency. –  Rsf Jun 2 '11 at 8:36
add comment

4 Answers

Your question as stated is not a solvable problem. Assuming this is the desperate cry for help of the tester trapped at the end of a waterfall process the flip answer is "get more agile."

Of course the usual problem in this situation is that the QA team has very little leverage beyond "Test Sign Off". Worse, if bugs are released the QA team is called on the carpet.

Here are some leverage points to consider.

  • Take your problem statement and turn it around. Ask "What are we really trying to accomplish?". This question has a lot of power to get to the heart of problems that seem intractable.
  • Ask why five times.
  • Ship with less testing as an experiment. Have a way to "pull the plug" if things go wrong.
  • Tell the Dev team that the QA team no longer does "test sign off". The Devs decide when a feature is stable enough to ship. The QA team is on the hook to provide the measurements and data to help them decide.
  • Pick one way to improve the overall process and get one developer to try it out for a release cycle.
  • Make one thing a little better every day. Sometimes chipping away at a hard problem will cause it to unexpectedly crumble.
  • Keep a positive attitude. Focus on what the team is doing well as often as you can.
  • Repetition. Pick a catch phrase for one improvement you want to see ("testable by design" was a good one for me) and repeat it until it catches on.
  • Make sure you aren't part of the problem. Don't say "We can't ...", "We'll never ...", "Bob is an idiot.", etc. Instead consider saying "We don't yet ...", "We haven't started ...", "Bob isn't wrong about ..."
  • Start saying "and" when you mean "but". Say "we" when you mean "they". "We can't complete a testing cycle in less that two weeks AND we need to ship once a week."

Solving seemingly intractable problems is all about finding some leverage. Finding leverage is a classical exercise in thinking outside the box. You have to tap into your passion, patience and be persistent. Look for the small wins and celebrate them to get some momentum.

share|improve this answer
add comment

It would depend on what your goal for stability is, do you need the 2 weeks to be able to generate errors that have been reported or has it "always been done" this way?

I had to maintain a stability environment in a previous job, the rationale for this was because we needed to run load and try to find defects that often only showed up after X# of days. Even though we still had releases going out the stability environment had to run for its time, although we would only update after we reached the expected time then did the update.

If you have to run your test for this amount of time, there probably is not much you can do, if you can shorten it as suggested then discuss with the people who set up the test what you can do to make it realistic and fit within a time you are more comfortable with.

share|improve this answer
add comment

Ok so if I understand correctly here is your scenario:

  • A. you have a long period of testing say 2 weeks and that cannot be shortened.
  • B. you have to release multiple times within that 2 week period say, once a week.
  • C. you cannot run two releases concurrently.

Time for some tough love, something has to give you either need to :

  1. Identify a smaller subset of tests to reduce what you are testing, but this violates A.
  2. Reduce the number of releases so you release only once every 2 weeks but this violates B.
  3. Run two releases concurrently, which violates C.
  4. Use automation to reduce the execution time, which again violates A.

So, simply put, one of the three rules will need to be broken, otherwise you simply have to accept the fact that your testing will never be completed on any given release.

share|improve this answer
add comment

Your options here would be controlled by dependencies & overlaps between the two test cycles.

Analyze them first both short & long term -- resources,test infrastructure,test results,business impact

Then,as you mentioned that you can not run two releases concurrently, try and spot overlaps,see what work could be done which would help both the test cycles.

Is there any ? If yes - great,you can support that.

if not(and for not overlapping areas of work) try decoupling resources,activities or even look at outsourcing to another team

and as a last resort,prioritize and come up with a suggestion list for the business/project leadership to take a call,on which items needs to be dropped ?

being fearless and breaking the news of inability to sustain is a better approach rather than reeling under the pressure of both deliverables and failing in both...

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.