Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

In a large set of tests written for RSpec there are some tests who are expected to fail (at least, for a couple more months). These known failures makes it difficult to sort through the results reported by RSpec for regressions (new failures since the last run).

Is there a easy system that works with RSpec results to keep track of old testrun results and only report on new failures?

share|improve this question

Although I'm not familiar with RSpec, I've been in the same boat before. In the end, if it's expected to fail, and it fails, should it not pass? Additionally, is there any way to ingore them other than just putting them into a new suite? (one of my favourite things about NUnit).

share|improve this answer

Why run tests you know are going to fail?

I would recommend breaking them into their own test suite(s) and removing/commenting them from being run.

Seeing as how they're expected to fail, I think it's safe to assume they are logically connected somehow and potentially deserving of their own suite regardless of their passing or not.

share|improve this answer
On those lines - this Stack Overflow question might be useful:… – testerab May 26 '11 at 22:14

How about implementing a retry logic? If tests are flaky and cant be avoided then retrying the failed tests might help.

share|improve this answer
Arpit, I think you misunderstood the question - the poster is asking for a way to handle tests that will pass when known problems in the application being tested are corrected. This is a case of a regression having been introduced and deemed a low priority fix, so the tests are correctly reporting a failure, but it is a known failure. – Kate Paulk Jul 25 '13 at 11:38

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.