Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

In a large set of tests written for RSpec there are some tests who are expected to fail (at least, for a couple more months). These known failures makes it difficult to sort through the results reported by RSpec for regressions (new failures since the last run).

Is there a easy system that works with RSpec results to keep track of old testrun results and only report on new failures?

share|improve this question
add comment

3 Answers 3

Although I'm not familiar with RSpec, I've been in the same boat before. In the end, if it's expected to fail, and it fails, should it not pass? Additionally, is there any way to ingore them other than just putting them into a new suite? (one of my favourite things about NUnit).

share|improve this answer
add comment

Why run tests you know are going to fail?

I would recommend breaking them into their own test suite(s) and removing/commenting them from being run.

Seeing as how they're expected to fail, I think it's safe to assume they are logically connected somehow and potentially deserving of their own suite regardless of their passing or not.

share|improve this answer
    
On those lines - this Stack Overflow question might be useful: stackoverflow.com/questions/5004286/… –  testerab May 26 '11 at 22:14
add comment

How about implementing a retry logic? If tests are flaky and cant be avoided then retrying the failed tests might help.

share|improve this answer
    
Arpit, I think you misunderstood the question - the poster is asking for a way to handle tests that will pass when known problems in the application being tested are corrected. This is a case of a regression having been introduced and deemed a low priority fix, so the tests are correctly reporting a failure, but it is a known failure. –  Kate Paulk Jul 25 '13 at 11:38
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.