Unskipping skipped tests once bugs are fixed?

Question

I'm wondering how others in the industry deal with this. Right now I do this in my tests that have known bugs and aren't worth taking the time to run:

from nose.plugins.skip import SkipTest
    def empty_documents_test(self):
        raise SkipTest('Bug #1723 not fixed yet as of 2012-03-22')

This works great for skipping over the tests. But what about UNskipping them? Right now I'm manually checking our issue tracker to see if the issue number has a new status of "Developer Done" (i.e. a fix has been committed).

Is there a better way? Maybe a Python decorator that implies "run this test, but we're expecting it to fail, so do something special when it does"?

Answer 1

Yes, there is another way. I call it “workarounds”

Ingredients: 1. In the separate file, define all bug numbers as contestants

cont int BUG_48484 = 48484
cont int BUG_5555 = 5555

2. Create a dictionary/hash with bug names and descriptions

MyBugList = (

*# Commented: BUG_48484 => “Application crashes”,*
BUG_5555 => “The Large Hadron Collider plugin hungs”
)

While the bug is in list – that mean it is not fixed. To mark the bug as fixed – just remove or comment it from the list

3. Write a simple function that looks if the bug_id exist in the list. It returns true if the bug is not in the list

def Is_bug_fixed(bug_id)
   bug_found =false
   if MyBugList[bug_id] != null
       bug_found =true
   return not bug_found

4. Create your own SkipTest

def MySkipTest(bug_id)
   if not Is_bug_fixed(BUG_5555)
     raise SkipTest(bug_id  + MyBugList[bug_id])

5. Usage:

def empty_documents_test(self):
        raise MySkipTest(BUG_5555)

Now all your bugs are described in a separate file. You will still need to update the status manually, but it will be easy now.

You can add additional logic and logging to Is_bug_fixed

You can extend the MyBugList with advanced information.

I would like to recommend my presentation, but, sorry, it was made for Russian testers community, so it is in Russian. But the code snippets is in C# :) (start from slide 22)

http://blog.zhariy.com/2012/03/blog-post.html

You can find the code also here: https://gist.github.com/2267565