Skip to main content
Tweeted twitter.com/#!/StackSQA/status/164082438228815874
added 75 characters in body
Source Link
Josh
  • 91
  • 3

Use case: Fitnesse is used for automated testing of the web site.

SUT (software under test) contains a known bug. Say, we expect that web page contains "Changes saved successfully" string but this string is missing because of the bug. So in Fitnesse, this test case is marked as red.

Suppose, in another test case we expect that web page contains "A user created successfully" string. And it worked just fine until the last test execution. So, now this test case is also marked as red.

So, now we have red light for two test cases: a well known bug and a newly found bug. The problem is they are both marked as red. So when I have a look at test results I can't distinguish which of them are known and new.

Of course, I can compare test history and see the difference between two runnings (with and without a newly created bug).

Or I may not execute a test case with a known bug.

Or I can tune it so that this test case has always been green and change it when the bug is fixed.

But all this is very inconvenient. What I want is to distinguish two kinds of bugs (a well known bug and new bug) so that:

  1. By looking at test results, I could easily say: this is a new bug and those are old. For example: No bugs - green, Already known bugs - yellow, New bugs - red.

  2. It is easy to change test case when the bug is fixed.

What are the best strategies for acceptance tests, in general, and Fitnesse, in particular?

Use case: Fitnesse is used for automated testing of the web site.

SUT (software under test) contains a known bug. Say, we expect that web page contains "Changes saved successfully" string but this string is missing because of the bug. So in Fitnesse, this test case is marked as red.

Suppose, in another test case we expect that web page contains "A user created successfully" string. And it worked just fine until the last test execution. So, now this test case is also marked as red.

So, now we have red light for two test cases: a well known bug and a newly found bug. The problem is they are both marked as red. So when I have a look at test results I can't distinguish which of them are known and new.

Of course, I can compare test history and see the difference between two runnings (with and without a newly created bug).

Or I may not execute a test case with a known bug.

Or I can tune it so that this test case has always been green and change it when the bug is fixed.

But all this is very inconvenient. What I want is to distinguish two kinds of bugs (a well known bug and new bug) so that:

  1. By looking at test results, I could easily say: this is a new bug and those are old.

  2. It is easy to change test case when the bug is fixed.

What are the best strategies for acceptance tests, in general, and Fitnesse, in particular?

Use case: Fitnesse is used for automated testing of the web site.

SUT (software under test) contains a known bug. Say, we expect that web page contains "Changes saved successfully" string but this string is missing because of the bug. So in Fitnesse, this test case is marked as red.

Suppose, in another test case we expect that web page contains "A user created successfully" string. And it worked just fine until the last test execution. So, now this test case is also marked as red.

So, now we have red light for two test cases: a well known bug and a newly found bug. The problem is they are both marked as red. So when I have a look at test results I can't distinguish which of them are known and new.

Of course, I can compare test history and see the difference between two runnings (with and without a newly created bug).

Or I may not execute a test case with a known bug.

Or I can tune it so that this test case has always been green and change it when the bug is fixed.

But all this is very inconvenient. What I want is to distinguish two kinds of bugs (a well known bug and new bug) so that:

  1. By looking at test results, I could easily say: this is a new bug and those are old. For example: No bugs - green, Already known bugs - yellow, New bugs - red.

  2. It is easy to change test case when the bug is fixed.

What are the best strategies for acceptance tests, in general, and Fitnesse, in particular?

grammar
Source Link
user246
user246

Use case: Fitnesse is used for automated testing of the web site.

SUT (software under test) contains a known bug. Say, we expect that web page contains "Changes saved successfully" string but this string is missing because of the bug. So in Fitnesse, this test case is marked as red.

Suppose, in another test case we expect that web page contains "A user created successfully" string. And it worked just fine until the last test execution. So, now this test case is also marked as red.

So, now we have red light for two test cases: a well known bug and a newly found bug. The problem is they are both marked as red. So when I have a look at test results I can't distinguish which of them are known and new.

Of course, I can compare test history and see the difference between two runnings (with and without a newly created bug).

Or I may not execute a test case with a known bug.

Or I can tune it so that this test case has always been green and change it when the bug is fixed.

But all this is very inconvenient. What I want is to differdistinguish two kinds of bugs (a well known bug and new bug) so that:

  1. By looking at test results, I could easily say: this is a new bug and those are old.

  2. It is easy to change test case when the bug is fixed.

What are the best strategies for acceptance tests, in general, and Fitnesse, in particular?

Use case: Fitnesse is used for automated testing of the web site.

SUT (software under test) contains a known bug. Say, we expect that web page contains "Changes saved successfully" string but this string is missing because of the bug. So in Fitnesse, this test case is marked as red.

Suppose, in another test case we expect that web page contains "A user created successfully" string. And it worked just fine until the last test execution. So, now this test case is also marked as red.

So, now we have red light for two test cases: a well known bug and a newly found bug. The problem is they are both marked as red. So when I have a look at test results I can't distinguish which of them are known and new.

Of course, I can compare test history and see the difference between two runnings (with and without a newly created bug).

Or I may not execute a test case with a known bug.

Or I can tune it so that this test case has always been green and change it when the bug is fixed.

But all this is very inconvenient. What I want is to differ two kinds of bugs (a well known bug and new bug) so that:

  1. By looking at test results, I could easily say: this is a new bug and those are old.

  2. It is easy to change test case when the bug is fixed.

What are the best strategies for acceptance tests, in general, and Fitnesse, in particular?

Use case: Fitnesse is used for automated testing of the web site.

SUT (software under test) contains a known bug. Say, we expect that web page contains "Changes saved successfully" string but this string is missing because of the bug. So in Fitnesse, this test case is marked as red.

Suppose, in another test case we expect that web page contains "A user created successfully" string. And it worked just fine until the last test execution. So, now this test case is also marked as red.

So, now we have red light for two test cases: a well known bug and a newly found bug. The problem is they are both marked as red. So when I have a look at test results I can't distinguish which of them are known and new.

Of course, I can compare test history and see the difference between two runnings (with and without a newly created bug).

Or I may not execute a test case with a known bug.

Or I can tune it so that this test case has always been green and change it when the bug is fixed.

But all this is very inconvenient. What I want is to distinguish two kinds of bugs (a well known bug and new bug) so that:

  1. By looking at test results, I could easily say: this is a new bug and those are old.

  2. It is easy to change test case when the bug is fixed.

What are the best strategies for acceptance tests, in general, and Fitnesse, in particular?

Source Link
Josh
  • 91
  • 3

Failed automated tests: how to distinguish known and newly introduced bugs?

Use case: Fitnesse is used for automated testing of the web site.

SUT (software under test) contains a known bug. Say, we expect that web page contains "Changes saved successfully" string but this string is missing because of the bug. So in Fitnesse, this test case is marked as red.

Suppose, in another test case we expect that web page contains "A user created successfully" string. And it worked just fine until the last test execution. So, now this test case is also marked as red.

So, now we have red light for two test cases: a well known bug and a newly found bug. The problem is they are both marked as red. So when I have a look at test results I can't distinguish which of them are known and new.

Of course, I can compare test history and see the difference between two runnings (with and without a newly created bug).

Or I may not execute a test case with a known bug.

Or I can tune it so that this test case has always been green and change it when the bug is fixed.

But all this is very inconvenient. What I want is to differ two kinds of bugs (a well known bug and new bug) so that:

  1. By looking at test results, I could easily say: this is a new bug and those are old.

  2. It is easy to change test case when the bug is fixed.

What are the best strategies for acceptance tests, in general, and Fitnesse, in particular?