How to write test cases when different test data gives different results?

Question

Example test case: Check mobile bill payment after due date.

For the above test case, different telecom operators have different sets of rules. For example, some allow payment after due date, and some don't.

I am confused on how to write a test case for this situation; the same test case may PASS or FAIL based on the test data.

Do I need to repeat the same test case for different sets of data based on rules?

I have found this on one site "Data-driven testing is taking a test, parameterizing it, and then running that test with varying data". How I can add parameters and results in to an Excel document in which I'll write test cases?

Please suggest a way to prepare test cases for scenarios like this.

Answer 1

Always approach this from your test objectives which should be taken directly from your requirements.

You have one usecase, but this has multiple scenarios. Your usecase, 'Check mobile bill payment after due date' is actually quite weak. It is ambiguous in many ways. All tests should follow Hoare logic which is easily done using BDD for your test scripts

Given the due date has not passed / Given the due date has passed
When the mobile bill is shown
Then expected data is shown.

There are two alternatives to the scenario I've posed above, before the due date and on (after) the due date. These scenarios produce different results and the scenario includes their expected result, even if this is a message that says not yet available.

These scenarios could also be further data driven, for example by accounts with different telecom providers, different account types, different due dates around bank holidays, month and year ends, leap years etc.

Answer 2

If you are moving forward for automation approach, applying BDD with "Scenario Outline" feature to your case, it can be fit in one readable scenario.

Scenario Outline: Bill payment after due date
    Given <operator> bill, due date is <dueDate>
     When payment is made today
     Then payment is <result>
Examples: 
    | operator | dueDate   | result   |
    |    A     | yesterday | rejected |
    |    B     | yesterday | accepted |
    |    C     | yesterday | rejected |

While step definitions of these keywords requires parameters:

Given("""^(A|B|C) bill due date is (yesterday|today|tomorrow)$""") { (operator: String, dueDate: String) =>
    // Set a context, which operator to test and when is the due date.
}

When("""^payment is made today$""") { 
    // Execute payment action as of today.
}

Then("""^payment is (rejected|accepted)""") { (result: String) =>
    // Assert the actual payment result whether payment is rejected or accepted.
}

This is how BDD Scenario Outline works and if you have good parameterized step definitions, you can re-use it to create many more test cases for a payment with a concern of due date. For example:

Scenario Outline: Bill payment on due date
    Given <operator> bill, due date is <dueDate>
     When payment is made today
     Then payment is <result>
Examples: 
    | operator | dueDate | result   |
    |    A     | today   | accepted |
    |    B     | today   | accepted |
    |    C     | today   | accepted |

Hope it helps.

Answer 3

With my data driven tests, I have a single test and I push my data through that.

The test could be written in anything that accepts parameters. The data you are using should have the inputs as well as the expected outputs (the expected result of the test based on the inputs). Therefore you write one test case but can test with as many variations as you want.

I frequently use Excel to create my test data for data-driven tests. SoapUI has built in support for this in the form of data sources.

You define each column in the spreadsheet so you can use each cell as a parameter to the test.

You build the test steps, including the check to verify the expected result versus actual result

Then loop....

Here is a link to a JUnit example https://github.com/junit-team/junit4/wiki/Parameterized-tests.

Answer 4

You can run same test case with different test data very smoothly by just writing some utility functions either in vb script or in java or any language you are comfortable with. Automation testing is the best way to do this kind of testing. If you trying web service testing you can simply install this free app in your mobile and test with various test data. Download and install in mobile :https://play.google.com/store/apps/details?id=rester.restapitester