Testing Strategy: Modular Test Cases

Question

Currently, my company follows a modular test case approach, in the context of business workflow tests. In this approach, we have a lot of component-level test cases, that, by themselves, are not part of our regression suite; however, these test cases are each used in larger, workflow-style, end-to-end, data-driven test cases. For example, if Gmail was our application, we would have something (much more built out, but hopefully you get my point) along these lines:

Component Test Cases

-Login, parameterized (username, password) – not a regression level test, by itself

-Compose message, parameterized (to recipient, subject, message, cc recipient) – not a regression level test, by itself

-Add attachment, parameterized (filepath/name) – not a regression level test, by itself

-View Inbox Message, parameterized (from recipient, subject, message, attachment) – not a regression level test, by itself

-Forward Message – not a regression level test, by itself

-Reply – not a regression level test, by itself

-Logout – not a regression level test, by itself

Regression Test Cases

-Forward a message: --Login (“Sender”, “password1”) > Compose Message (“Receiver”, “Automation”, “Question”, “”) > Send > Logout > Login (“Receiver”, “password1”) > View Inbox Message (“Sender”, “Automation”, “Question”, “”) > Forward Message > Compose Message (“Person to Forward to”, “Fwd: Automation”, “Question”, “”) > Send

-Send a message with attachment: --Login (“Sender”, “password1”) > Compose Message (“Receiver”, “Automation”, “Question with Attachment”, “”) > Add attachment (“File”) > Send > Logout > Login (“Receiver”, “password1”) > View Inbox Message (“Sender”, “Automation”, “Question”, “File”)

Here is the predicament we are facing: very few Test Case Management tools (QualityCenter and possibly QTest) support this style of testing. Many of the other TCM tools we have investigated instead support exclusively component-level when it comes to data-driven testing. (i.e. can login 50 times with 50 different sets of data, can send 50 different messages with different data). We are starting to wonder if we are the outliers, which is where my question comes in…

Do a lot of companies follow the workflow-style tests composed of component-level tests, or do companies instead typically iteratively execute very small test cases with different data permutations, and leave workflow testing to exploratory testing?

Answer 1

You are not outliers - workflow-style tests happen, but the important thing to note is this: No tools offer good record-playback support for workflow-level tests. Record-playback tools and out-of-the-box data driving is built around the assumption that all tests will be uber-simplistic.

This is where using models like the Page Object Model come in, particularly with fluent-style coding.

You would have something like this for your regression tests (using c#-ish syntax because that's where I'm most comfortable):

[Test]
ForwardAMessage()
{
    MessageSentPage = LoginPage.Login(dataSource.Sender, dataSource.SenderPassword)
        .ComposeMessage(dataSource.To, dataSource.Topic, DataSource.Body, "")
        .SendMessage()
        .Logout()
        .Login(dataSource.Recipient, dataSource.RecipientPassword)
        .ViewMessage(dataSource.SenderName, dataSource.Topic, dataSource.Body, "")
        .ForwardMessage()
        .ComposeMessage(dataSource.FwdRecipient, dataSource.FwdTopic, dataSource.FwdBody, "")
        .SendMessage();
    // assert that the targeted message has been forwarded using properties on the message sent page - in Gmail, probably the inbox
}

[Test]
SendMessageWithAttachment()
{
    MessageWithAttachmentPage = LoginPage.Login(dataSource.Sender, dataSource.SenderPassword)
        .ComposeMessage(dataSource.To, dataSource.Topic, DataSource.Body, "")
        .AddAttachment(dataSource.FileName)
        .SendMessage()
        .Logout()
        .Login(dataSource.Recipient, dataSource.RecipientPassword)
        .ViewMessage(dataSource.SenderName, dataSource.Topic, DataSource.body, dataSource.FileName)
    // Assert that the message properties match what you expected to see
}

Your simple tests - your component tests as it were - would look more like this:

[Test]
LoginToEmail()
{
    InboxPage = LoginPage.Login(dataSource.Sender, dataSource.SenderPassword);
    // Assert that the correct user is logged in
}

You would have separate objects defined that hold the parameterized routines you're using to perform the actions you need: your login page object might contain a Login(string user, string password) method, where the inbox page object might contain ViewMessage(string sender, string topic, string body, string attachmentname) and so on.

Then it's simply a matter of structuring your data so you have the correct tests: each test defines a data source, and has one or more rows to work with.

If you want to go deeper into the data-driving, you can take things further and wrap everything into a driver that takes a test type and directs to the appropriate test routine for the type via case statement - something like:

[Test]
RunATest()
{
    switch(dataSource.TestType)
    {
        case Forward: Runner.ForwardAMessage(dataSource.TestObject);
            break;
        // etc.
        default: ;
    }
}

This style of coding is more complicated and less readable, but it does allow you to add tests for any supported actions (as defined in the case statement) without needing to add more code. Unless you're working with something massively complex, you probably don't need to go this far - page objects with fluent styling should work fine for you.