Writing Better Stubs To Local Behaviour Testing

Question

I have recently faced an embarrassing bug in PPE and want to improve my testing strategy so that I can mitigate such bugs in future during local testing itself.

I have an API that interacts with a downstream service to invalidate a previously created quote for an order.

---(user-token)--->API------(service-token)----> Quote

You cannot use the passed user-token to call Quote. Only generated service-tokens are allowed.

Bug Description: While calling Quote, I was passing the user-token instead of the service-token in my code. This was detected during integration testing in PPE.

How It Passed Local BDDs: We were passing our own service tokens even to call the API. Since those tokens were whitelisted with Quote, the tests passed.

I even ran the entire journey on my local environment with actual services' endpoints, and all seemed to work perfectly. This was because I kept passing my own service's service-tokens to call the API.

My question is how can I detect such bugs in local BDDs? Writing stubs to such fine details is a gargantuan and difficult task.

I apologise for the abstract description, but I really need help here.

Answer 1

This question requires more clarity but, I would want to add something here that could help other testers and developers who are passionate about quality.

The first question is

Are service tokens getting created each time dynamically ? if so why are you hard coding that in your scripts ? –

And to answer your question about:

"We were passing our own service tokens even to call the API. Since those tokens were whitelisted with Quote, the tests passed. ".

Why would you use a live API key as a stub? and also the requirement clearly states that you cannot use the API key as a service token and you missed the basic requirement in the whole of testing. The scripts should have marked the QA build as fail, from the first test itself as it was using apikey to validate the quote.

This is where sanity testing and entry criteria come into effect if the build doesn't satisfy the basic requirement then it is not qualified for testing. Before you could move to test other features, you should always verify that the most important features are working as expected. Discuss with the team and create an entry and exit criteria for your tests.

My question is how can I detect such bugs in local BDDs? Writing stubs to such fine details is a gargantuan and difficult task.

The main idea of BDD is a collaboration between different teams and stakeholders to understand the business process and customer mindset.

So the best thing you could do is to talk to the QA that found out the issue in the integration phase. We QAs requires independents when it comes to testing but not in knowledge sharing. Whenever someone else detects some issues in your code ( Or testing approach), appreciate them for their efforts and sit together with them and understand what made them think from that perspective. Find out the ways in which they identified the bug.

Three amigos are one of the backbones of BDD ( where BA, QA, and PO, collaborate together to identify BDD scenarios) but these three amigos are not limited to these three roles. You can add anyone who could help in improving the quality.

So my suggestion would be to talk to the tester who identified the issue and cover that scenario, and as you know sometimes exhaustive testing is not possible and don't blame yourself for a missed bug but use it as learning opportunity.

Answer 2

The pattern you are looking for is a Spy. It's a test double that registers info on the interactions performed during the checking process, allowing you to verify them as well.

Generally speaking, you may verify your boundary object behavior with the external boundary like this:

@Test
public void WhenAskToInvalidateQuote_TheResponseIsPassedAlong() {
  QuoteAPI quoteAPI = mock(QuoteAPI.class);

  QuoteService service = new QuoteService(quoteAPI);
  QuoteResponse response = service.invalidate();

  verify(response.body).equals(blablabla")
  verify(bar, times(1)).setupToken(is(ServiceToken.class));
}

Most unit checking frameworks have Spying features out-of-the-box, but you can create your own for most of the cases with little overhead.