5

I have a single service that acts as "middle-man"/gateway between a few internal services and multiple external complex APIs, over 20 to 30+ different ones.

I want to direct my testing efforts in such a way to for example, with some level of automated checking, have some logical validation between my gateway's way of consuming each of the external providers, in order to detect earlier the symptoms of issues where I have the implementation of the consumption of one provider sometimes breaking the consumption of another one.

The restraints of the problem are that I have zero control of the development process, that is itself somewhat chaotic, and I can hardly enforce or let alone promote, for example, proper unit checking of the implementations of the consumer, plus code quality standards, and overall API documentation.

I've thought about attacking this problem with some form of contract testing, where I define the model of external providers, taking into account their public apis and their documentation, and worry only about the implementation consumer-side part of testing.

The thing is, this sort of approach, from my understanding, seems to be "not advised" by people who work on contract testing libraries, for example:

(Not recommended) Testing providers where the consumer and provider teams do not have good communication channels. (Source https://docs.pact.io/getting_started/what_is_pact_good_for#what-is-it-not-good-for )

I've had a look at question "Testing interactions with external systems" but it's not exactly what I'm looking for.

Any ideas are welcome.

EDIT: I'll give some more example details to help frame a possible problem scenario:

  • I have full access to the code that implements this consumer for all the external APIs;
  • I have access to test environments of both my service and the external provider services;
  • I have the ability to raise my service on a local level, but not the external services themselves;
  • I have access to some documentation and postman collections of some of the external apis;
  • Implementation of the feature "consuming external service A", sometimes jams/breaks "consumption of another service B", or even disrupts other unrelated parts of "my gateway service";
  • Implementation of "my gateway service" is bound to an edge case scenario: weak development process, tightly coupled and extensive code, inexistent unit checks, low code quality checks, and overall handicapped/rushed development and requirements gathering process.
2

Let's start from the bottom line- unless you revolutionize everything there will be no magic solutions.

Martin Fowler describes here the most common approach- test against a simplified copy of the external services, and occasionally verify that those copies behaves the same as the source.

The problem with this approach, beyond the obvious of indiscrepancies between the two APIs, is that it mainly tests the API and some basic logic but not the overall behavior like timing, multi-processing of requests or (legal) changes in syntax of things like json's.

You could try to explain and convince the teams from both sides to freeze their code before a release and then have a full integration test cycle, or at least freeze one part at a time and test against it.

  • 1
    True, there are no magic solutions in this case. I see some other problems with that approach, 1) the time invested on creating a simplified copy increases the cost of testing "the problem"; 2) we'll always fail to test against a truthful behaviour of the external API, since the simplified version we build is based on our observation of that API, and not a deep knowledge of its internal behaviours. The convincing of one side or both to code-freeze also seems impossible taking into account the mindset of both involved parties: "rush, move stuff, break stuff". Still, thank you for your answer. – Filipe Freire Sep 7 '18 at 7:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.