Introducing contract-based testing

Question

I have just read articles about contract-based testing for verifying contracts between microservices:

• Testing Strategies in a Microservice Architecture
• Consumer-Driven Contracts: A Service Evolution Pattern

In my company our team has built a REST service (A) that is used by other applications (B, C, D, ...) of other teams. I wonder how should we implement this idea. I envision the process like that:

1. We provide other teams access to our test repository.
2. Other teams, let say team of B, provide pull request w tests that test simulate behaviour of their application B against our contract A.
3. We accept their pull request
4. We run tests from other consumers (teams whose applications consume our service) as a part of our build.

My questions:

• Is this correct understanding? Am I missing anything?
• Is there any rule how consumer tests should look like? Should it be a real application B, C, D that calls our service or rather it should be a test that simulate those applications?
• What are the tools supporting contract-based testing?

Answer 1

Here is how I have seen this implemented before:

1. The consumer B produces a "B to A consumer contract file" that contains example requests and responses that it expects from A. B will contain a consumer test that produces this file and checks that B behaves correctly when it receives these expected responses from A.

2. Each time B builds, it publishes the contract file and triggers an "A to B producer test". This test is owned by the producer A. The producer test uses the consumer contract file of B to verify that A behaves as B expects it to. This test is run against the following version combinations:

   • Latest A and Latest B
   • Latest A and Production B
   • Production A and Latest B
   • Production A and Production B

To answer your questions:

1. You are missing that you need to verify more than just Latest A and Latest B if you want to be able to deploy A and B independently. You need to be confident that each time an application is built, it can be safely released, whether it is released first or last into production relative to the application it is a consumer/producer for.

2. Two parts:

   • The consumer test should use the real consumer application with a simulated producer to test that the consumer behaves correctly when the producer behaves as the consumer expects

   • The producer test should use the real producer application with a simulated consumer to test that the producer behaves as the consumer expects.

3. The only tool I can have personally used is pact-jvm

Hope this helps!

Answer 2

To answer your question

Is this correct understanding? Am I missing anything?

The mechanism of sharing API Contracts depends on the contract testing approach you choose.

Consumer Driven Contract Testing

Tools like Pact have a broker to facilitate the collaboration between your provider A and consumer B, C and D.

• Team B, C and D emulate A by setting up a mock server with Pact DSL and run their respective component tests which in turn exercises the mock and generates consumer contract
• This consumer contracts is then pushed to the pact broker as part of your consumer application's CI pipeline
• The provider application A has to now run these consumer contacts (those published by B, C and D to the pact broker) as tests and post the test results back to pact broker
• At this point each application (A, B, C and D) can ask the "Can I deploy?" question to the pact broker. Please refer to the Pact documentation.

Provider Driven Contract Testing

Spring Cloud Contract in the producer contract testing approach follows this technique.

• Team A defines Contact with Spring Cloud Contract DSL and runs the verifier to make sure it is adhering to the contract it defined. In the process it generates stub jar files that can emulate it.
• Team A now uploads these stubs to a artifact manager such as nexus, artifactory, etc.
• Teams B, C and D can now leverage the stub jars in their respecting component testing to emulate A. As long as B, C and D are able to work with this stub jar they can be confident that they will work with the real application A.

Bi-Directional Contract Testing

PactFlow supports this approach where both consumer and provider have their own contracts.

Contract Driven Development

Specmatic leverages API Specifications such as OpenAPI as the API Contract. (Disclosure: I am lead dev and CTO at Specmatic)

• Team A and Teams B, C and D can collaborate over the API Design and author the OpenAPI Specification
• The API Specification is maintained in source code repo and any changes are made through a pull request mechanism so that all stakeholders can participate
• Consumers and Providers can now refer to this OpenAPI Specification in the Git repo to independently build their respective applications in parallel.
  • Consumers B, C and D - Run API Specification as API Mock
  • Provider A - Run API Specification as Test

Is there any rule how consumer tests should look like? Should it be a real application B, C, D that calls our service or rather it should be a test that simulate those applications?

Here is a writeup on "Anatomy of Component Test"

What are the tools supporting contract-based testing?

Here is a list of tools that I have worked with.

• Pact
• PactFlow
• Spring Cloud Contract
• Specmatic