3

We are using a code pipeline for our CI-CD.

Front-end and back-end are separate repositories.

We run cypress end-to-end tests on the deployment of both frontend and backend.

The code gets merged from feature branch -> dev -> staging -> prod and starts respective pipelines.

I am looking to integrate contract testing. We already have swagger documentation generated/stored when the API gateway is deployed.

I am thinking that to simulate consumer-driven contract testing, at the end of successful execution of end 2 end tests on the deployment of either frontend or backend, I will get a download of the swagger JSON doc from API gateway and store it in an s3 bucket so that I can use it as a reference to detect breaking changes on dev/staging stage.

  1. Should I be comparing dev and staging to prod to detect breaking changes? I don't like to idea of referring to JSON from an s3 bucket in the prod account while deploying to dev or staging.
  2. How can I refine this process?
  3. What are my options?

I know everyone seems to be using pact for contract testing but I just think it's overkill when we already have the swagger API docs.

3
  • 1
    Swagger and Pact have different goals, they are orthogonal. I suggest reading this topic: docs.pact.io/faq/convinceme/#but-i-use-swaggeropenapi Oct 28, 2021 at 8:34
  • 2
    Could you expand a bit on the high-level problem you're trying to solve? I think I can maybe guess at it (wanting to detect integration problems earlier than the E2E tests run?) but having you describe the context (including if there are multiple separate development teams in play, and how they communicate today) could perhaps be helpful
    – grunet
    Nov 2, 2021 at 2:54
  • 1
    If you want to enforce Swagger / OpenAPI specifications themselves as a Contract to identify breaking changes then take a look at Contract Driven Development with Specmatic (specmatic.in). Here is a short explainer video (youtube.com/…). Full Disclosure - I am core contributor on Specmatic Nov 8, 2022 at 14:31

1 Answer 1

1

As per my understanding you are trying to leverage your Swagger / OpenAPI specifications to detect breaking changes. Contract Driven Development should help you achieve this.

Here are some recommendations to refine your process

  • Generating Swagger / OpenAPI Specifications
    • Generated API Specifications may not always be inline with your implementation. Example: If you are leveraging Swagger Annotations to generate API Specifications there is a possibility that we can miss some annotations, if you are capturing traffic to generate API specification again depends on variety of traffic, etc. In short generated Specs may not be completely reliable. I suggest leveraging the generated spec may be as a starting point and then thoroughly review it.
    • Run your API Specification as Contract as Test on your API to check if it is actually adhering to the specification before sharing it with consumers
  • Sharing OpenAPI specifications with stakeholders
    • S3 bucket may not be a good fit for sharing API Specifications with Consumers
    • Internal Consumers - API Specifications are code and is better stored in a version control system such as Git. However keeping it in multiple different repos can also make it hard for collaboration. Store your specifications in a Central Contract Repository
    • External Consumers - Generate a release artefact from your central contract repository and publish your API Specifications through API Gateway or other standard approaches
  • About your question "Should I be comparing dev and staging to prod to detect breaking changes?" - IMHO waiting for production deployment of your provider to an API Gateway which in turn generates the API Specification is slow feedback cycle.
    • It is better to start with the API Specification instead of depending on the Provider to generate it.
    • Here is a video about how you can leverage OpenAPI Specs to detect breaking changes on your local and CI itself instead of waiting to deploy to higher environments.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.