We are starting a reporting project that integrates Looker and Snowflake to gather data from multiple micro services. We were wondering if anyone is doing something similar and how do / did you create automation / integration tests around this?
Currently we use postman for service level integration and write automation tests in C# for worker-related integration.
I believe this "reporting service" will be written in a worker that utilizes SNS and sqs topics to get data transferred to Snowflake.
Although the question is asking specifically for technologies, maybe generic tips may help:
1 - Toby Clemson's article on microservices testing can be a good start point: https://martinfowler.com/articles/microservice-testing/
2 - A tool that you may want to introduce is Pact. It performs Contract Testing (mentioned on Clemson's article). The basic idea is to validate if the individual services tests, where dependencies are mocked, are still valid. With it, teams can work independently and validate which services would be affected by their change, without having to setup an whole environment and interact through a single point (End-to-end tests).
The point of microservices is to break the system in independent components. Therefore, testing them should be performed independently as well. Rely mostly on end-to-end tests would create slow feedback cycles and hard to debug failures.
