What to test and how
- You definitely need to test full flows of the system, to see whether services understand each other, in particular, when one of service responds with error messages. This is a place where I usually find problems. Mocks will definitely help here.
- Also, if you're working with queues, then many there might asynchronous events and time-outs that may come into play. You can try simulating each such scenario individually but performance tests when many concurrent threads are loading one of queues with unexpected number of messages can reveal interesting bugs.
- Many scenarios are easier to automate at unit testing level, where you do not need to start an HTTP server to host a service or service mock. This is particularly the case for negative scenarios. I usually work closely with developers of such services and every time I find automating a scenario is to expensive at system level, I ask developers to add a test to their unit testing suite.
- Martin Fowler identifies multiply levels of testing for microservices: unit tests, integration tests, contract tests, and end-to-end tests. However, make sure you're really having microservice architecture there. I've seen many microservices ending as distributed monolith, sharing code or even sharing databases. This problem has been described in InfoQ.
How to organize your repository
It is not only matter of your test strategy but also how you work as a team.
- Are microservices owned by different teams? Is the code for those microservices in the same repo?
- Who is going to maintain and update those tests? Only you, other testers or also developers of microservices?
I usually prefer to have tests close to the code they test, so if anyone new comes to the project, she can start automation with one or two code checkouts.
- How will you launch those tests in a development pipeline? Do you want to have one single status whether all tests passing or failing, or you prefer to launch first tests for single services and only if they pass launch end-to-end tests?
You do not need separate repos for each group of tests. Packages enable organizing structure of tests in a way you wish, so it is easy to grasp test coverage looking at your packages names and structure. Then there are test groups that can be used to annotate your tests. They will ease launching separate groups of tests, especially if you want to have separate jobs in your CI server (e.g., Jenkins).
- Whether to have a separate repository for service stubs, mocks and other helping tools is another question. Are you going to use those tools across other testing projects as well? Are other people going to use those tools outside of your project?
I usually prefer to start small, with one repo, well structured. Creating many repositories from the beginning makes maintenance unnesessary complex. When the repository starts growing, you will need to refactor it from time to time, update the structure. If refactoring is not enough then you can think whether to move some tools or tests to separate repos.