2

The solution in question is a financial transaction based platform made up of a number of Microservices. The question relates to the testing performed at each stage of the release pipeline.

We currently have a Dev env where devs code and unit test their services, then we have an integration env where we perform regression testing (which are system integration tests) and the same for new features. Then we have a staging env which is as ISO-prod as possible where we currently do not do any testing and then Prod where some Critical E2E tests are carried out.

The question is what testing should be carried out in integration env and which in staging?

Some additional context:
We have two main testing phases, in-sprint where we are supposed to test user stories (one feature can be spread across several teams as tasks to complete a user story) and then release testing once a feature is complete and the artificers from each Microservices have been assembled together.. any insight would be much appreciated and I will answer any further questions to assist in answering. Lastly, we are the beginning of our automation project (setting up framework etc)

Improve this question
4
  • 1
    Testing is an investigation activity to answer questions empirically. The question is backward: Given a certain risk, a testing strategy is developed, which includes which techniques, tools, and environments would be used to fulfill it. In summary, you need to first state your context, risks so people can assess how your strategy, tools, and environments fit on them, determining points for improvement.
    – João Farias
    Commented Jun 2, 2020 at 22:42
  • A question about the devs unit tests - do they mock and stub 100% of the external services such as database? If not "unit" tests are actually integrations tests. This is true at a great many companies today.
    – Michael Durrant
    Commented Jun 3, 2020 at 10:03
  • @JoãoFarias I think you should credit James Bach for your quotes. Even though James has a point in theory, and even though you should know the risks involved, practically you simply can't afford to dismantle a big system to every possible risk and then work backwords to find ways to test the risks.
    – Rsf
    Commented Jun 3, 2020 at 13:17
  • It's a rewording of some main RST concepts ;) But risk-based testing precedes RST in many years also. I didn't say to him to start from scratch now and all the time, I said he should first explain the risks and then explain how is the current strategy, then people can advice of how to stir the ship. What he's done was to skip the base reasons and say "here is what I have, what testing can I do?". He missed the "testing for what?" part.
    – João Farias
    Commented Jun 3, 2020 at 13:39

1 Answer 1

Reset to default
2

Given 4 following 4 environments: dev, qa, stage and prod I would use the following guide of what to run where.

dev

  • Unit tests - fully mocked and stubbed

qa

  • Unit tests - fully mocked and stubbed
  • API Tests
  • Integrated Tests - actual dependencies

stage

  • Unit tests - fully mocked and stubbed
  • API Tests
  • Integrated Tests - actual dependencies
  • Device Tests

prod

  • Smoke Tests (subset of API and UI)

You also need to have hard talks about the people doing the work. "where we are supposed to test" is a common phrase, but one that indicates you are not empowered to test what you think is the right thing to test, but instead are just following instructions from others. That is not agile.

I recommend focusing on what will be the correct Agile Testing Pyramid for your company. This means incredibly hard conversations with the business about why testing everything through the UI (how a non developer think of 'testing') is not a long term strategy and leads to ever slowing and more brittle test suites. The best argument to make to the business is that under this process things will get slower and slower and slower. Most business today recognize that they need to go faster, faster, faster or their competition will and they will be out of business.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.