1

Our system is publishing price update to an external system through HTTP and listens to a queue for the processing results. Based on the results, different messages will be shown to the users.

We are implementing E2E test cases now. Obviously, we do not have the control of the external system. How can we implement at least one happy case and one fail case?

  • 1
    We are considering mocking the external dependencies. Technically it is easy to implement. But we are not sure whether it is a good practice for E2E. We have searched in a few places, but answers are different. – Hammer Nov 30 '17 at 3:30
  • 1
    Why do you think it can be bad practice in your case ? obviously at one point or another you will need to test your system end to end, but what do you think is wrong with mocking ? – Rsf Nov 30 '17 at 10:07
1

Testing integration with real system (end-to-end setup) offers you the biggest credibility but it has a number of limitations:

  • external system might not be available at the time you execute your tests
  • external system is a shared resource, so your end-to-end tests are more fragile to changes in the external system, e.g., tests can slow down or become unstable
  • also your tests might impact other users of the external system
  • it might be hard to simulate certain responses of an external system, e.g., trigger error responses

An alternative, as you said, is to mock external system. This gives you more control, for instance, it is easier to simulate certain responses of the system. However, mocks are always less credible than real systems. They also need to be maintained: kept up-to-date with an external system. If an external system changes its API/contract, e.g., adds a field in a response, you need to add this field in a mock response as well. There are two possible solutions to keep your integration tests more realistic and up-to-date.

One is to record traffic from your external system in production environment and playback in your tests. And repeat this record-playback every time either your system or external system is updated.

Another is to implement contract-based testing: authors of external system provide you with tests your system needs to pass to satisfy the contract with their system and you provide them with tests their system needs to satisfy to cooperate properly with your system. This obviously requires more involvement from both teams.

  • 1
    Thanks. We actually investigated pact-js for contract-based testing. However, some http based can use, but event based like rabbit MQ call-back has not been supported... – Hammer Dec 5 '17 at 7:07
1

For End-to-End testing, no, it's not great, but you may not have other options. If that's the case, as the test team, you need to make sure that the powers that be are aware of the risk that's being taken by mocking versus true end-to-end testing. They also need to realize that the mocking isn't free; you're going to have to maintain and update the mocking tool. You might want to give them classes of problems (performance, serialization, etc) that you won't be able to properly test.

With that said, though, there might be other options. Some companies that provide API services also provide some sort of test interface as well; instead of sending requests to http.company.com, you'd point to test.http.company.com. And you listen to test.queuemanager.company.com, not queue manager.company.com.

Another option, unpleasant as it can be, is time-based. Is there a time during the week/month/quarter that the overall service isn't needed? If so, you can always do your end-to-end testing then. Just be aware that this usually tends to happen from, say, midnight-8 AM Sunday, or something similar. You could still do most of your testing with mocking, and then have a final checkout with the real system. Some companies will deliberately schedule downtime for systems to allow this sort of testing to be done. But this requires a great deal of planning and coordination; there's no "Just one more test . . ." opportunity.

And, as @dzieciou says, record/playback is also an option, though it depends on what sort of changes you're testing.

And to answer your specific question: happy path testing with mocking is easy. It's testing failures that can be hard, because there are many, many different types of failures. And with End-to-End testing, it's often the failure states you need to focus on. You don't need 1 happy path test and 1 sad path test, you need 1 happy path test and many sad path tests.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.