6

Scenario: I want to test a new functionality in the system "login and verification". I assume that developers have done the unit and integration test, and I am just doing a system test

Do companies (that want to do things right) tend to separate tests for API using one tool (like soapui, that will later let you do performance tests) and then write separate tests in another tool like Selenium for the web interface? Or do people tend to do just the web testing automation because if there is a problem, the tests will show it anyway?

I particularly want to know why companies would have both API tests and web tests when this causes duplication of effort.

Improve this question
3
  • I have edited your question to make it less opinion-based. As always, you are welcome to revert if you disagree.
    – Kate Paulk
    Commented Apr 26, 2017 at 11:22
  • Thanks, Kate! It was my first contribution to the site, so I appreciate the feedback
    – Declamo
    Commented Apr 26, 2017 at 18:14
  • Another simple reason might be if the API services more than just the one website. If you only use selenium testing, you only prove that THAT website can use the data from the API, where if you only do API testing, you only prove that the API can provide data, not that the website could use it.
    – Cronax
    Commented May 5, 2017 at 7:56

4 Answers 4

Reset to default
5

Yes, this is typically done in a commercial context, but it does not need to be exactly as described.

As you already noticed, developers should write unit tests. Those tests as you probably know only test the smallest distinguishable things in the code.

Higher up there should be integration tests - either as an extension to the unit tests or maybe in the form of testing the API. If the integration tests directly call the APIs, I would assume that they aren't written by a tester, but by a developer. If the tester is focused on testing the functionality, the APIs are not his main concern. If the tester has a more technical role, then they could be one of his responsibilities.

You as an (automated) tester write automation web tests via Selenium. That should be the highest level in terms of testing, as it tests all features combined. These tests will call a lot of APIs, but not through the APIs directly, but through the web interface. As that is another layer on top of the APIs, this is not the best place to specifically test the APIs.

The advantages of having three levels of automated testing is that you are able to test on each level specifically what functionality is added. The unit tests tests each function, the API tests test whether the APIs work as expected, and the Selenium tests are mimicking the behaviour of a real user. Without the API tests you are tempted to write more Selenium tests to cover the APIs as well, with the disadvantage that the Selenium tests typically require more maintenance and more time to run.

Another advantage is that API tests are more easily setup as performance tests: running the API tests on 100 machines with 100 calls per second is much easier than doing the same thing with Selenium tests.

Concluding: in my opinion having three levels of automated tests has a lot of advantages. Of course, it implies duplication of effort, but I do believe that it is worth the effort. It is also possible to have a more technical person (maybe even a developer) write the API tests, so that you as a tester already have a better feeling of the quality before you even look at it.

Improve this answer
0
4

why companies would have both API tests and web tests when this causes duplication of effort.

In short, because those tests are different in nature, help discover different types of bugs and have different properties.

End-to-end tests, e.g., tests operating on UI level of a Web application have two main advantages:

  • they simulate a real user: this helps managers and decision-makers easily determine how a failing test would impact the user.
  • they help discover bugs not discoverable at API level: differences in layout and JavaScript interpretation across different Web browsers, HTTP integration problems, e.g., Javascript application may not be able to understand a response from the API.

However, they suffer from a number of issues. They:

  • are long, so getting feedback whether your change caused a problem is delayed
  • are less reliable, they tend to ocassionaly fail more often, not because of regression in the system but because of a bug in test
  • make isolation of failures harder, because they touch more elements: both frontend and backend.

On the contrary, API tests are usually:

  • faster,
  • more reliable,
  • make isolation of failures easier
  • and help discover bugs that are not discoverable at UI level, e.g., it's possible to manipulate HTTP requests in API tests even if UI of the tested system does not allow for that.

So, when building a test suite for a login page I would choose different types of tests depending on what you want to stress:

  1. Login page UI makes AJAX calls to backend API with username and password. In case of successful answer from the API it redirects a user to the home page, otherwise it shows an error message. Testing integration and redirects is a good candidate for an end-to-end test.
  2. Login page UI constrains the maximum length of username you can enter, however a developer could forget to validate username length also at the backend level. API tests can help discover this bug.
  3. Part of the login page includes support for password reset, and when the user provides a new password, JavaScript is checking in real time whether the new password satisfies password policy requirements, e.g., minimum length. It makes more sense to test this part not even in end-to-end tests but in JavaScript unit tests (where backend is mocked) on different browsers.

When choosing how many tests I would have, I would follow a testing pyramid:

Testing Pyramid

  • a lot of unit tests covering many edges cases, both for backend and JavaScript parts (if necessary)
  • few end-to-end tests
  • a medium number of integration and API tests

You can find more information on that in the following articles:

Improve this answer
0
1

I advise our teams to write mainly unit-tests, but at least ONE test on service (e.g. API) and UI (e.g. end-to-end) levels of the test-pyramid. Just a happy path test is sufficient.

Why do I advice that? The main reason is to make sure that each level is build testable.

If the levels are testable, then when a defect occurs and after it is fixed, it should be easy to add a test to prevent that defect from re-occurring. If for some reason that defect is hard to test at unit-level. Having no tests on other levels might make it easy to take a short-cut and not test it at all.

Duplicate test coverage for other reasons seems a waste of time. Extra coverage costs just extra time in maintaining it.

Improve this answer
-1

Yes, it is a common practice to have multiple levels of test, even if there might be some overlap. Unit tests, API tests, and UI driven tests.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.