Difference in Web Services & REST/SOAP API from Black-box Tester perspective

Question

I have practically tested web-services/APIs (testing back-end requests of mobile apps, JMeter scripting for a web-application) but when it comes to differentiating between web-services/APIs(REST/SOAP) my concepts are quite vague on it.

As a black-box tester, all these look same to me. All of them take some parameters which we send in a request and then verify the response for it.

I want to know..

• if there are any differences in these technologies from black-box testing perspective. If Yes, then what are the differences?

• And what level of understanding is really required by a tester (specifically a black-box tester) who is directly testing these technologies.

PS: There are many search results for this topic on google but they all focus on listing the technical differences which I feel are relevant to developer. My intent here is to understand these technologies and their differences from a Tester's perspective.

Answer 1

I want to know if there are any differences in these technologies from black-box testing perspective. If Yes, then what are the differences?

• Serializing/deserializing messages. SOAP Web services define message format through WSDL standard, so it is easy to automatically generate client stubs from it. There's a similar standard, WADL, for RESTful services, but is not widespread, so usually you need to take care of serializing/deserializing yourself or rely on clients provided by service providers, dev team, etc.
• Verifying response compliance to the schema. For SOAP you can validate whether a SOAP response payload complies to the XSD schema. For RESTful services, as said before, usually there's no such schema, so validating of the structure must be done in a different way.
• Transport protocols. SOAP Web services can use different bindings as a transport protocol, HTTP, IIOP and other, so you may need to test it as well, because often same service exposes different functionalities under different bindings. RESTful services goes always over HTTP.
• Architecture validation. RESTful services rely on the grammar of HTTP protocol, so part of verification might be whether methods like POST, PUT, GET, etc. are used properly, or whether HTTP response status codes (200, 4xx, etc.) are used for the right errors. WSDL for SOAP Web Services leaves more freedom here. There are also many other concepts in REST style, like stateless requests (no cookies) that can be validated.
• Security. RESTful services rely on HTTPS for secure transport layer and may rely on HTTP Basic Authentication. SOAP Web services additionally may rely on WS-Security extension.
• Verifying documentation. If you care about documentation, then first thing for SOAP Web services is read docs in WSDL and XSD files. RESTful services do not have a fixed way of documentation.

And there are much more differences between those services, nicely compared in the paper from WWW 2008 conference, "RESTful Web Services vs. “Big” Web Services: Making the Right Architectural Decision".

And what level of understanding is really required by a tester (specifically a black-box tester) for these technologies.

Depends, how those services have been defined in detail, how far you want to go with testing and what matters for you and your company:

• It might be helpful to learn XPath or JSON path, if your services are using XML or JSON for payload. XQuery is optional here. For most functional tests it's enough to stop here.
• It might be helpful to learn how to write or generate client stubs for both types of services in your favourite programming language, if you don't want to use SOAP UI.
• It might be worth to learn how to automatically validate XML against XSD (XML Schema) with your favourite programming language.
• For both it might be worth to learn about basics of HTTP protocol, what most common headers, methods and error codes mean.
• If you care about proper architecture, read about basics of RESTful architectural style.
• If you care about security testing, for both it might be worth to learn about some HTTP related attacks, e.g., HTTP response splitting.

Software for accessing services:

• REST services: for HTTP GET method you can simply use Web browser, for other HTTP methods you may need dedicate software, e.g. Postman add-on for Chrome, curl Unix command or SOAP UI.
• SOAP services: you need dedicated software, e.g., SOAP UI.

You can also generate client stubs, but that depends on your programming language. E.g., for testing REST services in Java I use REST-Assured and Retrofit libraries. For SOAP I used Apache CXF to generate client stubs.

Answer 2

Based on your question I'm assuming:

1. You want to know the distinction between web services, APIs, as well as REST and SOAP
2. Whether the technical implementation of these matter to you when it comes to testing in a black box capacity

So lets start with some definitions:

Web services and APIs:

I always like the W3C description of a web service:

'A Web service is a software system designed to support interoperable machine-to-machine interaction over a network.'

To me this means a Web service is a system somewhere on a network (either internal or on the web) that provides resources or actions to other systems such as a mobile app, a web browser or another Web service.

The API is a means to access the web service. Much like, for example, selenium-webdriver has bunch of methods that form an API to trigger your browser events.

REST and SOAP

So with the above in mind, REST and SOAP are architectural approaches in how a Web Service and it's API is created and how it communicates with other Web Services. I'm not going to go into the differences here, but you can find details on each style here:

• REST - https://en.wikipedia.org/wiki/Representational_state_transfer
• SOAP - https://en.wikipedia.org/wiki/SOAP

Crudely put you can see the differences in REST and SOAP in the requests you make, specifically in the payloads.

So in answer to your second question:

Yes it does matter in a black box capacity. The choice of REST or SOAP architecture matter to you because they determine how you would generate your requests for the service. For example, sending a SOAP request to a REST service will probably result in the web service rejecting it.

Additionally, what tests you design would be determined by what the Web service does and what the API allows. So if it's a tax calculator service you will be interested in how the service crunches numbers whereas if it's a database service you will be interested in how it stores and retrieves data.

I would recommend that although you might not know the code inside and be able to test in a White box capacity it's worth finding out what the service's responsibility is and how it has been implemented to ensure you run effective and relevant testing.

Answer 3

It depends.

If you're testing a GUI that interacts with some back-end service, and your only concern is that what you enter in the GUI gives you the expected outcome when the service is done, it doesn't matter what type of back-end service is used (An example might be testing credit card authorization in your system with a new protocol - you don't care how the protocol is implemented, all you care is that the test card number that should give you a approval actually does give you an approval).

On the other hand, if you're testing some feature or aspect of the service itself, then it matters a great deal: you need to know what format to put the request in, what format the response should take, and you need to be able to read the logs if the response doesn't give you the result you expect. If you're automating, the kind of service has a big impact on how easy/difficult it is to automate in your specific tool.

Answer 4

short answer- No a little longer answer- No there's no significant difference testing wise I think you answered the question yourself

All of them take some parameters which we send in a request and then verify the response for it.

only the mechanism is different