How do we as testers best approach API test cases without entering the realm of insanity?

Question

I'm looking for some clarity on how to approach API testing holistically, I have a project to automate a Restful web service, containing 2 controllers(foo & bar) with a collective sum of around ~30 calls available.

Some implications - There are no test cases written for this, so I'm not coming along and automating what is already existing, I have to write those cases myself.

I have added integration capabilities into my test framework already and did all the heavy lifting, so now I'm just left with my internal service which takes an object, parses it to json and using rest assured makes the calls, returing me the response, here is a small example:

@Test
@Step
@Severity(SeverityLevel.CRITICAL)
@TmsLink("19793")
@DisplayName("SaveCourse: User can add a new course")
public void savecourse_user_can_add_a_new_course_successfully_returns_200() {    
    log.info("-- Test: Successfully adding a new course --");
    Course c = new Course(0, data.randomString(20), data.randomString(20), 0, 1, false);
    Response response = ets.saveCourse(c, statusOK);
    resultModel = new GeneralResultModel(false, ets.getCourseSuccessfullySaved(), response.path("Result").toString(), null);
    GeneralResultModel result = response.as((GeneralResultModel.class));
    assertThat(resultModel).isEqualTo(result);      
}

Lets consider the saveCourse takes a course object(JSON) in the request and lets assume it contains the following information:

id (long)
name (String)
description (string)
locationId (long)
iconId (long)

Payload example:

{
    "id": 0,
    "Name": "abc;",
    "Description": "aa",
    "locationId ": 0,
    "iconId ": 1
}

When crafting JSON requests for this call for the controller to handle, how do we best approach this for test case derivation that isn't bordering on insanity? I could think of hundreds of combinations and invalid data types, sql injection, xss, empty, duplicates etc! the list could go on forever here.

How do we, as testers best approach this to cover the API to a good level while maintaining its security etc. Worth noting that this API is a private API and just used with a front end in a micro service setup.

Thank you for your time.

Answer 1

Most of the question (and answer) isn't specific to API testing; combinatorial explosion is a problem whenever you have a function that takes multiple inputs.

So, first, you'd need to understand how the API interacts with the backend service. Is it actually doing anything, or is it just passing a call on to some other program? If so, was any testing of that program done? If the other program has been tested, you may want to take the risk of doing minimal testing of the API, and just verify that the data that you get back from the program is the same as you get from the API call. If the API call is doing compositing or the like, or the function hasn't been tested before, you can't take this approach. Also, note that exposing an internal function to the outside world may change how, or how often, it gets used.

If you've decided you do need to test the API call, then you'd test it like you'd test any function with many inputs. You'd want to test limits and null values for each input, and some expected good inputs as well. You'd want to specifically test any documented requirements (ie, input X should always return Y, or always throw an exception). And then you have to worry about the combinatorial explosion issue; for that, I'd first point you here: Should A Formal Test Plan For Dynamic String Testing Include All Combinations?. Then, I'd suggest looking into combinatorial test design if you feel the need to go further. But realize that the more inputs that are required to trigger a bug, the less likely it is for that bug to be hit.

Beyond that, you also may need to worry about different access patterns/response time requirements than the function itself had. You probably should do some amount of performance testing, to find bottlenecks. You may want to do security testing, but I'd argue that that's better for specialized security testers to do. How far to go in any of these areas depends on how important the REST API is, what data it's protecting, and so on.

Answer 2

I would suggest to do the following:

• schema verification of the response body (specially when the schema is being changed a lot or the routes are new); along with this you may want to test the requiredness of the fields you have for your payload

• the expected response body & status verification (of course)

• from my perspective, its good to cover positive tests first then add negative test scenarios (such as 400s, 401s, 404s, etc.). also ensure that each controllers have one file each