What is the best approach to the REST APIs automation?

Question

I have been working on REST API test automation in JAVA and have many approaches in my mind but not able to decide which one is better. Here are few approaches that I can think of-

1. Very Simple and straight approach where the testing team uses the same code repo on which dev is working, And we add the API test cases just like the unit test case in Test package of the same repo. PROS: less work because the testing team need not create DTOs etc., test cases will be available to the dev team, CONS: every time we add new test cases, we will have to do a pull request to merge the test cases, test cases will require immediate attention in case of dev code change otherwise there would be compilation failure.

2. Using a separate project with the third-party library, for example, REST ASSURED to automate the APIs testing. Here the testing team will import the dev code and use the DTOs, database access objects from the imported dev code. PROS: less work because the testing team need not create DTOs etc. CONS: test cases will require immediate attention in case of dev code change otherwise test cases will give compilation failure.

3. Do not use Dev code at all and write all DTOs and everything and automate the test cases using REST Assured in a separate project. PROS: Not any dependency on DEV code. CONS: The testing team will have to create lots of DTOs for mapping the request and response JSON. This will take too much efforts and time and will be a replication of task which is already done by the dev team.

4. Do not create DTOs and use JSON Schema to create DTOs at runtime and use REST ASSURED in a separate project. PROS: No need to create too many DTOs.Test case management and maintenance would be easy. CONS: Not found yet, please suggest what cons could be of this approach.

5. Or Any other approach which is better than the above approaches. Kindly suggest.

Answer 1

A few thoughts:

• Automation code should be reviewed, so I don't think that's a con
• Not sure why failing test cases would keep your build from compiling. Even if the test code lives in the same repo as the application code, your build scripts should ignore the test code when compiling deliverable code. There might be a failure further down your build chain in a CI system when tests run, but that's separate from blocking the build itself.
• You missed the advantage of having the test code in the same repo as the application code in that it's versioned in unison. That is, if you're disciplined, a checkout of any point in history will work as it's using the DTOs from that era, and the assertions will match. If in the future the DTO changes, the tests will get the new ones, and the checkout will have the tests that have been updated to deal with the new versions of the DTO (i.e. extra assertions for new fields)
• There's another advantage that having the test code in the same language as the application under test means that developers can contribute without knowing another language or framework. Ideally, if they change the DTO, they'll update the tests they break

In my current shop, the APIs are written in Java. Unit and integration tests are in Groovy (so they can use the Java classes), while end-to-end functional tests are in Python. For most of our projects, test code lives in the same repo as the application code. The end-to-end tests have the exact concerns you mention, even though we tend to use POJOs for almost everything - there's a fair amount of time re-implementing what's already been done in Java/Groovy (especially asserting over the shape of request/response objects).

In any case, try something, figure out what works/what doesn't and iterate.

Answer 2

As with any project, it depends :). That said,

1. (1) and (2) require close coordination with development team. Depending on the team structure it may be a good thing or a bad thing :). I also suspect in both cases some code will have same development (unit) tests as your tests. For example, a DTO might have a bug which may not be caught by both the teams.
2. (3) and (4) allows the test team to work as a SILO. I prefer one of these options. Depending on the skill level option (3)'s writing DTOs might not be very time consuming.

I will chose option (4) above option (3) and in specific cases will write DTOs for specific modules - modules that perform some sort of computation comes to mind.

Answer 3

As it was mentioned before, it will boil down to what is your organizational culture, how tightly coupled your Dev and QA teams are in your area etc.

For any Agile project, this is what I recommend for API testing.

• Keep your API test code along with your Dev code base
• Automate preferably using the same language that your developers use
• Automate using toolsets that integrate well with continuous testing pipeline
• Integrate all the API tests with CI pipeline
• Use stubs and service virtualization to test hard to find data

The biggest benefit is that anytime there is a fork / pull request QA has to determine if a test needs to be added / modified. Once you have all of that in the pipeline, you will know the quality of the change as soon as the code is compiled.

Answer 4

A generic framework should be independent of the API's behavior. There is another better approach to automate REST APIs which is using the DataDriven approach. Place all your request building and response validation data that includes endpoint, httpMethod, request body, headers, queryparams etc in an excel sheet to drive your test data. Write your Java could to access this data. In this way, you will not be dependent on Dev's POJO classes and time to test will be drastically lessened.