2

We're about to implement API tests (in a .Net environment) for our new projects. I have no prior experience with API testing.

After checking out multiple frameworks and tools, it seems they all focus on (de)serializing to/from objects, and validate the values of those objects.

However, this seems like a lot of code for writing simple tests. I have found it much easier to compare the actual response JSON to an expected JSON (via deepEquals) without converting them to objects first.

Can someone offer some insights in the advantages of deserialization? (Or why using literal compares might not be optimal?) I don't quite see the use of having a class for every endpoint if I already now the exact JSON to expect.

Improve this question

3 Answers 3

Reset to default
1

You would deserialize the JSON string to a JSON object so that you can extract certain values in the object easily. This allows you to interact with object and validate its properties rather than the whole. This is especially nice when the test data is not static. You can Assert that the email address was updated and in the error message print out just the email address. Like Assert.AreEqual("[email protected]", account.Email);

It is not really a lot of work. If you have the JSON string you can copy+paste it into some online tools (like http://json2csharp.com/) that will generate the object Type for you.

Here is an example of deserialization using Json.NET:

Type:

public class Account
{
    public string Email { get; set; }
    public bool Active { get; set; }
    public DateTime CreatedDate { get; set; }
    public IList<string> Roles { get; set; }
}

Usage:

string json = @"{
  'Email': '[email protected]',
  'Active': true,
  'CreatedDate': '2013-01-20T00:00:00Z',
  'Roles': [
    'User',
    'Admin'
  ]
}";

Account account = JsonConvert.DeserializeObject<Account>(json);

Console.WriteLine(account.Email);
// [email protected]
Improve this answer
5
  • Good answer but I'm not quite convinced. With a DeepEquals you need to manually check the difference (or not if you write some recursive helper methods) but that's still (imo) worth it if that saves me a ton of code by not having one or more classes for every endpoint (and not deserializing to them).
    – FDM
    Commented Mar 24, 2017 at 6:47
  • If your test data is static, then you're probably fine just doing sting comparisons.
    – kirbycope
    Commented Mar 24, 2017 at 15:49
  • My data is not static at all. I just prepare them directly in Json which is very easy with copy pasting from swagger, for example.
    – FDM
    Commented Mar 24, 2017 at 16:49
  • The test data is static in that you can copy+paste and expect the strings to match with each test run. A more dynamic approach would be doing a POST (getting a new record ID), GET, PUT, then DELETE within the test code. I have an API test suite that checks endpoints for our different vendors and no way am I doing a copy+paste for all of them. I use Data Contracts and assert on a few properties.
    – kirbycope
    Commented Mar 24, 2017 at 17:30
  • Fair enough and a valid setup, but I still think my approach is as dynamic as yours. You have to prepare the data as well, and I do retrieve unique values (such as accessToken) to create a flow of endpoint calls. Our datacontracts are checked implicitly with DeepEquals.
    – FDM
    Commented Mar 25, 2017 at 18:11
1

Maybe you want to consider these general points when you want to use deepEquals:

  • Writing such tests and assertions are easy at the beginning, but hard to maintain in long term.
  • Does the API under test always return the same result? If not then perhaps you have to update the expected JSON manually.
  • Does the API return the response always in the same order? What if your deep-equal fails when the order is not the same?
  • Still it makes sense to test if 2 JSON matches exactly, but usually we put these kind of test under "API Contract Tests" + "API Schema validation". If I am not wrong, this is not what exactly you want to test.
Improve this answer
6
  • Thanks for your answer. Let me respond briefly: 1. why hard to maintain? Or harder than deserialized tests? 2. Deep Equals does not care about order (at the same level) so that's not an issue.
    – FDM
    Commented Jun 26, 2017 at 13:32
  • 3. Why shouldn't the API return the same result for the same call? Whether to deserialize or not, different data will fail the test. 4. It's kind of an API Contract test also, because DeepEquals inherently tests structure as well. So for me it's a benefit (data + structure is tested).
    – FDM
    Commented Jun 26, 2017 at 13:34
  • First of all, I have to emphasise that I am talking about general problem with this approach. Now I can give you one example why this approach can be 1. hard to maintain and also 3. why the return value can be different? "You have an API which returns the date of today". Which approach would yo go with? to generate an expected JSON everyday, or deserialise and do the assertion? I'm sure you can find many more similar examples. 4. Contract testing guarantee that the provider and consumers are committed to their contract (fields, type etc) and has nothing to do with the values of the field.
    – masood ghz
    Commented Jun 26, 2017 at 14:34
  • I understand your example. My solution (like with generated token values) is to set the value to NULL in the Json so all other expected values pass. If needed I can check that extracted value just as if it was deserialized.
    – FDM
    Commented Jun 26, 2017 at 14:41
  • Also fully agree with 4th point but what's the harm if you validate both at once?
    – FDM
    Commented Jun 26, 2017 at 14:42
0

With Karate you can perform a deep equals (and still ignore fields), and there are multiple examples you can refer to.

Disclaimer: am dev.

Yes, Karate runs on the JVM - but since it is based on Cucumber, the scripts are "language neutral". And the JUnit reports should work fine with your existing CI infrastructure.

Improve this answer
2
  • Thanks but Newtonsoft.Json is flexible enough. My question was conceptual rather than technical.
    – FDM
    Commented Mar 24, 2017 at 6:42
  • No worries. My point was that you don't need to serialize to objects at all with the right framework. For example, think how you would do the last line in this example using Newtonsoft.Json
    – Peter Thomas
    Commented Mar 24, 2017 at 8:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.