What should be unit tested in an API client?

Question

I've been reading about unit testing day in, day out for a few days and the more I did, the more frustrated I became.

The System Under Test is essentially a client calling external APIs that take XML parameters and generate documents. It's legacy code and I'm trying to write tests for it.

A simplified version of this class would look similar to this

class DocumentService
  def create_document(item, buyer, seller, other)
    payload_hash = {}
    # 1. format API request payload using item, buyer, seller, other
    payload["abcd"] = item.abcd;
    ...
    payload = Gyoku.xml(payload_hash)

    # 2. call external API
    response = RestClient.post(@@external_url, payload)

    # 3. handle the response
    case response.code
    when 200
      # some processing logic here
      return GeneratedDocument.new(
        document_id: document_id,
        document_pdf: document_pdf,
      )
    else
      fail "Failed"
    end
  end
end

The First part generates the payload.

1. I'm not sure whether/how to test if the correct payload is generated from the parameters passed(item, buyer, seller).
   I've thought about extracting this into generate_payload(item, buyer, seller) function and passing Fake item, buyer, seller objects, but doesn't that amount to testing a private function?
   Where should this effect be sensed and tested?

The Second part makes a POST request using the payload.

2. I guess, if I Fake the post request call on the Second part, I could test the payload from there. Something like this:

class DocumentService
  def create_document(item, buyer, seller, other)
  ...
  response = call(@@external_url, payload)
  ...
  end

  def call(url, payload)
    RestClient.post(url, payload)
  end
end

class FakeDocumentService < DocumentService
  @payloads = []  
  def call(url, payload)
    # do nothing
    @payloads.push(payload)
  end
end

def test_payload
  fakeDocumentService = FakeDocumentService.new
  # instantiate fake objects
  fakeDocumentService.create_document(fakeItem, fakeBuyer, fakeSeller, fakeOther)
  expect(fakeDocumentService.@payloads[0]).to eq (expected_payload)
end

However, this feels like an indirect way to test the payload logic.

A bigger issue is that, since the call is a command (as in command query separation), I feel like the call needs to be Mocked than Faked to be tested appropriately.

Should and can I mix Fake and Mock by testing the payload generation part using a Fake and testing if correct endpoints are called by Mocking? (A popular article (https://martinfowler.com/articles/mocksArentStubs.html) seems to suggest that you are either a Classicist or a Mocist and you need to choose to either Mock or Fake, which frustrates me more.)

The Third part processes the result from the external API.

3. How should I test this? Should I Stub the API call to return different types of responses and test how they're handled? If so, I'll be Faking the external call to test the First part, Mocking it to test the Second part, and Stubbing it to test the Third part. Am I doing this right?

4. Am I trying to test too much? I saw how Stripe tests their Ruby client (e.g. https://github.com/stripe/stripe-ruby/blob/master/test/stripe/customer_test.rb) and it looks like they are only testing whether a request to the correct url is made. This may be the payloads to their calls is not as lengthy and complicated as ours, and they are making calls to their own service, whereas our code makes calls to APIs of other companies.

I have so many other questions but these are some of the big questions.

Answer 1

Should and can I mix Fake and Mock by testing the payload generation part using a Fake and testing if correct endpoints are called by Mocking?

Yes, you can freely mix Fakes and Mocks. In fact, a Mock behaves exactly like a Fake with an extra layer of verification added on top (the verification of the calls being made to the Test Double).

(A popular article (https://martinfowler.com/articles/mocksArentStubs.html) seems to suggest that you are either a Classicist or a Mocist and you need to choose to either Mock or Fake, which frustrates me more.)

You may have misunderstood that article. It is not true that only Mocists can use Mocks. Both Classicist and Mocist testers can use Mocks. The main difference is in what they prefer to use when the Unit-under-test (UUT) has a dependency.

• A Mocist will say: "I see a dependency that is involved in the verification part of my tests, so I will substitute that with a Mock."
• A Classicist will say: "Can I test everything easily and fast with the real object in place? If so, I will use the real object. Otherwise, I will use a Test Double (possibly a Mock) to make my life easier."

Am I trying to test too much?

Most likely, yes.

• Verification that the right payload is being sent over the API can best be done in an integration test. A unit test does not add that much value, because there is no easy way to prove that the expectation expressed in the unit test are correct (except by effectively running an integration test and monitoring the traffic).
• Verification that a POST request is made to the correct end-point is something that can be valuable to test in a unit test, but it depends a bit on where you can have your Mock hook into the UUT. Preferably, you would mock the RestClient object, because that allows you to test the complete code as it would run in the production environment. If you have to create a FakeDocumentService that replaces the call to RestClient, then the value of the unit test is much less.
• The third part (processing the response) is something I would definitely test in a unit test. You have full control over the responses that will be received, so you can verify what happens when an error is returned or that a malformed response doesn't cause a crash.