2

I'm using Jasmine, FrisbyJS, and Jest to create API tests for my team's product. I want to be able to handle when an expect (which I'm pretty sure Frisby is just using the underlying Jasmine expect) fails. I'm not expecting it to fail. I don't want to use a "toThrow", I'm not expecting this call to fail. I just want better error logging when an expect that I expect to pass, fails.

I would like the line of the failing expect, an expanded error message including a relevant stack trace, and potentially the ability to write my own error message attached, like "The call to projects failed". Maybe this is a shortcoming with the Frisby module, in which case, how should I handle this from the Jasmine perspective?

I had this code:

frisby.get(environment.ApiUrl + environment.dashboardProjectsApi)
      .expect('status', 200)
      .done(done);

And I was receiving this very non-descript error that was lacking any further information:

 FAIL  tests\dashboard.api.spec.js
  ● Dashboard api › should return list of projects this user can access

    AssertionError: HTTP status 200 !== 500

      at FrisbySpec.status (node_modules/frisby/src/frisby/expects.js:25:12)
      at FrisbySpec._addExpect.e (node_modules/frisby/src/frisby/spec.js:408:23)
      at then (node_modules/frisby/src/frisby/spec.js:299:26)
      at _fetch._fetch.then (node_modules/frisby/src/frisby/spec.js:231:18)
      at process._tickCallback (internal/process/next_tick.js:109:7)

  Dashboard api
    × should return list of projects this user can access (289ms)

I have a couple of issues with this:

  • I'm getting a stack trace of the expects code in Frisby, not where this code is called in my tests
  • I don't get an expanded message with the error message or stack trace from the API response

And then I made this code, which is better and closer to what I would want to log if any of my expects failed:

frisby.get(environment.ApiUrl + environment.dashboardProjectsApi)
      .expect('status', 200)
      .then(function (res) {
          if (res.status != 200) {
              console.log(res);
          }
      })
      .done(done);

Which emits a much better message to my console.log:

console.log tests\dashboard.api.spec.js:19
    FrisbyResponse {
      _response: 
       Body {
         url: 'https://mywebsite.net/api/dashboard/projects/get',
         status: 500,
         statusText: 'Internal Server Error',
         headers: Headers { _headers: [Object] },
         ok: false,
         body: 
          PassThrough {
            _readableState: [Object],
            readable: false,
            domain: null,
            _events: [Object],
            _eventsCount: 3,
            _maxListeners: undefined,
            _writableState: [Object],
            writable: false,
            allowHalfOpen: true,
            _transformState: [Object] },
         bodyUsed: true,
         size: 0,
         timeout: 5000,
         _raw: [ <Buffer ... > ],
         _abort: false,
         _bytes: 1900 },
      _body: 
       { message: 'An error has occurred.',
         exceptionMessage: 'Sequence contains no matching element',
         exceptionType: 'System.InvalidOperationException',
         stackTrace: ' ...' } }

I have a valid understandable exception message and a stackTrace that I can take back to my developers to further debug. My issue with this though, is have to do a lot of handling to make this raw message into something I want logged (which is maybe what I have to do, but I would think someone would have something helpful out there) and I have to have extra code checking the same status again to verify that "hey, your call failed". I would expect an easier .ifFailed(/do stuff here/) or something. If there is something obvious I missed, I apologize. I tried to Google "Jasmine error handling" without much success.

  • Your API probably shouldn't display stack traces to the consumers of an HTTP response. Check what's in the response body. If it's not sufficient, correlate the failed test with application logs based on the time the expectation failed. Personally, I'd consider visible stack traces in API responses a problem. – toniedzwiedz Sep 27 '17 at 5:35
  • Sounds like I need to talk to my devs about the stack trace bit. But I would still like the error message the api probably throws back. Sometimes it could be something as simple as "Internal Server Error", but I expect if it's an expected error to a bad input it should tell me what about it is bad. I feel like I should be able to log this out somehow to my test report without have to use the then, to check the status after I already checked the status in the expect statement. – S.Huston Sep 27 '17 at 19:04
  • If it's bad input, the status code should be one from the 400 family. 500+ status codes usually represent non-recoverable, unexpected errors. That's not something and API client should be expected to handle. – toniedzwiedz Sep 27 '17 at 19:07
  • Yup, I've got a bug against my dev team about it now. :) But my stance from above still stands, I shouldn't have to recheck the status to verify whether or not it did fail and then log some useful output. In this case I'm working on getting the error code and message changed to 401: Unauthorized. Sometimes you'll get extra information in the error response, I want to log that information. – S.Huston Sep 27 '17 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.