6

Should custom message in assertion describe expected outcome

assertThat("user accounts are the same", account1, equalTo(account2));

or reason for failure

assertThat("user accounts were different", account1, equalTo(account2));
3
  • I prefer, "Expected user accounts to be the same."
    – user246
    May 23 '16 at 21:15
  • @user246 so it is matter of opinion and habits?
    – dzieciou
    May 23 '16 at 21:16
  • I think it's good to be consistent. The standard you choose is less important.
    – user246
    May 23 '16 at 21:20
7

When using this API, you should be describing errors

In JUnit, this signature is described quite clearly in the Javadoc (emphasis mine)

Parameters:

reason - additional information about the error

actual - the computed value being compared

matcher - an expression, built of Matchers, specifying allowed values

What you're describing by providing the reason is the error, a situation in which the assertion failed.

Therefore I'd say that out of the two suggestions you mention in your question, the latter is preferable:

assertThat("user accounts were different", account1, equalTo(account2));

as to what passes for a good error description, it's a very open ended question... but I'm pretty sure "user accounts are the same" is a terrible error description in this context.

Why it probably shouldn't matter if you provide this message at all

The fact that you need to explicitly specify this message may hint that there's something wrong with the test in which you're using it.

Every unit test should ideally verify one thing and, in my opinion, that thing should be clearly stated in the method name itself. In my experience, if you have well-described methods (as well as variables) in your test and one assertion per test case, the additional message is usually redundant. Just reading the method's name should be enough to know what went wrong and why.

It's hard to say anything more without knowing the broader context but the following things should be obvious when looking at the test:

  • what the accounts compared are
  • under what circumstances the comparison happens
  • whether or not they should be the same
  • which objects involved are relevant to the test case and which are not

account1 and account2 just look like bad names carrying no additional information above what's already expressed by the type of the variables.

For the sake of demonstration, let's say that the application you're working on allows for users to change display names (like the stack exchange sites do) and that you're testing if your subject under test can recognise an account after the change

@Test
public void displayNameShouldNotDetermineUserIdentity() {
    Account preNameChangeAccount = AccountBuilder.fromId(1407656)
                                          .displayName("tom")
                                          .build();
    Account postNameChangeAccount = AccountBuilder.fromId(1407656)
                                          .displayName("toniedzwiedz")
                                          .build();
    assertThat(postNameChangeAccount, is(equalTo(preNameChangeAccount)));
}

If you keep everything descriptive enough, both the code and the output should be clear without defining custom assertion messages.

4

My preferred practice for assertion messages is to make assertion messages unnecessary. And my usual way of doing that:

  • Write one assertion per test method.
  • Name the test method to describe the responsibility being tested.

This way, when the assertion fails, the name of the test tells me most of what I need to know to diagnose the problem. And the mismatch description tells me the rest of the story.

On the rare occasions where I need more than one assertion per test method, the assertions usually distinguish themselves naturally (without an additional message) based on:

  • the matcher's description.
  • the value's description of itself (which the matcher includes in the failure description).

To help with that, I sometimes implement toString() or SelfDescribing.describeTo() in my test subjects specifically to support tests, even if I don't need those methods for other reasons.

On the very, very rare occasions where I need an additional message, I write the message to identify the subject—to describe where it came from, or the context in which it was created or evaluated:

  • assertThat("user's account", account1, equalTo(account2));

The assertThat() method will compose a perfectly good diagnostic message from:

  • the given "reason" string
  • the matcher's mismatch description
  • the test subject's description of itself

So my bottom line:

  • Write great test names.
  • Make test subjects describe themselves well.
  • Make custom matchers describe themselves well.
  • Whenever I can get away with it (which is very nearly all the time), write one assertion per test method, so that the test name, matcher, and subject do most of the work of describing failures.
  • When necessary, supplement the assertion with a message that simply identifies the assertion subject.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.