Are conditional assertions in tests an anti-pattern?

Question

Tests I review often contains the following pattern

@Test
public void test() {

  if (systemIsInStateA()) {
    assertThat(system, shouldPerformInWay(X));
  } else {
    assertThat(system, shouldPerformInWay(Y));
  }
}

Is this an anti-pattern?

To me it seems an anti-pattern because wether a system is in state A appears to be outside of control of the test and it may happen that the test will cover only one execution path. Instead I would expect a test to move the system under test to state A to test first path/assertion and to state B to test the latter.

I wonder if there are situations when such pattern is justified?

Answer 1

You can make the test scenarios conditional for purpose of "smoke test" of the whole component to show that you are bale to pass trough in any case, when you do not have control over the case and which variant will "come".

But for detailed testing you should remember, that you should test both variants at least once.

You was not specific what is causing the variants, if this is caused by test data or any other external option or behaviour of the application.

If it is completely out of your control, you should test it in the cycle until all available variants are "passed". If you don't, it could from some kind of viewpoint to be considered as an anti-pattern, or you should present the automated test results as partial results.

Answer 2

Your example (without further context) looks as if it would be better as follows:

@Test
public void testStateA() {
    assertThat(system, shouldPerformInWay(X));
}

@Test
public void testStateB() {
    assertThat(system, shouldPerformInWay(Y));
}

It looks like two separate scenarios, so test them as such. If your original code fails (in method shouldPerformInWay) you don't even know which of both cases fails - or you would have to provide additional logging, or debug.

Of course, when using data-driven testing, you have to have a certain balance between code logic and data variability.

I don't know if this applies for your example, and if my explanation will be clear without code, but let's assume we're testing a login screen.

Extreme on one side: complex data (determines logic to follow), complex code. Here, you might have one test method (simple) and based on the data, the PageObject must execute its logic: check for a succesfull login or an error message, or... So your code (PageObject) is complex, because it has to parse your data for the expected behaviour (e.g.: does my data source provide an error message or not?). The data itself must also provide an expected result.

Extreme on the other side: simple data, many test cases (for each outcome). Here, you might have many test methods (one for each possible scenario) each taking specific data input for that case. So the data is simple because it doesn't contain any logic. The test methods are perhaps repetitive but clear on what they do. The PageObjects themselves don't have much logic going on compared to the previous case, because they don't have to parse data for behavior.