0

Tests I review often contains the following pattern

@Test
public void test() {

  if (systemIsInStateA()) {
    assertThat(system, shouldPerformInWay(X));
  } else {
    assertThat(system, shouldPerformInWay(Y));
  }
}

Is this an anti-pattern?

To me it seems an anti-pattern because wether a system is in state A appears to be outside of control of the test and it may happen that the test will cover only one execution path. Instead I would expect a test to move the system under test to state A to test first path/assertion and to state B to test the latter.

I wonder if there are situations when such pattern is justified?

1

Your example (without further context) looks as if it would be better as follows:

@Test
public void testStateA() {
    assertThat(system, shouldPerformInWay(X));
}

@Test
public void testStateB() {
    assertThat(system, shouldPerformInWay(Y));
}

It looks like two separate scenarios, so test them as such. If your original code fails (in method shouldPerformInWay) you don't even know which of both cases fails - or you would have to provide additional logging, or debug.

Of course, when using data-driven testing, you have to have a certain balance between code logic and data variability.

I don't know if this applies for your example, and if my explanation will be clear without code, but let's assume we're testing a login screen.

Extreme on one side: complex data (determines logic to follow), complex code. Here, you might have one test method (simple) and based on the data, the PageObject must execute its logic: check for a succesfull login or an error message, or... So your code (PageObject) is complex, because it has to parse your data for the expected behaviour (e.g.: does my data source provide an error message or not?). The data itself must also provide an expected result.

Extreme on the other side: simple data, many test cases (for each outcome). Here, you might have many test methods (one for each possible scenario) each taking specific data input for that case. So the data is simple because it doesn't contain any logic. The test methods are perhaps repetitive but clear on what they do. The PageObjects themselves don't have much logic going on compared to the previous case, because they don't have to parse data for behavior.

|improve this answer|||||
  • Well with data driven testing you can split your dataset into two and have two test methods each with its own dataset. – dzieciou Feb 28 '16 at 14:29
  • 1
    Yes, that is the last paragraph. I usually take the middle ground. If your tests are not data-driven, I would probably split them up... There's no reason to have one test method if you're testing two different outcomes. – FDM Feb 28 '16 at 15:16
1

You can make the test scenarios conditional for purpose of "smoke test" of the whole component to show that you are bale to pass trough in any case, when you do not have control over the case and which variant will "come".

But for detailed testing you should remember, that you should test both variants at least once.

You was not specific what is causing the variants, if this is caused by test data or any other external option or behaviour of the application.

If it is completely out of your control, you should test it in the cycle until all available variants are "passed". If you don't, it could from some kind of viewpoint to be considered as an anti-pattern, or you should present the automated test results as partial results.

|improve this answer|||||
  • +1 but... if the state of environment is out of my control then it may happen it will never be in the state necessary to test second path. I think in such case it's better to mock part of the system instead of running tests infinitely untol it passes. – dzieciou Feb 29 '16 at 17:37
  • @dzieciou: if you can mock part of the system, it means that you have the control. With mock you know, what you can expect and conditional test is not necessary. Then it could be an anti-pattern. – Dee Mar 1 '16 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.