Assertion on static value or calculated one

Question

This may be more of a theoretical question than a practical one, but I'd like to know the community's opinion.

Some context:

On some integration UI test, in which we are testing that a field appearing on a specific area is actually showing the expected value that is calculated on backend, so for example, for an input like:

Consider  Value    Probability
---------------------------------
  N        100        20%
  Y        100        10%
  Y        200        15% 

The system will show on a field of the page those three lines in a specific format, and in another field:

Total: 40

(notice 40 = 100*10% + 200*15%, first line is set to not be considered)

In the scope of a selenium test that is creating those initial lines onto database via preconditions, what would you consider a good practice and what a bad practice?

A) Do the assertion on, literally, the "magic string" 40.

createPreconds();
assert(totalField.getValue()).equals(40);

B) Do the assertion against a calculation done on the test.

createPreconds();
assert(totalField.getValue()).equals(calculatedExpected());

float calculatedExpected(){
  // get all the created preconditions, and compute what should be the correct output
  return number
}

It is to be mentioned that the input is actually much more complex than what I'm explaining, and its logic may change depending on the state of the system (i.e., first column may or may not exist).

Question in short, is: which would you say it's a better approach? I personally see problems on both approaches:

Option A is using "magic strings", option B is pretty much duplicating the behavior of the code to be tested, neither of them usually considered good practices.

Option A is straight forward testing UI displaying the value and does not pollute automation code. Option B makes the test more resilient to changes, as well as for people in the future understanding why that value is that value.

Apologies for the long question, and thank you in advance

Answer 1

Think from this perspective, lets say there is a function, Foo that takes in as input a set of values, say I and gives an output O. So, we have n equation like, O = Foo(I). Additionally, we are provided with test data, and an overview of the expected logic. Now, the developer's job is to implement Foo using some algorithm. And tester's (your) job is to validate developer's implementation for Foo. Now, what is the best of way to validate? Using the test data (Approach A) seems like a more verifiable approach as test data (known relation between input and output) appears to be the only correct data available. Using some formula (Approach B) adds another layer of implementation (function), so another layer of required validation (test). Therefore, Approach B should be avoided (although it seems be more flexible) unless the used function is valid enough. Also, for Approach A it is better to use a table of expected values rather than an expected 'magic' string to ensure better functional testing services.

Answer 2

I would consider the whole idea of verifying such logic from the UI perspective as a bad practice. See info on test pyramid.

If you need to test the calculation logic of your service - you can do that on the unit and/or integration testing level, which would be much easier to do, since you will be free from external dependencies in that case.

UI representation (the fact that some value passed from a backend is shown in your page) can be verified on the front-end side by unit test as well.

With Selenium you can verify the basic e2e flow without digging into the details.

Answer 3

From my experience, option A is most suitable in this situation. Option B satisfies the unit tests or integrations test too. You can use data driven testing in this scenario which is more understandable by a non technical staff too.