Should there be any business logic in automated tests at all?

Question

I work with developers who love to compare results in automated tests. So what they do is, they:

• look at the functionality they want to test
• let the software do what is supposed to be tested
• grab the data and replicate the functionality inside the test
• compare the results of the software and the test

In my opinion this is not testing the software if it works correctly, but it is rather comparing two different implementations (one in c++, one in robot/python) whether they behave the same.

Would it make sense to put up a rule: "No business logic replication inside robot tests" in order to get them on a data driven test track?

Answer 1

I agree that it is a bad idea to reimplement the SUT's algorithm inside the test. It is bad because the test's algorithm is just as likely to be buggy as the SUT's algorithm. Or worse, the test may replicate a bug in the SUT.

If your test hard-codes the expected result rather than calculating the expected result, it may need to include additional comments explaining where the result came from.

Answer 2

Your test should have as little logic as possible. This helps prevent additional bugs in your test which may lead to false positives/false negatives. Hardcoding your expected results is an acceptable approach. You shouldn't reimplement your feature inside of your tests.

Answer 3

Should we replicate the business logic in tests?

NO.

Ideally once the software has done, what is supposed to be tested, a test should just assert the actual state with the expected pre-determined state of the SUT.

As far as calculating pre-determined expected state is concerned ,a test's primary objective is to 'Verify' an expected state , not to 'Replicate' it.

In our team it is directly hard - coded in tests, copied over from Use Stories as captured in BDD( Cucumber) format as an example scenario with data which act as a communication tool between Business Analyst, Developer & tester.

But NO behind the scene operations in between the test like DB manipulations/ API calls except test data setups/cleanups before & after the test. No re-calculations of business logic inside tests. A big NO.

If we do that then we need tests for these tests also ,as then they are no longer tests anymore as they themselves implementing the functionality same as the application code.

Answer 4

As per my understanding, the steps defined in bullet points is the action to be performed on a software which achieves certain functionality.

Software testing is to perform action and compare the results with the expected.

Quoting from tech target: https://whatis.techtarget.com/definition/business-logic

Business logic is the programming that manages communication between an end user interface and a database.

Is the question intent? keyword driven framework v/s data driven framework for robot tests

Answer 5

When you have an system that is reliable perhaps its been working for a long time, has zero/low test coverage and you're tasked with making changes to the code base then the following technique can be used:

• What one does is to run some input data in to the system and wait for it to produce subsequent outputs and capture the black box interactions in the form of a test.Though this may not lead to understanding the business rules of the system, it will get test coverage really quickly and with very little technical effort.
• Once these tests are in with enough coverage, one will have the confidence to make code changes.

These are called "categorisation tests".