3

I work with developers who love to compare results in automated tests. So what they do is, they:

  • look at the functionality they want to test
  • let the software do what is supposed to be tested
  • grab the data and replicate the functionality inside the test
  • compare the results of the software and the test

In my opinion this is not testing the software if it works correctly, but it is rather comparing two different implementations (one in c++, one in robot/python) whether they behave the same.

Would it make sense to put up a rule: "No business logic replication inside robot tests" in order to get them on a data driven test track?

  • What do you mean by "grab the data and replicate the functionality inside the test "? – Vishal Aggarwal Sep 10 '18 at 13:42
  • Are we talking about unit tests or UI tests here? – Vishal Aggarwal Sep 10 '18 at 13:54
  • robot tests are neither unit tests nor ui tests, but rather integration I'd say. – tuexss Sep 10 '18 at 14:53
  • by "grab the data and replicate functionality" let's take a look at an example: In the software I have a feature that filters a set of items by a filter term. What our developers do to test it is, they take the original data set, they write an algorithm in the test that filters the data set by the search term. they compare if this is what the software gets as a result. What I'd propose is, I hardcode the target set in the test instead of writing a business logic replica. – tuexss Sep 10 '18 at 14:55
  • Thanks @ tuexss for details.I completely agree with your viewpoint. – Vishal Aggarwal Sep 10 '18 at 15:07
3

I agree that it is a bad idea to reimplement the SUT's algorithm inside the test. It is bad because the test's algorithm is just as likely to be buggy as the SUT's algorithm. Or worse, the test may replicate a bug in the SUT.

If your test hard-codes the expected result rather than calculating the expected result, it may need to include additional comments explaining where the result came from.

  • Exactly, if I'd be a developer, and I'd have a logical error in my algorithm, I will probably apply the same logical error in another framework/language. You are right, by hardcoding them it might be unclear to a reader why this should be the expected result. – tuexss Sep 10 '18 at 14:58
2

Your test should have as little logic as possible. This helps prevent additional bugs in your test which may lead to false positives/false negatives. Hardcoding your expected results is an acceptable approach. You shouldn't reimplement your feature inside of your tests.

1

Should we replicate the business logic in tests?

NO.

Ideally once the software has done, what is supposed to be tested, a test should just assert the actual state with the expected pre-determined state of the SUT.

As far as calculating pre-determined expected state is concerned ,a test's primary objective is to 'Verify' an expected state , not to 'Replicate' it.

In our team it is directly hard - coded in tests, copied over from Use Stories as captured in BDD( Cucumber) format as an example scenario with data which act as a communication tool between Business Analyst, Developer & tester.

But NO behind the scene operations in between the test like DB manipulations/ API calls except test data setups/cleanups before & after the test. No re-calculations of business logic inside tests. A big NO.

If we do that then we need tests for these tests also ,as then they are no longer tests anymore as they themselves implementing the functionality same as the application code.

  • 1
    my question is about how to get to the expected pre-determined state. do you calculate with code inside the test, or do you calculate it once, and just insert the result data to compare against. – tuexss Sep 10 '18 at 14:52
0

As per my understanding, the steps defined in bullet points is the action to be performed on a software which achieves certain functionality.

Software testing is to perform action and compare the results with the expected.

Quoting from tech target: https://whatis.techtarget.com/definition/business-logic

Business logic is the programming that manages communication between an end user interface and a database.

Is the question intent? keyword driven framework v/s data driven framework for robot tests

0

When you have an system that is reliable perhaps its been working for a long time, has zero/low test coverage and you're tasked with making changes to the code base then the following technique can be used:

  • What one does is to run some input data in to the system and wait for it to produce subsequent outputs and capture the black box interactions in the form of a test.Though this may not lead to understanding the business rules of the system, it will get test coverage really quickly and with very little technical effort.
  • Once these tests are in with enough coverage, one will have the confidence to make code changes.

These are called "categorisation tests".

  • 1
    Interesting approach to analyse a legacy system. I guess you could quickly verify assumptions and hypotheses on what it is doing that way. But I would say this is a different challenge altogether. – tuexss Sep 17 '18 at 10:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.