3

I want to write automated e2e / functional / acceptance tests for a statistic service which calculates the numbers only once a day.

For the tests I need to make sure that I know which numbers the test should expect.

I have one option in mind: Create an endpoint which calculates the data before running the expectations. Downside: This would introduce extra logic into the app only for the tests.

Do you have more / better options? Or am I on the wrong path, should I only care about numbers - but not exact numbers in the e2e tests - and cover the real numbers on a different test level?

Thanks a lot for your ideas / thoughts.

Improve this question
6
  • Just so we understand correctly- the service accepts data (continuously or in one batch), do statistical calculations and returns some data back?
    – Rsf
    Nov 7, 2019 at 14:53
  • In the system new resources and data points are created continuously, once a day the system calculates / updates the statistical data which I want to write a test against. Does this help to understand the context better?
    – Paul Vincent Beigang
    Nov 7, 2019 at 15:13
  • it does, does it matter whether the data was uploaded continuously or can you upload everything in one batch ?
    – Rsf
    Nov 7, 2019 at 15:14
  • it does not matter
    – Paul Vincent Beigang
    Nov 7, 2019 at 16:41
  • How are you doing the data seeding for the test? I.e. the @BeforeTest-like procedure that set up the database(s) in a known test state?
    – João Farias
    Nov 7, 2019 at 19:12

2 Answers 2

Reset to default
1

I hope I understand your question correctly. You have a system that looks something this:

  • Input data sources
  • Scheduled task to calculate a statistic
  • Output store for scheduled task results
  • A way to see the output, maybe a user interface

A test that verifies that process could look like

  • Prep fixed input data
  • Trigger the calculation process
  • Verify the output in the user interface

I have some systems that use scheduled tasks in Windows todo calculations:

  • We added a schedule service HTTP end-point to trigger the calculation
  • The windows tasks uses a script to call the HTTP end-point
  • Our tests use the same HTTP end-point to trigger the schedule
  • We manual test that windows scheduling system works once

Some notes

  • Test most of the calculations in lower level tests (unit/component tests)
  • In the end-to-end test maybe only check the totals, not the details
  • I like to extend applications to make it more testable, having some end-points only used by tests is not a bad thing. In this case we could also add a button to the user-interface to trigger the schedule. Maybe users might also like that.
Improve this answer
0

I feel like there are actually two parts to verify based on your description, the calculation part, and the display of updated data.

For the calculation part, because you said it runs once a day, I assume it's a cron job, a script. Then some audits to verify the calculation will be enough. (test inputs with expected outputs).

I normally won't go with your original idea of creating an endpoint to calculate yourself, because the potential bug in your own calculation logic will create noise and not easy to maintain.

For the second part, the things you actually care about in your acceptance test are that the calculation job ran and the data is updated. The actual checkpoint to test is just "the data is updated". so if you have access to DB, test the data shown match the calculation result data in DB, if no access to DB, just make sure the data is updated.

These two parts should be sufficient for the feature, but the auditing part should be separate from your e2e test suite.

Improve this answer
3
  • how would you test "the data is updated"?
    – Paul Vincent Beigang
    Nov 11, 2019 at 9:52
  • The general idea is keeping a copy of the old version of data, and verify the current data is different.
    – E.Du
    Nov 11, 2019 at 23:32
  • For example, you store the whole table as 'old data', then the second day, you verify current data in the table is not exactly the same as 'old data' and update 'old data' with current data for the next day. if it's too much data checkpoints in the table, you can just use visual testing, take a screenshot of the page, and verify the differences between screenshots exceed a certain threshold (based on your actual data trends and system behavior).
    – E.Du
    Nov 11, 2019 at 23:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.