0

I am currently working on implementing a service which generates documents of different types (docx, xlsx,...) from templates.

The question I now encountered is what the correct way is to test the library automatically, i.e. how to verify the generated documents.

Do you know of any specific patterns, frameworks, models, practices,... to do this?

Improve this question
4
  • What exactly do you want to investigate with your testing? Which risks have you identified so far? Do you know good oracles for your context? – João Farias May 19 '20 at 18:37
  • Verify that documents are generated or verify the data and design of documents? – Prome May 20 '20 at 5:52
  • @Prome the aim is to verify the content and its layout, so second one – Anton May 20 '20 at 10:16
  • @JoãoFarias we want to investgate whether the document generation method does what it should, and persist this in automated tests to ensure continous validation of the service re: oracles, no I do not know one, I am looking for one as well. – Anton May 20 '20 at 10:18
1

Write the program ('the service') that calls the library and places output in some directory.

Then write a test program that calls the application code, exercises it, then verifies that the output in the output directory is correct with correct filetypes and extensions.

The details are gonna depend on everything - OS, service info, etc.

Improve this answer
3
  • My question is what the best design for such a test program would be – Anton May 20 '20 at 10:19
  • This is a design. Please add more to your question. We don't write the code and only you know the process and can detail it more. What are you actually stuck on. – Michael Durrant May 20 '20 at 10:27
  • patterns, frameworks, models, practices ? Nah, just write some code that check file types. Don't over-engineer or over-think. Seeking larger picture approaches is good. Just not needed here. – Michael Durrant May 20 '20 at 10:29
1

KISS - Keep It Simple (and Stupid).

Start with your service generating the simplest document type possible (if .txt is an option, start with that). It's up to you whether you work with a test-driven approach and write your tests first or not, but if the tests are that the files get created correctly I'd think about this set of tests for each template:

  • A file is created in the destination directory that has the name you're expecting
  • The file is of the correct type
  • The file has the correct permissions (if file permissions matter to your service)
  • The file matches the template you used - this is something you can either code for or (preferably) use an intelligent file comparison command line interface to work with.
    • The comparison doesn't need to be too fancy: if you aren't using timestamps, you can take a predefined sample and use it as your baseline - then a simple binary same is all you need.
    • If you do have timestamps or anything else that's different each time, you can either record what you expect it to be when your test generates the file, or you can set differences between your files to permit those specific differences only. It depends on your needs.

Once you have your set of tests running for one file type, start adding file types. Add the simplest first - CSV files can be compared with pretty much any text comparison tool, as can XML files. XLSX and DOCX files need plugins and probably a paid comparison tool. PDF files are more challenging, especially if they generate by creating an image of the text they're laying out. That said, if there's no timestamps, may comparison tools can determine whether two files are identical at the binary level.

That's it.

As long as you keep your service code and your test code DRY, you can refactor as you go.

My personal rule of thumb is that if I find myself writing something similar enough to be called as a method more than twice, it gets refactored into a method. Similarly, if it looks like I'll need a particular set of methods frequently, I'll start splitting them into their own project so I can call them as a library.

I find letting the structure of automated tests evolve based on the needs of the AUT tends to work better than trying to figure out everything up front - I don't end up with architecture I don't need and I don't wind up creating tests or other structures that aren't needed.

Improve this answer
2
  • Wouldn't be more simple to implement visual testing if the main goal is to verify the layout of the documents? – Prome May 21 '20 at 9:13
  • IF the main goal is layout, yes. If it's verifying content and layout, not so much. – Kate Paulk♦ May 22 '20 at 11:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.