Verify correct generation of documents

Question

I am currently working on implementing a service which generates documents of different types (docx, xlsx,...) from templates.

The question I now encountered is what the correct way is to test the library automatically, i.e. how to verify the generated documents.

Do you know of any specific patterns, frameworks, models, practices,... to do this?

Answer 1

Write the program ('the service') that calls the library and places output in some directory.

Then write a test program that calls the application code, exercises it, then verifies that the output in the output directory is correct with correct filetypes and extensions.

The details are gonna depend on everything - OS, service info, etc.

Answer 2

KISS - Keep It Simple (and Stupid).

Start with your service generating the simplest document type possible (if .txt is an option, start with that). It's up to you whether you work with a test-driven approach and write your tests first or not, but if the tests are that the files get created correctly I'd think about this set of tests for each template:

• A file is created in the destination directory that has the name you're expecting
• The file is of the correct type
• The file has the correct permissions (if file permissions matter to your service)
• The file matches the template you used - this is something you can either code for or (preferably) use an intelligent file comparison command line interface to work with.
  • The comparison doesn't need to be too fancy: if you aren't using timestamps, you can take a predefined sample and use it as your baseline - then a simple binary same is all you need.
  • If you do have timestamps or anything else that's different each time, you can either record what you expect it to be when your test generates the file, or you can set differences between your files to permit those specific differences only. It depends on your needs.

Once you have your set of tests running for one file type, start adding file types. Add the simplest first - CSV files can be compared with pretty much any text comparison tool, as can XML files. XLSX and DOCX files need plugins and probably a paid comparison tool. PDF files are more challenging, especially if they generate by creating an image of the text they're laying out. That said, if there's no timestamps, may comparison tools can determine whether two files are identical at the binary level.

That's it.

As long as you keep your service code and your test code DRY, you can refactor as you go.

My personal rule of thumb is that if I find myself writing something similar enough to be called as a method more than twice, it gets refactored into a method. Similarly, if it looks like I'll need a particular set of methods frequently, I'll start splitting them into their own project so I can call them as a library.

I find letting the structure of automated tests evolve based on the needs of the AUT tends to work better than trying to figure out everything up front - I don't end up with architecture I don't need and I don't wind up creating tests or other structures that aren't needed.