I have the following problem, I'm writing test cases for a application that produces visual output (e.g. graphs) the application is not done yet. So how can i provide expected results, that not just states "must be displayed correctly"
The test process you are looking for is called inspection. It is primarily a manual process.
For a given set of data, the result must be displayed in a certain format and needs to be compared to an exemplar. How you produce the exemplar is up to you. I might suggest that a hand-drawn representation of the display output is sufficient early on, while you might have better results from professional tools (Excel or R) later on when you are refining your presentation. Mark up the exemplar with salient points of comparison.
The second part of the inspection has to do with the graphical representation: colours, groups of colours, fonts, sizes, textual display, etc. Again, for any given set of input data, there should be a checklist of things that must be present and displayed properly.
You can mention the following additional points in the expected results for your visual output of tests run:
- The kind of variances that this visual output (graphs, charts, signal waveform) can have for the variations in your steps,test data,conditions, etc.
- The visual output must be in appropriate orientation & scale.
- The visual output is transferable in other mediums such as print & PDF.
- The output has proper color scheme, font which is easier to read and interpret.
This are the ones that hit my mind at the moment although I am sure there can be more.
A few ideas come to mind:
- If you have known data sets, then you should have known charts, so the inspection can be simplified
There are a few possibilities, and the most suitable depends mostly on
- Does the test designer know for sure he will also test the case
- Are there some sort of criterias existing stating what is the correct output
So if it is possible that someone else is going to test the case, a bit more effort should be given to describe the desired test. On this case it would mean more, but smaller steps. ( "Check this", "check that" instead of "Check the whole thing" ) This make it easier to at least test the right things.
If there are criterias existing, refer to them. You might not need to repeat the criterias in the test steps, just add reference in the test case. Sometimes the "must be displayed correctly" might be a good test case if there is decent sized requirement / design document you inspect against.