The module I'm testing creates some graphical reports which include things like scatter plots and multipage line graphs. The bitmap will never be exactly the same between test runs, since some variables, like current timestamp labels, are drawn. Currently I'm testing this output by converting the PDF which is created into a bitmap, then doing a simple test of the percentages of red, green, blue, white, black, etc. pixels present in the entire bitmap. I'm mainly using primary colours in the graphics so it works reasonably well. I have an error tolerance (set to 0.01%) which allows minor changes to get through like the timestamp label.
This current solution is a bit fragile - e.g. when a valid change happens to the graphics, then the colour balance may change and so I'll need to update the percentages expected in the test. It feels like I'm not really testing the content of the bitmap, and rather I'm just testing that the test data generates the same bitmap every time -- i.e. its detecting a change rather than testing its doing the right thing.
I'm interested to hear if anyone thinks this is a good approach to testing graphics? Has anyone used any other approach to test graphics which are in the form of a bitmap and/or PDF. The app is a desktop application written in C# and WPF 4.0. I'm running tests using the built in unit test framework of Visual Studio 2010.
UPDATE: method of testing two images for similarity
There's some more detail about the method here: