I would like to automate the testing process of a software that displays image slideshows on an IoT device, ensuring that transitions between images (say, IM1 -> IM2 -> IM3) produce the intended effect.
Endurance tests are sometimes run to test this, but failures happen semi-rarely so checking for errors manually is not the best solution.
I am considering using image analysis with potentially OpenCV, but am open to suggestions on how to approach this problem.
In order to avoid errors to happen on the User Interface, it good to make unit-like checks on lower levels, so you don't need rely solely on UI checks. If the device communicates with a server, the server-side calculations can be tested more easily.
Since you are dealing with intermittent errors, you could log states when failures happen, so you can spot patterns and find errors in code. This way, your checks don't longer say "It's is right"/"It's wrong", but "I found an error in this situation, please take a look".
Regarding the checks themselves, comparing two images pixel-by-pixel are very brittle. There are a manifold of image comparisons techniques, from simple RGB color difference to artificial intelligence. This link has some examples, but you should use something feasible for your stack and environment.
