We have several Angular applications (Micro UIs) and they are all 100% covered by our UATs (User Acceptance Tests) written with Protractor (Angular Test Framework based on Selenium).
There are several problems with this approach, although this is probably the best and suggested way for Angular Testing:
- Time consuming... Do not understand me wrong. These tests are super handy, whenever we make a change, we let these run (these are actually integrated in our CI/CD Pipelines) and be sure that the app works still as we want. But still, even if we test a use case at least 1 time manually anyway, this has to be also written in Protractor/Jasmine. And takes time..
- These Tests are very dependant on HTML Structure. Some locators find the html elements with xpaths (very bad), some with id etc. but at the end, any change in HTML, would somehow cause a change in our tests.. although the change does not change anything on UI for the end user.
- The whole CSS is ignored. Like the typical Selenium Tests, all tests run against the DOM, and except tests which checks the hidden/shown fields, over 90% of our CSS is ignored.. if one day, a single line which never supposed to be shown in 2 lines shown, really is shown in 2 lines, we will not know that automatically.
I imagine a test framework like this: you open your app on browser, and start the keyboard/mouse capture tool of your test framework. Then you start testing a scenario on your app in your browser, for example you move your mouse on a specific location, you click on a button, you choose a value from a dropdown, a redirect occurs, and at the end you look on your screen and see if the desired X text is shown on the right side.
The only thing which the capture tool cannot know is what you really checked (X text on the right side). For this you take a screenshot of this area, and put in your test framework.
These mouse/keyboard captures will then be replayed while testing on the Pipeline and the checks/asserts are done against these screenshots.. even with some pixel offset maybe, if you do not want your tests fail in case of tiny css changes.
Would it be usefull at all? Do you know some frameworks like this? Specially which we may use for our Angular apps? Do you think this approch is better regarding time and quality?
The benefits would be:
- Since you just let your capturing tool run while you are manually testing your app just after you finish your implementation, and make some screenshots and put them in framework somehow, tests would not consume so much time like writing code for UATs.
- They would mostly run independent of HTML Structure. So as long as our button is located on its location and on its behaviour, it would not bother us if it now has a one more div parent in the DOM.
- Our Css would also be tested since we compare pictures.
And at the end, this is just what we want, to look on the app and see if it looks right, click and enter some keyboard events and check again if we see what we want. Not like protractor or selenium tests, which just blindly communicate with the DOM.
