Do you guys separate fails and errors in your automated tests report?
- fail - assert error
- error - all code errors (except from asserts)
In my opinion we should. Lets say we have such testcase scenario:
- login -> logged in
- Create contact (example) -> contact created
If we'd have 2 automated tests for these...:
- ASSERT if logged in
- create contact
- ASSERT if contact created
... and login will be bugged, we could simply filter test results with fails only and find out that login failed (because we have assert on that) by ignoring other ERROR types (we could have 20 of them) that occurred due to that login fail)
What's your approach/experience?