Is there a better way of carrying out basic text and form validation instead of doing it at E2E?

Question

My understanding of E2E tests is that we should be carrying out tests that run through the whole system to check if everything works together via Selenium (for a web application). However, I've seen places where they would consider the following test as part of E2E:

• Navigate to a certain page (which would take 1/2 minutes to execute) then check to see if the page has the correct title, paragraph, form titles etc.
• Navigate to a certain page (which would take 1/2 minutes to execute) then check to see if a certain form shows/hides correct validation messages e.g. entering a character in the DOB field.

My understanding is that, as the application grows this will become very expensive to maintain and execute. It almost seems like we are shifting right, and pushing all our tests to the GUI level when they can be executed lower down the stack.

My questions:

1. Is my understanding correct in that these tests should not be part of E2E?
2. Can these tests be executed lower down the stack, is so at what level? For example, surely the developers can mock the data needed to load a specific page, and even then they could use selenium if they really wanted to check if the form was working, or if the page was displaying the correct data. Otherwise can such tests listed in my example be tested at unit level?

The perfect answer will help clear my understanding on where these tests should be executed and how. Any examples related to web applications written in Angular with a choice of potential tools will be perfect. Thank you. :)

Answer 1

Is my understanding correct in that these tests should not be part of E2E?

Yes it is. If a software team decides to forgo unit, integration, api, and other test types, then all testing is pushed to the UI. I've been on teams like this where the thought process is "all testing is the realm of QA, not developers." If a team takes this approach, automated E2E testing will be expensive to maintain and execute, and could become brittle if the app changes frequently. Some people use these points for reasons not to automate, which is the wrong approach. You are also correct that this resembles a "shift-right" mindset.

Can these tests be executed lower down the stack, is so at what level?

Yes. For form validation, a lot of testing can be done via unit tests. All the input can be passed to those functions in a unit test, which will assert any output. This can also be mocked. Now, this can put more work on the development team, so if they are not used to doing unit tests, there could be push back. However, unit testing is traditionally a developer task.

If you take this route, the unit tests are much cheaper to maintain and they execute in milliseconds/seconds instead of minutes. Any regressions will surface a lot sooner, so they can be fixed quicker and easier. This is a shift-left approach.

UI E2E tests should be testing the user flow and integration from one feature to another (user scenario tests). Your happy path tests should be automated at the UI level. An example of this is login, perform some action(s), logout; or open page, search for a product, add product to cart, checkout (in case of ecommerce).

If you want to throw in some sad path tests here to check for UI error messages and elements that support it (error icons appear, text or input box turns red, hidden elements become visible, etc) that can be done. Here, you are testing the different UI states a feature can represent. The keyword here is some sad path tests, not all.

Answer 2

Let's suppose you have the situation where you have a page that loads a list of items names created in the last 24 hours, on the DOM tag ul#recent_items (ul with id recent_items).

It will probably be implemented like:

Frontend:

• (1) There is a component that represents this list, which makes a call to a service, organizes the data, and renders it nicely using the frontend framework (React, Ember, etc...) methods.
• (2) A reference to this (1) component on the page, placing it with the #recent_items id.

Backend:

• (3) There is a endpoint /items?filter=recent which returns the items filtering only the recent ones.

This can be broken in many types of checks:

• For (1), you can mock the service and render only this particular component, checking if it handles the data you provided on the test correctly, UI-wise. Example here.
• For (2), you can build your frontend app locally and visit directly the page, mocking anything service call that is necessary. Here, you will check only that the component at #recent_items is there - no UI check. Example here.
• For (3), you can make a HTTP call to the endpoint and check what is the data returned. Example here. This will depend on how this endpoint is coded, of course. Database setup, unit tests, etc, apply here in order to break this big service test further down.
• Additionally, it is necessary to inform the service how it is expected to be used. This is call Consumer-Driven Contract Testing. Basically the unit tests of the consumers (in this case the frontend app) will be registered and the provider (the service) will check how itself behaves with this usage. For instance, the frontend unit tests could mock a call to /items?filter=recent24hours - and if the service only provides /items?filter=recent, it will not integrate well. The Contract Test would fail and the service team would decide how to handle this - either changing the endpoint or talking with the developers of the frontend app.

The E2E checks would serve to verify if the building process was acceptable. The four checks above are local checks, the system are not integrated during the test (although the Contract Test checks the integration, functionality-wise). If the build pipeline indicates that the frontend will call the service at the IP XYZ, but the service is at TVX, it will not work.

Now, talking about more software engineering-wise, understanding the necessary checks after the implementation is quite difficult, because the bundle is already done and integrated. E2E seems very tempting, because it is straightforward. So, it would be easier to build these smaller checks while you build the components themselves (the frontend, the services, etc), rather than after the whole feature is done.