Best practice for testing data integrity between server and UI

Question

I am a QA. This is the software chart:

Search Engine (Apache Solr) <-> Server (expose itself to UI by an API) <-> UI.

The server makes queries on the search engine and send the returned data to the UI. I want to test the correctness of the data, to make sure that the returned values is the expected returned values.

I don't manage the Search Engine. I do have an access for manage the data inside it, and making queries on that Search Engine.

As a tester I can use the server's API.

I see two options:

1. Implementing the query on my own and make sure its the same results as the results of the developer’s query (which does not make sense in tens of tests and doesn't indicates 100% of correctness.).

2. Insert specific data to the search enginge and check the retured values from the server or from the UI knowing the desired returned values for query in advance (which seems very fragile because sometimes new data would be inserted for new tests, and the new data may cause a failure to the existing results because of changing the assert's ecpected value).

What is the best practice from those options? I feel like I'm missing something because both of those ways are not perfect, is there a better way for doing this?

Answer 1

Assuming that you are working on an isolated test environment (ie, that the only data the whole system works with is your test data), I would recommend approach #2, using a specific dataset.

From your description of approach #2 it seems like the test data also contains data from tests other than yours. That is the problem. If you can control the dataset, then you will know exactly what answers your query should return. Your test case would be something like this: create your own dataset with a known set of answers, load it in a fresh empty copy of the system (ie, an empty database (or whatever other storage is used)), run the query, check that it produces the expected answers.

In this approach, you will need a separate test dataset for each query that you are testing.

Answer 2

To me it sounds like a typical Server-client test (Database <> Server <> Client) but you have a search engine instead of DB. Most of the time you don't want to test if DB itself is working as expected. Therefore, tests could be designed from different aspects:

• Data validation: sequence of queries return the expected data. The focus is more on queries/macros etc. Some test scripts will do the job.
• Logic of the server/API: GET/POST/PUT etc are returning the expected response. Of course, it is expected from developers to mock the DB and write good test for the API methods. if so, no extra test is needed.
• UI test: focus is more on the flow and users' interactions with the UI. Many of them could be mocked, so frontend developers are able to write test more efficiently.
• end-2-end: Run all components and do a full end to end test to see if they work well together. As you know it is usually flaky and expensive, so if previous tests are not enough, e2e makes sense. Typical high level test frameworks e.g. Selenium could help here.
• There are many other tests depends on your need that could be added e.g. non-functional tests, contract test etc.

If any of the above tests are missing, try to agree with development team how and who can implement them.