I am a QA. This is the software chart:

Search Engine (Apache Solr) <-> Server (expose itself to UI by an API) <-> UI.

The server makes queries on the search engine and send the returned data to the UI. I want to test the correctness of the data, to make sure that the returned values is the expected returned values.

I don't manage the Search Engine. I do have an access for manage the data inside it, and making queries on that Search Engine.

As a tester I can use the server's API.

I see two options:

  1. Implementing the query on my own and make sure its the same results as the results of the developer’s query (which does not make sense in tens of tests and doesn't indicates 100% of correctness.).

  2. Insert specific data to the search enginge and check the retured values from the server or from the UI knowing the desired returned values for query in advance (which seems very fragile because sometimes new data would be inserted for new tests, and the new data may cause a failure to the existing results because of changing the assert's ecpected value).

What is the best practice from those options? I feel like I'm missing something because both of those ways are not perfect, is there a better way for doing this?

| improve this question | | | | |
  • 1
    Is the search engine built by your organization or is it owned by a third party (like Google)? Your tags imply that the query is run against a database and that an API is involved - does the API belong to the search engine? How much control do you, as the tester, have over the database? Can you access the API directly? Without this information any answers would have to be quite general and might not be much help to you. – Kate Paulk Sep 30 '19 at 12:04
  • Is the purpose of this test to prove that the developer's query in the server is correct, or that the search engine works like it should? – deasa Jan 7 at 18:17

Assuming that you are working on an isolated test environment (ie, that the only data the whole system works with is your test data), I would recommend approach #2, using a specific dataset.

From your description of approach #2 it seems like the test data also contains data from tests other than yours. That is the problem. If you can control the dataset, then you will know exactly what answers your query should return. Your test case would be something like this: create your own dataset with a known set of answers, load it in a fresh empty copy of the system (ie, an empty database (or whatever other storage is used)), run the query, check that it produces the expected answers.

In this approach, you will need a separate test dataset for each query that you are testing.

| improve this answer | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.