Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I had an interview question:

You were told that a search engine result by target query does not satisfy a requirement by its position. How would you test it?

How would you test it?

I had some thoughts though:

  • You need a list of input search queries
  • You need a target position for each of the query
  • You match each target position with actual

But it was not enough.

share|improve this question
    
I would just add you're testing search/ranking algorithm, not the way result snippets are presented or they way query autocompletion works. They both may impact ultimate retrieval effectiveness. –  dzieciou Oct 16 '12 at 19:10

4 Answers 4

up vote 8 down vote accepted

I am not expert in search engines, but there are many factors that contribute to how search results are displayed. The query is is essentially the starting point, and how the engine parses the query is important. Also some search engines will 'refactor' the query based on natural langauge algorithms. The order of returned results is often dependent on various metrics such as popularity, # of matching words, keywords, tags, etc. Also, some of these measures are given more weight than others.

So, this question by itself doesn't really provide you with enough context, or perhaps the purpose of this question was to get you to ask questions to derive more context. Sometimes interviewers will ask vague questions to see if the interviewee will start taking "stabs in the dark" without gaining sufficient context (e.g. start banging away at problems and hope to stumble on a solution, or think about the problem and consider potential solutions with forethought).

share|improve this answer

I worked for a SEO company in 2004. Our target was to get natural/organic search results on the first or second page of Google for the targeted key phrases (Natural/organic search results are the regular search results, not the ones sponsored by Google Ad Words). The company was a start-up and decided to focus on Google only since it had more than 80% of the Search Engine market at the time and studies showed that users very rarely looked past the first two pages of Google for a search result match.

The way we tested the results was to type in the targeted key phrases (combination of keywords) into Google and see if the listing showed up on the first or second page of Google, if yes then it was considered a success, if not then it was considered a failure.

Today things have changed with Google's targeted advertising so people might receive different results from different computers and if they are logged in to Google Plus this might affect the results too, so I don't know if its even possible to test this anymore.

share|improve this answer
    
So simply Google results were treated a a reference == expected results? –  dzieciou Oct 16 '12 at 18:52

Actually, the question how to test search engines is part of information retrieval (IR) problem. I will list just a few approaches how they can be evaluated/tested.

Traditional IR approach

One way in traditional IR is to prepare test setup, including:

  • Candidate queries
  • Dataset of indexed pages
  • Expected result set (included expected ranking) for each query

Next you obtain actual results by running your search engine over candidate queries and compare actual results with expected results using one of information retrieval metrics, e.g. precision/recall or nDCG. That would give you a quantitive answer how bad or good is your search engine with respect to expected result set (requirement).

Obviously, the answer will be context-specific. It will particularly depend on choice of candidate queries, they intention and wording, the dataset of indexed pages, and the way expected results were judged. So what was the context?

Goal-driven/Interactive IR approach

Another approach, maybe more practical, would be to involve potential users and let them assess quality of results for a given query. They could bring their own queries or you could define queries for them. Those could be a few people or a whole community, from which you could get more implicit feedback about results quality (e.g., results that are clicked are expected ones).

Problem isolation

Finally, the problem might be not to evaluate the overall quality of the search engine, but only narrow down the reason for low ranking of this particular query result. So, I would follow the problem you got:

You were told that a search engine result by target query does not satisfy a requirement by its position.

and asked myself:

  • What were the other results that were ranked higher than this?
  • Why were they ranked higher? Can I tell this from result snippets? Maybe they match query keywords but with a different meaning? So maybe the wording is incorrect? Would another wording of query intention improve the result?
  • The same query maybe worded differently, and the same wordings may have different query intention behind them. Do I really know what was intention of a searcher behind this query?
  • What precisely is the requirement? To boost higher sponsored pages? Or rank higher pages that matches query intention?

Those can help you reproduce the issue for other similar queries.

share|improve this answer

For searching your position TrafficTravis is a great online tool you can check your keywords current position in search engines. For checking your website speed Google Pagespeed test is very useful tool. I found one great post about this topic in http://techhowdy.com/website-speed-google-pagespeed-insights/ Please check out, It very useful to you.

share|improve this answer
    
The question was about testing how well search engine works in finding Web page, not whether a particular Web page could be found with a search engine. –  dzieciou Jun 6 at 21:30

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.