What make good search result listings?
You type in a query in a search box at Google or Yahoo or live.com or Ask, and hit the search button.
In the search results, you see lists of links to pages that should have something to do with the keyword phrase that you typed into the search engine. Which do you click upon? How might the words in the Search Result Listings, the caption – the title, snippet, and URL – influence what people will click upon?
That’s the question raised in Microsoft’s The Influence of Caption Features on Clickthrough Patterns in Web Search (pdf)
I really like it when the search engines share some of their findings on usability issues around the way people search. Here’s the quick answer, from the paper’s abstract:
The findings of our study suggest that relatively simple caption features such as the presence of all terms query terms, the readability of the snippet, and the length of the URL shown in the caption, can significantly influence user’s Web search behavior.
Skipping to the quick answer misses the other questions and observations raised in the paper, such as potential differences based upon whether the search is informational or navigational in nature. There’s also some nice discussion of how captions were compared.
Choices of snippets displayed by the search engine may also play a role in which link is chosen by a searcher, from search result listings. When a link is redirected, the search engine may choose to display the original URL or the redirected URL. A title or description (as a snippet) from a Web directory such as the DMOZ might be shown instead of information found on the page listed.
The paper provides a nice rundown on some of the previous research involving search result listings and document summarization used in search engines, including a review of captions that depend upon the query searched for, and those provided where the summary doesn’t depend upon queries.
The study doesn’t involve a small pool of users or a comparison of results selected through methods like eye-tracking, but instead explores user logs from the search engines, and compares clickthroughs of pairs of results that are adjacent to each other in search result lists.
This finding is also interesting:
The results support claims that missing snippets, short snippets, missing query terms, and complex URLs negatively impact clickthroughs. While this outcome may not be surprising, we are aware of no other work that can provide support for claims of this type in the context of a commercial Web search engine.
Search engineers do make a number of decisions when they present title and snippet and URL for a page in search result sets, such as the length of title that they will show (long titles will get truncated if they are too long), which snippet they choose when keyword phrases appear more than once on a page, and which URL to show when there is a redirect in place. It’s good to see that they are performing user testing on what they do show.
Hi!
If there is a great part of page layout influencing the user’s click behaviour, there’s also a preexisting idea of which result to obtain that plays an important role.
In other words, when I am trying to buy a Ford and I’ll enter ‘Ford’ on Google, but I’ll be much more interested in address or snippet elements related to ‘car sales’ than to the search term itself.
This last behaviour will lead on a rather short term to the development of an important number of powerful vertical search engines for instance in people search (- or car sales search) in order to refine the user’s search results.
Search engines such as Ziki.com or more recently Spock.com are well placed to fullfill the user’s expectations on people search.
The only question that remains is how will Google, Yahoo! and MSN react? Will they extend their search engine portfolio (today composed of videos, blogs, pictures, etc.) or adopt new technologies (semantic search) to make general search engines
even more powerful?
Hi Dmitri,
The traditional model isn’t quite so traditional anymore, but at this point in time is much more relevant to a mass audience than some of the upcoming search engines. We do need to keep aware of some of the upcoming search engines, as you point out.
Bill, the search results listings you’ve described pertain to the traditional model of SERP, where each result is represented by a short snippet of text. However, there is a number of new (small) engines that display a more relevant and/or sizable content to represent a result – sometimes not even showing the query keywords. SenseBot.net goes beyond that and crosses the boundaries of individual results, creating a coherent summary on the topic of the query. When these tools eventually seep into the mainstream, the SEO techniques may need to undergo significant changes.
There has also been an extensive research done on tracking eye movements of searchers reacting to the search results and what links they are more prone to click.
Wonder if there is any correlation between these two studies.
Hi Rupert,
Great point on the influence of intent on the choices made. I suspect that the richer the title and snippet, the more likely a choice like the one you point at is easy to make, but it is a consideration that needs to be taken into account. It is more than layout of a caption that will influence a choice between results.
Dmitri,
Changing around the features of captions in the way that you describe does influence something like this considerably. The additional information that could be displayed, even by older search engines could play a significant role in which results people choose. Be interesting to see a similar experiment with expanding information, and the choices that people make.
Web Presence,
Eye-tracking is often used as a supplemental source of information in usability studies. It would be fun to see someone do an experiment with similar goals using eye-tracking as part of the experiment.