If you have the Google Toolbar installed on your browser, you may soon start seeing some odd behavior at times when you click on a search result.
For some pages, Google might deliver you to a page and may display a popup/information box on the bottom right of the page that covers part of the page. That information box may show one or more excerpts of text from one or more parts of the page that are “relevant to your query,” like in the following image:
If you click on one of the text excerpts, your browser will deliver you to the part of the page where that text appears, and possibly highlight the relevant section.
A paper by Google researchers Anne Aula, Rehan M. Khan and Zhiwei Guan published last month asks the question How does Search Behavior Change as Search Becomes More Difficult? (pdf)
The paper describes two studies in which participants were given informational tasks to perform – a mix of hard and easy questions – to see if searchers adopted different strategies for searching when they were faced with questions where there were definite answers where answers to those questions might be difficult to find. An example of one of the difficult tasks (can you find the answer?):
You once heard that the Dave Matthews Band owns a studio in Virginia but you don’t know the name of it. The studio is located outside of Charlottesville and it’s in the mountains. What is the name of the studio?
The first study had 23 people performing searches, finding answers to questions like the one above, and examining the searches they performed and the pages they visited to see how they went about finding answers. The second study expanded to 179 searchers, and based some of the processes used on things they learned from the first experiment. A general conclusion from the second study:
We’re not often given too much insight directly into how a search engine like Google might check on the quality of their search results, and the algorithms that achieve those results. When we are, it can be interesting to look at some of the processes that their researchers might use, the assumptions that they follow, and the conclusions that they find.
What kinds of experiments would you perform if you were from one of the major search engines, and you wanted to compare two different algorithms that provided similar quality search results? Or you wanted to learn more about how people use the search engine, and if small changes might impact that use?
A couple of recent papers from Google describe experiments that the search engine performed.
Search Task Time and Searcher Satisfaction
How much might the usability of a web page matter to a search engine? If that search engine were to look at an approximation of the layout of a web page, it could try to understand how good of a user experience visiting that page might be, and evaluate the page based upon certain characteristics that it finds upon the page.
A patent application from Yahoo provides a long list of factors that it might look at to determine how usable a web page might be.
So why would a search engine be interested in determining the usability of a web page?
The authors of the document tell us that:
At SIGIR 2007, one of the workshops held at the July Conference in Amsterdam was on Web Information Seeking and Interaction.
Web information seeking and interaction involves looking at the way that searchers interact with Web-based content and applications when they are looking for something. The conference covered a wide range of research, and I want to go into a little more detail on a couple of documents that were authored or co-authored by Google Employees.
The papers and working notes from the workshop contain a nice mix of topics, which are worth taking a look at. The papers at that link that initially caught my attention was one on experiments with eye tracking and mouse movements, and another that explored strategies for Web search.
Exploring How Mouse Movements Relate to Eye Movements on Web Search Results Pages
Kerry Rodden (Google) and Xin Fu (University of North Carolina, Chapel Hill)
If you’ve used Firefox as a browser for a while, you’re probably used to the tabs that allow you to switch back and forth between pages without opening or viewing another instance, or window, of the browser being open. If you’ve been using Firefox for a while, you may have noticed at some point, that the way of closing those tabs had changed.
A usability test by a group of researchers, one from Google and two from NASA Ames, were consulted with by Mozilla and showed that new users of Firefox had problems understanding how tabbed browsing worked, and didn’t understand that they could close tabs by clicking on the “x” in the top right corner of the page.
A paper prepared by those researchers for this years CHI 07 Conference, When Two Methods are Better Than One: Combining user study with cognitive modeling (pdf), describes some of the user interface enhancements explored to make it easier for people to learn how to easily use tabs.
One of the methods explored was to put an “x” on each tab, and the team doing the research used a combination of cognitive modeling (using computer programs that simulate human performance of cognitive skills) and user testing to explore how well that change might work.
If, like me, you find yourself interested in how search engines work, the usability of search engines can be a fascinating topic.
Here are some recent papers and a set of course notes from people at Google On Usability
Relevance and User Satisfaction
A couple of the aims of a search engine are to provide relevant results to searchers, and to have those searchers satisfied with the results that they see. It might be easy to assume that the more relevant search results are to a query, the more satisfied a searcher will be with those results.
A paper from Google explores that area, asking How Well does Result Relevance Predict Session Satisfaction?
In these days of Universal Search, with images, videos, news, and other results that aren’t just links to web pages, there are factors related to results that go beyond which web pages show up in search results. The paper refers to those as “extra” elements that need to be accounted for in search results.
You type in a query in a search box at Google or Yahoo or live.com or Ask, and hit the search button.
In the search results, you see lists of links to pages that should have something to do with the keyword phrase that you typed into the search engine. Which do you click upon? How might the words in the caption – the title, snippet and URL – influence what people will click upon?
That’s the question raised in Microsoft’s The Influence of Caption Features on Clickthrough Patterns in Web Search (pdf)
I really like it when the search engines share some of their findings on usability issues around the way people search. Here’s the quick answer, from the paper’s abstract:
The findings of our study suggest that relatively simple caption features such as the presence of all terms query terms, the readability of the snippet, and the length of the URL shown in the caption, can significantly influence users’ Web search behavior.