Is Google Keeping an Eye on Your Mouse Pointer Placement?
Information about where searchers hover their mouse pointers over different parts of search results, as well as advertisements and Google Onebox results, may be collected by the search engine to be used as ranking signals to determine in part how relevant those items may be seen by Google users in response to a search query.
When I view the contents of a web page, I often find myself moving my mouse pointer along with the areas that I am viewing. There are a couple of reasons for this. One is that it makes it easier to focus upon the part of the page that I’m looking at. Another is that it’s easier to click upon a link that I find interesting if my pointer is near what I’m viewing. Does behavior such as this make mouse pointer placement helpful to search engines?
According to Google, I may not be alone in this kind of behavior. Google may track mouse pointer movements on its search results pages to help rank pages that show up in search results, to determine the quality of sponsored ads within those search results, and to decide whether or not showing onebox results such as maps or definitions or news or stock quotes is appropriate for some search queries.
When Google ranks web pages, it considers a wide range of ranking signals, such as how relevant a page might be to keywords used by a searcher, the quality and quantity of links pointing to that page, and user-behavior data collected about that page.
A number of patent filings and whitepapers from Google have told us that Google might collect a fair amount of user-behavior data about how we browse web pages such as; how long we might spend on pages, how far we might scroll down those pages, which pages we might click upon in search results, which pages we might not click upon, which links we might follow when we visit pages, if we print or bookmark or save pages, and more.
A newly granted patent from Google explores how they might look at our mouse pointer placement on search results pages as a ranking signal.
Problems with Click Through Rates
One method of ranking search results based upon user-behavior is to see which pages people click upon when they perform a search at a search engine, and which ones they don’t. But, there’s a potential problem with that approach.
Let’s say that you search for Barack Obama’s birthday in Google, and the top result shows the birthdate in Google’s snippet. There’s no need to click through to the page, and your informational need is satisfied. Google probably wants that page to continue ranking well, but if the search engine relied upon click-throughs to measure the relevancy of a page for a query, then a lack of clicks to the page would make it seem to be not very relevant. If they track mouse pointer placement, they may know what you looked at.
If Google instead looked to see that people hovered their mouse pointer over a snippet that contained the President’s birthdate, and then moved on to a completely different search, Google might consider that page to be very relevant to that query.
If that search result was the third or fourth listing in that set of search results, but a large number of people hovered their mouse pointers over that snippet, and then went on to a new set of queries, Google may rerank that particular search listing and move it to a more prominent place in the search results, possibly even to the first listing.
The Google patent is:
System and method for modulating search relevancy using pointer activity monitoring
Invented by Taher H. Haveliwala
Assigned to Google
US Patent 7,756,887
Granted July 13, 2010
Filed: February 16, 2005
Abstract
A method and system of modulating search result relevancy use various types of user browsing activities. In particular, a client assistant residing in a client computer monitors movements of a user controlled pointer in a web browser, e.g., when the pointer moves into a predefined region and when it moves out of the predefined region.
A server then determines a relevancy value between an informational item associated with the predefined region and a search query according to the pointer hover period. When preparing a new search result responsive to a search query, the server re-orders identified informational items in accordance with their respective relevancy values such that more relevant items appear before less relevant ones.
The server also uses the relevancy values to determine and/or adjust the content of an one-box result associated with a search query.
The Value of Mouse Pointer Placement
When you are looking at a set of search results, Google may track where your mouse goes on its results page. They tell us that:
A typical user’s behavior is to move the mouse pointer (or any other pointing indicator) over or near a target informational item, keep the mouse pointer there for a period of time while the user reads the item’s information (e.g., title and snippet), and then click through the underlying link or move to another item.
Sometimes, a user may review multiple informational items responsive to a search query, moving a pointer over or near each of the informational items that the user reviews. These various pointer activities can provide another way to evaluate the user’s feedback with respect to a particular informational item.
The patent presents some assumptions about how mouse pointer placement can be interpreted:
For example, a longer hover over a result may indicate a positive opinion about how relevant a listing on the results page might be to a query.
And, if someone moves their mouse pointer across a snippet line by line at a normal reading speed, it may indicate a higher level of attention to that result than if the pointer was kept in a static position or moved randomly.
So, the speed and movement of a mouse pointer as well as where it is placed on a search result page might be tracked to see how much attention a search pays to different search results. If someone hovers over one sponsored listing, or ad, but not another, that might indicate more attention and interest in the ad hovered over. If a local map is shown, or a definition, or some other OneBox result, and the searcher viewing the page hovers over those OneBox results for a while, that could be an indication that the map or the definition or other OneBox listing was helpful.
Client Attention Data
The patent refers information about the tracking of mouse pointer placement as “client attention data,” because this kind of measuring of browsing activities can give the search engine an idea of how much interest there is in different parts of a search result page, and how much attention visitors paid to each of those parts. If there are similar patterns about how a large number of viewers interacted with a page, that data may provide some meaningful information that can be acted upon by the search engine.
The patent also tells us that it might give different weights in determining a relevancy value for mouse pointer placement based upon different areas of a result. If someone hovers over the title to a search result, that might carry a different amount of weight than if they hover over the snippet of a result.
Conclusion
This patent was originally filed in 2005, and it’s possible that Google may not be using the methods described, or tested those methods and have since moved on to other ways of tracking searcher’s attention on search results pages. It’s also possible that Google may be using those mouse pointer placement today.
Looking at how someone may move their mouse pointer across a page does provide more useful information to the search engine than just looking at what items on a search result page that someone might click upon or not click upon. We haven’t seen too many patent filings from the search engines that go into so much depth on how they might measure one specific type of user-behavior and interpret it, like we do with this mouse pointer placement patent.
This patent was also filed before Google introduced Universal Search, and when it mentions OneBox results, it means those special results from other data repositories from Google such as Maps, News, Stock Quotes, etc., that were often shown at the tops of search results rather than blended into search results. It’s possible that a mouse pointer tracking approach could be used by Google to see how effective or useful blended results might be when they are located in places other than at the top of a set of search results as well.
If one sponsored listing, or ad, is hovered over for a while by many viewers, while others aren’t, should that play a role in the placement of an ad? Is that kind of user-behavior part of the quality score for sponsored listings?
Added (7/13/2010 – 12:30): Interesting observations on the Acuity blog about a June 2 presentation at Eyetrack UX in Belgium by Google Senior User Experience Researcher Anne Aula – Eye Gaze Data and the Correlation With Mouse Movement.
Added (7/14/2010 – 9:43 am): A paper to be presented next week at SIGIR’10 in Geneva, Switzerland, from researchers at Emory University is also worth a close look – Ready to Buy or Just Browsing? Detecting Web Searcher: Goals from Interaction Data. It describes how user-behavior data such as mouse pointer placement and scrolling on search results pages from search engines can be used to help understand the intent behind a search.
Is this applicable to Chrome? Google Mobile Browser? hmm
I would never suspected that the fact that a mouse staying certain time on an area counts for Google.
The addition of ‘mouse movement’ to Google’s algorithm is somewhat absurd. Every user is unique and may or may not direct the cursor towards desired information. I find myself reading through a page while my mouse sits at the top for at least a few minutes. Wouldn’t that skew the data Google collects? If they want to mark relevant content they should invent an interactive system that will quickly and effectively allow users to rank materials.
I know when I read a webpage I just place the pointer to the right of the page (out of the way of any text) while I use the scroller to scroll the page.
Does anyone move their mouse over the snippet as they read? It’s like reading with a finger in a book, I don’t think most people do that anymore.
It is interesting but I don’t think very practical or reliable.
Hi Pit,
The idea of considering clicks on links in search results to help a search engine decide which results might be more relevant has been floating around for more than ten years now, but I’ve always sort of questioned it, and the point that this patent raises about how some results don’t get clicked because they provide an answer in the snippet shown to searches does make a lot of sense.
The link I added to the bottom of my post shows that people don’t always move their mouse pointer where they are looking on a page, and perhaps do it less than the people who wrote this patent might have guessed. But I find it interesting that Google is still doing usability research on the topic now, more than five years after the patent was filed. The presentation described in the blog post does tell us that when someone is interested in a search result, and may be considering clicking upon a link to a page, that there’s a good chance that they will move their mouse pointer over that result. So perhaps there is some value to the approach described in this patent.
Hi Steve,
I was wondering about the possibility of that myself.
The patent describes how they might use code either placed on a server to capture this kind of information, or on a browser or browser addon. While the Google Toolbar might be one option, making that code part of a browser would make sense, too.
I checked the source code for Google results earlier today to see if I could find some javascript that might be attempting to capture this kind of information. While there was some that would report upon which links were clicked, I didn’t see anything related to the placement of a mouse pointer.
Google has published some usability studies in the past where they tracked mouse pointers, and one of them mentions passing the pages through a proxy server where a snippet of code was added to track mouse pointer positions every 10ms or so. It’s possible that Google might capture this kind of information through its servers rather than through a client, but not do it all the time.
Hi Steven,
I so sometimes find myself moving my mouse pointer as if I’m using my finger on a page while reading a book, especially when there’s a good chance that there might be a link I might want to follow (usually through a right click, to open it in a separate tab). On a page that’s mostly text, I’m less likely to do that. On a page filled with links, such as search results I may be interested in, I’m more likely to do so.
Hi Edward,
I’m not sure how effective a system like this might end up being, but it sounds like something that would be reasonable to try to see if it does work in having the search engine boost more relevant results in search rankings.
I think that you’re right in saying that everyone is unique, and unlikely to search or use their mouse pointers in the same way. That’s likely true with most user-behavior based systems that might be used to help rank web pages. If lots and lots of people look at the same set of search results though, and a good percentage of them tended to hover over the same result consistently, might that tell us something about that particular result?
It’s possible that a more interactive approach to search results may have some benefits as well. The team that came to Google from Applied Semantics published a couple of patent applications that described a search system that relied less on exact keyword matching and more upon the intent or meanings behind queries, and allows searchers to drill down through different categories and senses of words behind query terms in an interactive fashion. We may see search engines evolve more towards an approach like that in the future.
I can see how selecting text from a snippet (especially selecting and copying) might come in to play but hovering? Maybe if search results were in braille. 🙂 Then again, when looking at these statistics from Google’s all-seeing eye, if say even 20% of searchers of a specific keyword are hovering over a specific search result, that might be kind of absurd to ignore – even if there’s a second search or not.
I myself don’t really move the mouse pointer with my eyes. I just move it when it’s time to click on something or if I need to scroll down. If others are like me, then it might not really be a reliable source of data.
If you ever used mouse-tracking-functionality on any larger project you learned that mouse-gestures can give you a deep insight in what your users want/learn/use/fear…
So I think this one is pretty interesting ~ not for ranking but for userexperience.
Good stuff Bill – this sounds just like Google – using whatever data collecting methods they can to better their SERPs for the user.
Have you ever used Click Density? Pretty neat stuff 🙂
I have a theory that Google bought Urchin and gave Analytics out to the world for free so that they could get their hands on the data beyond the SERPs. Imagine how much more usable, actionable data they gather with GA being installed on a site. You know the metrics, page views, time on page/site, etc.. they can basically tell which content is best by user behavior beyond the SERPs.
I’m happy to have this opportunity to share this thought with you – as you’re the patent master 🙂 — I don’t go into the fine print of the EULA or patents, but I imagine that Google could already be testing (or using) Analytics data for ranking purposes.
I would LOVE your opinion on this Bill – am I crazy or do I make some sort of sense?
Hi Adam,
Interesting. Text selection could potentially be a clearer indication of interest than hovering, but I wonder how many people do that, and what the intent behind it is. When I copy text from a snippet, it’s often to use that text to do a new search.
I think the idea behind the patent filing is one that is worth experimenting on, to see how helpful it might be. I know that Google has done some testing with both mouse pointer tracking and eye tracking, to learn more about how people use their search interface, but I would love to see a published study from them on how helpful mouse pointer tracking might be to providing better search results.
Hi Andrew,
It seems like different people do use their mouse pointers differently.
Some might use it as a reading aid, to help them keep their place on a page. Some might be more keyboard focused, and not use their pointer much until like you, they are ready to click or scroll. Others still might keep their mouse closer to where they are viewing so that it can be an indication of when something might have captured their interest.
I think the behavior that you describe, of not moving a mouse pointer much might possibly less harmful to this kind of data collection than someone who mouses over everything. But, the point that you and a few others in the comments have raised is a good one – that not everyone uses their mouse pointer in the same way.
I added another link to the bottom of my post this morning, which describes some research from a pair of researchers at Emory University, who will be presenting their research on measuring scrolling and mouse movements on search results pages next week. They suggest that it might be possible to differentiate between people who use their mice differently:
Classifying users based upon how they interact with search results pages could possibly overcome the issue you raise about reliability.
Hi Heiko,
That’s a very good point. I know that user experience teams at Google have been working on how useful mouse pointer tracking might be as compared to eye tracking, such as in the SIGIR 2007 paper, Exploring How Mouse Movements Relate to Eye Movements on Web Search Results Pages (PDF, pp. 29-32). It’s nice to see that if they would consider using a method like this one for ranking, that they are doing some research to validate the method.
Hi Arsham,
Click Density is pretty interesting. I do like that Google Analytics provides a site overlay to view where people are clicking upon a page.
I don’t know that Google is using GA for ranking purposes, and I would suspect that they probably aren’t. They have enough data based upon tracking both browsing and searching processes from their users to keep them busy, without delving into the Google Analytics data too.
There are many other positive reasons for Google to have come out with Google Analytics, and provide it for free. For instance, here are a few:
1. Provide website owners with a better tool for tracking their success than ranking reports, which can put a serious drain on the search engine’s processing power.
2. Allow site owners to see more closely the possible impact of the adwords campaigns.
3. Allow adsense publishers to see which pages are most successful.
4. Show Google to be an innovative leader in the search space.
5. Enable Google to have the ability to compare traffic data across similar sites, to try to find abnormal click through patterns in ads shown on some sites.
Very interesting topic, Bill! I just had a coffee and thought about any ways to combine this with new touch-interfaces (iphone, ipad, pda…). would be interesting where the users finger is used to scroll on the touchdisplay. Is it always the same? Should I place my banners there to steal some clicks…?
Bill,
It is funny. While I was reading your post I actually caught myself moving the mouse pointer along in sync as my eyes read the text. It makes sense that they would use this. I think they will use it more as a means to identify which information they should analyze for on page SEO more than anything else. Instead of just taking into consideration the information that is above the fold, they might collect all information that actually gets read on a regular basis. Then count the number of word occurrences, phrase occurrences, font sizes, hypelinks, etc. before storing that information in a new special topic sensitive repository.
I think that is more realistic rather than trying to draw an opinion about a word or phrased based on a mouseover. It might just improve the quality of their index or possibly be used as a new type of index.
It seems to make sense that the only information that should factor into rankings and even be indexed at all is content that the visitors look at. You can guestimate this by applying some kind of typical heatmap filter like the “above the fold” or “center of the page” example. However, it would be far more accurate if you could measure it somehow. The content that is in the highest heat area should probably be considered a more important search relevancy factor. There might be some super highly relevant content just below the fold that 95% of visitors read. There might be high value content strangely located in the lower right quadrant on some pages that would otherwise be completely neglected.
Your thoughts?
Hi Heiko,
I know Google has done some te4sting and experiments with iPhones, including Computers and iPhones and Mobile Phones, oh my! A logs-based comparison of search users on different devices. I don’t recall seeing anything specifically on user interfaces and touch screens, but I wouldn’t be surprised if someone at Google considered researching that topic.
Purposefully placing banners where someone would be most likely to inadvertently touch them might be up there with making sponsored listings almost indistinquishable from organic listings in search results. I’m not sure that’s a path that Google would really want to pursue.
Hi Bill,
Very thought provoking post as usual. 😎
I personally don’t think that this would be a reliable signal of intention or relevance. Some like to hoover, some like to put the mouse to the side of the screen whilst scrolling and maybe some even like to highlight the text when reading certain sections, but like I say, I personally don’t believe that this would provide Google with any consistent and accurate data that could be used to determine whether a specific SERP listing should be re-ranked.
Thanks again.
When I go through the search pages, I just leave my mouse on the right side of the page as I use my mouse wheel to scroll. Maybe this will trick those sneaky researchers into thinking I almost clicked their ad’s! 🙂 It’s amazing the things technology is capable of today.
Hi Dave,
I don’t know if Google would consider using mouse pointer movements on pages that people viewed outside of search results pages – there are so many other variables to consider when you think about the wide array of designs and layouts on other sites.
I’d imagine search engines would love it if they could access information about where searchers were looking when viewing search results, to go along with the mouse pointer tracking. But that’s not something that’s easily tracked without actual hardware.
Hi Rob,
Thank you.
It’s possible that the jury is still out on whether or not Google might try to use a process like this, but I’ve seen a few articles in the past from them where they provided details about studies related to both eyetracking and mouse pointer tracking.
If you had mouse pointer tracking software on your web site, do you think it might potentially be helpful in learning more about what people did when they visited your pages?
Hi Lisa,
It’s possible that they might also be tracking how far down a page that you scroll as well.
Fantastic article. I think cursor movements are rich enough that if I watched someone’s cursor in a replay, I could tell a lot about what they were thinking… this seems like an area with lots of potential. I look forward to reading more research on this cursor tracking in the future.
Thank you, Casey.
I do think there is a lot of potential behind this approach, and I’m really happy to see some very recent papers on the topic – I’m hoping to see more as well.
That is so weird. Technology is becoming too much for me.
This is a fascinating article. I too question the value of tracking where the mouse points. I have restless arm syndrome and am, no doubt, causing the Google algorithm all kinds of grief. 🙂 There is a hazard, I think, of reading too much (or anything) to what a company does with its patent portfolio. It is easy to draw the wrong conclusions.
Nice post. What I did not understand though – is this only usable for google-hosted sites or is there any way to track other websites as well?
Hi Sterling,
I think it makes sense for a search engine to explore different ways that it might measure user-behavior data to see if it might be helpful in improving the results that are presented to us when we search. This particular approach is an attempt to find a way to fill a gap in the use of clicks when a snippet provides an answer, but doesn’t result in a click – which is something that makes sense to explore, even if the search engine eventually decides not to use it as a ranking signal.
Hi Sylvie,
Chances are that if a system like this might be used, that not all mouse movements from all searchers might be used, and that some of the data collected might be filtered out.
I agree that it’s possible to draw wrong conclusions from patents, but I don’t believe that they should be ignored. As a primary source of information, from the search engines themselves, about what they are researching, and about what kinds of assumptions that they might be making about searchers, search, and the Web, I think there is much valuable information that can come from paying attention to the patents published by companies such as Google, Yahoo, Microsoft, Facebook, etc.
If someone keeps in mind, when looking at a patent that it only provides a possible approach that a search engine might take, then there can be considerable value in looking at patents. I find the most value from patents not in applying something I read in one in a blind manner, but rather looking at the potential questions and issues that it provides, and the opportunities to research and experiment that it presents.
There’s a greater chance of potential harm from relying upon folklore and mythology and anecdotal evidence related to search engines that are based upon a limited knowledge of how search engines work.
Hi Dirk,
The idea behind this patent is that it might be helpful for the search engine to monitor how people move their mouse pointers on Google’s search results pages, and that the information gathered might be useful in helping them rank web pages in their search results, or change the order of ads presented, or decide whether or not they should show a specific Onebox (or universal search result other than web pages).
It doesn’t really apply to tracking mouse movements on other Google sites, or other sites outside of Google. There are mouse tracking programs that you can use for your own site if you might be interested.
This is not surprising but I did not know about it. I will think twice next time about what I am doing with my mouse on a page.
What next, will the colour of your shirt affect your search?
Hi SimplyCast,
I don’t think Google is considering shirt colors at this point, but who knows.
I was a little self-conscious about where I was placing my mouse for a while after reading the patent.