Biometric Parameters as a Ranking Signal in Google Search Results?

by

Sharing is caring!

Ranking using Biometric Parameters at Google

Do you search through Google on your phone? How do you know whether or not Google is watching you as you do and keeps on eye on whether you like the results you receive during your searches? Could Satisfaction with search results be a ranking signal that Google may use now or in the future? Are they keeping an eye on Biometric Parameters while you search, watching you through your phone’s camera?

A newly published Google patent application describes technology that would modify scoring and ranking of query results using Biometric Parameters of user satisfaction or negative engagement with a search result. In other words, Google would track how satisfied or unsatisfied someone might be with search results, and using machine learning, build a model based upon that satisfaction, raising or lowering search results for a query. This kind of reaction might be captured using a camera on a searcher’s phone to see their reaction to a search result, as depicted in the following screenshot from the patent:

Google biometric parameters

This satisfaction would be based upon Google tracking and measuring biometric parameters of a user obtained after the search result is presented to the user to determine whether those may indicate negative engagement by the user with a search result.

For example, someone searches for “Seafood Restaurants,” and the top result is a restaurant they had visited before and didn’t like, causing them to frown, which may be captured on their phone’s camera. That reaction may be seen as a negative signal by the search engine and could count against that restaurant ranking as highly for that query term. The patent tells us that such a reaction may influence search results for multiple searches:

The actions include providing a search result to a user, receiving one or more biometric parameters of the user and a satisfaction value, and training a ranking model using the biometric parameters and the satisfaction value. Determining that one or more biometric parameters indicate likely negative engagement by the user with the first search result comprises detecting:

  • Increased body temperature
  • Pupil dilation
  • Eye twitching
  • Facial flushing
  • Decreased blink rate
  • Increased heart rate.

The Biometric Parameters patent is:

Ranking Query Results Using Biometric Parameters
Inventors: Jason Sanders, Gabriel Taubman
Assignee: Google
US Patent Application 20160103833
Published April 14, 2016
Filed: February 28, 2013

Abstract

Methods, systems, and apparatus, including computer program products, provide query results using biometric parameters. One of the methods includes providing a search result in response to receiving a search query. If one or more biometric parameters of a user indicate likely negative engagement by the user with the first search result, an additional search result is obtained and provided in response to the search query.

Take Aways

When I think of how often I get my face right up on my phone’s screen while searching for something, the idea that Google might use the phone’s camera to capture my facial impressions as I’m looking at results doesn’t surprise me. Would Google use such Biometric Parameters signals to rank search results or build a model of biometric reactions to search results? It’s an interesting question. Instead of social media likes or dislikes, these rankings would be based upon what would be perceived as actual likes or dislikes.

Could you envision Google using a Biometric Parameter approach like this one in ranking search results?

Sharing is caring!

90 thoughts on “Biometric Parameters as a Ranking Signal in Google Search Results?”

  2. Hi Bharat,

    It’s an interesting idea, that Google has developed this process that is primarily based upon the use of a mobile phone, with a rear facing camera built into it. It shows that Google is interested in developing for users who connect to the Web using smart phones. And Google did develop the Android One (https://www.android.com/one/) which looks like it would possibly work with this process. The Android One was developed with India in mind, as a first smartphone. So maybe there might not be such a long way to go in India.

  5. Fascinating stuff Bill, definitely passes the “if I were Google” test.

    Hoping that Google’s implementation is a decade or two ahead of the current Microsoft Cognitive Services API before it gets treated as a worthwhile signal.

  7. Another cracking find BIll. I think it’s an interesting approach but would, I imagine, be pretty inaccurate. Those with facial problems, ticks or even those wearing glasses could cause a number of false positives (or negatives).

    I’d also be interested to hear why privacy advocates would say about it 😉

    I think the most interesting thing about this and other patents (like this one – https://www.seobythesea.com/2014/09/google-patent-watch-tv-as-ranking-signal/) is the variety of ways Google are investigating to optimise results.

  8. Is this can really become a search engine signals ?

    I mean what kind of biometric signals could be used by Google if I m searching for “Funeral Service Chicago”

    One friend is asking me if I m used to smile when I found a pizzeria 🙂

  10. I think it is possible but i am doubtful about its success because if i found something on google which i want that does not mean i will have a positive emotion on my face.

  12. Hey Bill,

    Just dropping several thoughts and two links. Long time fan of your work.

    People are usually emotionless in front of screens. They say a picture is worth a thousand words, so here’s two artists that have been trying to capture that void in emotions when people are plugged into the information devices: “People Staring at Computers” https://player.vimeo.com/video/25958231 and “Removed,” where a photographer who captures people using their mobile devices but post-edits and removes the devices http://www.removed.social/ .

    I’d also argue context also matters. If I’m in a hurry, I’m probably going to have my flustered face on regardless of the query. If I was interrupted or multi-tasking, then there’d be a confounding variable to my emotion that wasn’t due to my search result.

    So while interesting in theory, I’d be very surprised if this patent ever took off. Then again, maybe ML will figure out if changes to my iris signal satisfaction or dissatisfaction, all the while recommending that I go see a doctor.

  13. Hi Victor,

    Thanks for sharing those links and you research. Interesting stuff there. The patent refers to human raters a number of times, reminding us that Google has been working with humans in testing processes like this one out, and I suspect they would while testing the process behind this patent.

    Google acquired the company Wavvi, which did research before the acquisition on using sensors to detect if people were coming down with illnesses. So that recommendation to go see a doctor, if one comes out, might be worth listening to.

  14. Hi Toni,

    It’s interesting seeing possibilities like this being researched and considered. Will Google develop a process like this? We can be certain, but we know now that they’ve thought about it.

  16. Hi Walid,

    If you’ve had a site show up in a search that you had a bad experience at; you might possibly frown as a response. Interesting question about the funeral service – there may be questions that wil likely evoke some type of emotional response when searched for. I imagine that if such a system is developed, it should be tested carefully.

  17. Hi Andrew,

    If Google launches something like this, I imagine they will do a lot of testing with their many human evaluators. They would probably collect a lot of data about biometrics while they are as well.

    I suspect that they have turned this over to people who specialize in privacy issues as well, for their thoughts about how to present something like this to the public – it’s not going to reveal personally identifiable information to the public about any one person but I would like to see how they might address concerns about privacy.

  18. Hi Keval,

    I don’t know if Google’s concerned with showing results that people are positive about as much as they want to not show results that people aren’t very happy about (that is the impression I get after reading their example about a person’s reaction to a restaurant they didn’t like very much showing up in search results.)

  19. Walid raises a great point; I have to imagine that for something like this to work, it needs to be accompanied by a new method of segmenting queries.

    We already basically know that it’s done for commercial queries, where the topic+intent is timely (QDF), and likely others. Maybe we start to think about a Query Deserves Happiness?

    I’ll sit and watch for that headline to scroll by. Although it’s probably simpler. I suspect we’d just see Satisfaction compared with a more primitive signal like Task Completion Time to dynamically weight it in situations that it works better. Or instead, we’ll see more RankBrain press telling us that A/B testing the weightings of signals vs. queries using automation is too complicated for our non-AI brains to comprehend.

  20. Hi Bill
    Another great post Bill. I agree with Andrew in terms of privacy concerns. However how much attention do people actually pay to the privacy T&C’s on the apps, or possible “cross contamination” of data between apps. A lot of the data you are looking at here is already being collected by the current plethora of health apps and devices that are available.

  21. The implications of this technology are far-reaching. Privacy issues aside, it’s easy to imagine how this might factor into RankBrain or some of their other AI projects, as well as applications in IOT devices.

  22. It will not be legal “to spy” on us, and certain subjects entrainent more or less emotion, I have doubts about an approach like this one.

  24. Hi Alan,

    Thanks. The types of data collected by other apps may be similar, but it is definitely used in different ways, so I’m not sure how much of an argument that makes when it comes to the collection and use of that data. I think that the fact that such data wouldn’t be collected and used in personally identifiable ways is a better argument in favor of its use (but I’m still wondering how many people would still find it invasive anyway).

  25. Hi Aurelien,

    That is the reaction that I suspect that many people will have to the process described in this patent. I guess we will wait and see if it is something that Google tries to develop and use.

  26. Bill, I love this blog for some of the patent insights it brings. I’ve been reading about VR and MR (mixed reality) today, where they could go etc. Combined with wearables, retina recognition, cameras on devices, hell even IoT data, tech companies like Google are going to be looking at how they can interoperate our ‘real world’ movements to their benefit more and more over the coming decade.

    Wouldn’t be surprised to envisage a scenario where you search for a food product on Google, and Google then looks out to see if that product ends up in your kitchen as indicated by an RFID chip in the packaging, using that as a positive ranking signal if it does. Fascinating, maybe a little frightening too.

    Liam

    (PS – has that been patented? If not, I’m on the case!)

  27. Hi Bill,

    Interesting feature to implement. Do you think if one were to run an experiment if this rolls out, to understand if it really is capturing the user’s need to be satisfied. A small one like the one Rand Fishkin had tried to understand the true nature of Pogosticking, which also had a lot to do with the idea of understanding if the user was satisfied with the result. But this obviously works on a greater level to understand the satisfaction parameter. I wish it were possible to pick up focus groups of 20 around the globe and try to look for a generic query and show a consistent disregard for the query that ranks first and see if the search result is affected in any way. It is just interesting to understand how much power the user has over the results, now that such a feature is out there.

  28. It is really difficult to satisfy every audience for their search term, they are bound to go and check other articles available in the web. Checking their like & dislike for a article will be difficult task for google. But lets hope the best article finds the top place in the google search. Things are also going to be tough for SEO experts. What I think is to keep the things simple and to the point, instead of having all those wasteful ticks to rank. Thanks for sharing !

  29. I’ve never heard about this but it seems perfect to me. If this tecnology is already ready to run there is nothing more sensitivy than capture the person reaction. Congratulation for the article.

  30. Hi Pedro,

    It’s difficult to say whether or not Google has put this into place, or whether they might say something about it if they do try to use it. I guess we wait and see.

  31. Hi Harish.

    The focus of the process behind this patent seems to be to capture an emotional response, if one exists, to a business behind a website, rather than the website itself. It doesn’t involve any tricks for ranking of sites, nor any manipulation or spamming of the search engine. For a positive response, it seems like the site behind a result would have to create a positive response to a person viewing a site in search results.

  32. Hi Ashwati,

    The process behind this patent is to measure an actual response from a searcher to seeing a particular business in search results in response to a query; not to guess at one based upon a metric like a quick return to search results, like what might happen when pogosticking takes place (which is still only a guess at whether or not a visitor to a site enjoyed or didn’t like a site.

    The patent talks about using human raters to build a model about sites with, so Google is possibly experimenting with their own focus groups. Difficult to say how much control a searcher has over the ranking of results based upon this method. But if they don’t like one of the results based upon a possible bad experience with them in the past, they may not like seeing that result for a query to begin with.

  33. Hi Liam,

    I can’t say that I’ve seen a patent about food products, like you surmise, but that is an interesting idea. It’s possible that the internet of things, with sensors in household appliances may lead to a search engine producing a device like you are suggesting. It’s possible that we may see something like that developed; or you may develop it. 🙂

  34. Hey Bill

    If this is true then this is truly amazing, imagine capturing a persons expression due to search results and some kind of AI understanding that expression! Its actually a little creepy to be honest that Google would even apply to patent this.

  36. I would like to encourage all of you really like your blog. Did you design this website yourself or did you hire someone to do it for you? Please reply as I’m looking to create my own blog and would like to find out where u got this from. Many thanks bookmark this page to your most used service to help get the word out.

  37. Well, its been evolving all the time, so can’t be predictable whats behind the wall. Though good piece of content.

  38. Hi Bill,
    Very interesting, but wow it’s really scary (for an european guy :-))
    I do not think Google will ever use it in Europe, we have so many legal locks!

    David

  41. Hi David,

    Google doesn’t say that they would in any way share this information, which definitely would be something I think most people concerned with privacy would be upset about. We don’t know if Google will ever start using this process, but it’s possible that they might. I guess we wait and see.

  42. Thanks,

    It’s interesting seeing something that surprises a little like this from Google. We’ll see if it’s a direction that Google will take.

  44. Hi Jack.

    I don’t think applying for a patent in itself is a creepy step. The idea of measuring emotional responses isn’t either. If they had proposed within the patent that they would use images collected from this process, that would be creepy. 🙁

  45. Maybe it’s related more to a device like glass where the tech can have sensors closer to facial muscles/ pulse etc to measure response.
    You can already see the blog posts….10 ways to raise your visitors biometrics. 😆

  46. That’s really fascinating. I believe it’s for the future, though, but with every new system like this life we are living more and more looks like Sci-Fi movie.
    And to add a bit of fun into this, I’m thinking about situation where searcher find some really ugly website and laugh, so Google can misunderstand laugh and rank funny-ugly website higher. 🙂

  48. Hey. Interesting post but I don’t want Google to capture my facial impressions. I hear many news about that and someone says that G already trying that type of spying. So how we could protect ourselves ?

  49. Very interesting article Bill! Infact, I have been reading quite a bit of your blogs recently and I should its totally stuff that I dont find in many places and YES I believe satisfaction is going to be a ranking signal in the future. Further to some of the points you have mentioned, there are a few companies that are working on BCI (Brain Computer Interface) to study customer satisfaction.

  50. Hi Vinod,

    I try to keep an eye on what is being posted about Google and search engines in other places; and provide information here that is unique if I can. Yes, I’ve seen some articles about Brain Computer Interfaces, but for much more than just studying customer satisfaction. We do live in interesting times.

  51. Hi Alexes,

    I’ve read a number of articles that talk about people taping tape or post-it notes over the cameras that are on their desktop computers. I don’t know what is available out there to make sure that the camera on your phone isn’t on and your image is being captured; but I suspect there likely is something that is available if you look. I was surprised to come across this type of recording of expressions; and curious if Google would tell us whether they were using it, and what words they would use to do so.

  56. Awesome article again Bill. Doesn’t surprise me this is something they are looking at. We are well aware they are already looking at things like CTR and user engagement on a website through analytics / return to SERP rate – why would they not do what they can to see how the other results are faring up. Creepy how big brother it is, but deffo something to consider. Again – thanks for the awesome content you put on here, I never spend the time to thank you after taking away so much. Good luck with everything you are up to and look after yourself.

    Kyle
    Gorilla Marketing

  58. Gosh, I’m not really sure I like that idea. It feels a bit like they’ll be spying on me more than I’d feel comfortable with. Technology is awesome but sometimes it’s taken a step too far.

  59. Hi Claire,

    I guess it is possible to take things a little too far. It’s still an interesting idea, but I suspect that Google might think about this, and decide that people might consider it to be a little too intrusive. I guess we wait and see if they come out with it.

  62. @Bill,

    This is no doubt a very great technology by observing the user intent and psychological behavior, emotions and sentiments G can get to know which query is most valuable and which results should not come much often on a particular set of keywords. This is a kind of good judgment process G has initiated. And now its more like human than a bot which can think and make decisions as well. Good Going Google and nice post Bill! 🙂

  64. Hi Bill,

    Very interesting idea/concept.

    Personally I’m not sure users would want their facial expressions to dictate how Google serves them results and in which order. If you have a face like the grumpy cat then surely it will mislead Google, lol.

    Google are also always commenting on respecting people’s privacy. I personally wouldn’t like it recording my facial expressions through my mobile phone.

    However, this is Google, as some of the comments have stated, if they can gather even more data on mood, gender, expressions etc. then they probably will try to do it just to get even more data.

  65. Hi Rob

    I wonder if Google starts using a process like this one, if they will tell us whether or not they are using it. I’m guessing they might not.

  66. Hi AAMIR,

    Thanks! This patent appears to be focusing upon actual reactions from people who might have experiences with places that might appear in search results, rather than just keywords – it is a matter of an emotional response to a thing, rather than to a string or words. It really is a consideration of whether someone might really “like” something listed in search results, rather than pushing a button to show their like or dislike.

  68. Yeahh…It may be in future. But sounds good. Google is updating daily it’s search algorithm and trying to provide better search results to users.
    So I don’t think it is a shocking news.

  69. First Lets Look at Pros and Cons of this technology
    pros:
    1. Improved search result
    2. satisfaction

    Cons
    1. Privacy Issue
    2. Wrong capturing of facial expressions may result in search query variation
    Above things to taken care of for effective utilization of technology

  70. Isn’t this what they tried to achieve using google plus – I have no idea why they dropped this in the search, or authorship for that matter …

  71. Hi Ben,

    Google+ gave Google a chance to have people share content from the Web, and interact with others and learn about their interests and their expertise. It was a different approach than this Biometric approach, used for somewhat different reasons and in different ways. Google+ just turned 5 years old, and it is still ongoing. It is not a ghost town, and likely provides Google with significant data and information about people who use it and interact with others. We don’t know if Google is using this biometric approach; but if they do, I wonder if they would tell us whether or not they are.

  73. Wow, this is very interesting. I wonder what exactly they’ll do with this. It seems very weird and kind of creepy for them to use the camera to monitor user’s facial expressions.

  74. Haha. That sounds pretty good to me. If this is put to work, its really gonna help bring down irrelevant sites from the top that are always hanging there. I kinda like it though. But looking at how its gonna work, there would be complaints of users privacy being invaded. Google wouldn’t just be seeing your face alone unless the camera is just focusing on your forehead.
    Since most Android phones with internet browsing dont have good front facing cameras or dont have it at all, this might pose some limitations to the technology.
    I was like, how can all these be gotten from the front camera?
    1..Increased body temperature
    2..Pupil dilation
    3.. Eye twitching
    4.. Facial flushing
    5..Decreased blink rate
    6.. Increased heart rate.
    Wouldnt it lower the system speed?
    Will an average or midrange device with poor front facing cameras be able to cope?
    Will it force oems to make all their phones possess advanced cameras or if possible iris recognition technology?
    This i not going to be accepted worldwide especially in security conscious regions. All the same, its gonna be cool.

  75. Hi Christianihechi,

    It will be interesting seeing if Google adopts this satisfaction approach using a phone’s camera to measure biometric indications. I imagine that it’s possible that Google could move forward with this approach even if only a small percentage of phones could possibly be used in this process. We don’t know if Google will use this – or if they will announce that they are using it, if they decide to start.

  78. It’s really interesting to know that google is trying its best to deliver all these services to us. But I have a question in mind..
    Don’t you think that after the implementation of facial expression, the search engine would show only the results based on our queries and searches that we made on google in the past and won’t show us the latest generic results?

  79. Hi Richa,

    There’s nothing in the patent that would indicate that the use of biometrics as described in the patent would negatively impact previous searches or present day searches. It is also difficult to tell exactly how a new process might impact search; but it’s likely that Google would work as hard as possible not to let the introduction of a new process have too profound of an effect upon searchers – I don’t think they would want searchers to think that search is broken, and would avoid that as much as possible.

  80. Food for thought but there are so many variables that I think it would be difficult to implement in the real world. Wouldn’t put it past them to try something like this though.

  82. Hi Abhishek,

    It will be interesting if Google uses biometric information as a ranking signal. Not sure if they would tell us that they are using it. if they decide to.

  83. This is an interesting idea. However there are issues surrounding personal privacy. If Google could access your phone’s camera then who knows when it would and wouldn’t be watching you.

  84. Hi BCKwebsolutions,

    That is part of why I decided to write about this patent; so that more people would learn about it I suspect that most people don’t like the idea of their phone spying upon them.

  85. I’m telling you right now that Google is an advanced animal! This would not surprise me ONE BIT to find out that Google could use biometric rankings for search results!

    It’s amazing watching this animal grow! SEO is an ever growing field! I wonder where SEO will be in ten years?

  86. Hi Eric,

    It’s difficult to believe that Google is 20 years old at this stage. There seem to be so many possibilities that it is difficult to guess what Google will be like in 10 years – I guess we wait and see.

  87. Hi Bill,

    Another interesting read, I am a massive believer in searcher satisfaction and engagement. But to think that I would have to produce content that makes a searcher smile to increase rankings, I feel would be a step too far 🙂

    James

  88. The ability to do it is one thing. Whether it is the right thing to do is another. I find the idea that the back facing camera on my mobile may eventually spy on me for Google totally repugnant. Because for one thing, there would be more behind this than search satisfaction. Imagine what the data gathered would be worth to remarketers.

  89. Ultimately, Satisfaction is one of the key metrics Google are going after. If they can provide the best user experience for their users they are going to keep winning although they are going to have find other ways to measure success other than just a smile.

  90. Hi John,

    They do mention a number of biometric indicators of satisfaction that go beyond just a smile. Will they use this particular approach? We don’t know.

Comments are closed.