Sentiment- a general feeling, opinion, personal judgment, feeling, or sense about something.
At Google’s recent Searchology presentation, one of the new features described as being used by Google was sentiment analysis.
In the recap of the event from Google’s Matt Cutts, he tells us that:
If you sort by reviews, Google will perform sentiment analysis and highlight interesting comments.
I’ve seen many papers from Google on sentiment analysis and a recent patent filing, so I decided to look closer at some of those review search results.
Google Review Examples
For some search results, when you choose the “show options” link after your search and then the “reviews” link, you may see quotes from reviews in the snippet area for search results, surrounded by quotation marks. Testing this, I did see some results where the snippets were in quotation marks, and when I visited those pages, that quoted text tended to be mostly from actual reviews. I looked at some reviews for restaurants, music, and products.
Here’s an example from one result on a search that I did for [new york seafood restaurants]:
“Aquavit, which is located in one of the famous family’s former townhomes, maybe your best bet.” … “Save room for one of the tantalizing desserts.” … “With a menu almost exclusively devoted to seafood, Aquagrill is an excellent pick for diners who want great choice and unparalleled options.”
On a search for the band [Led Zeppelin], the following quotes were culled from two different reviews on Amazon.com where there were several reviews:
“This was Led Zeppelin’s finest hour, and therefore rightly holds the claim to #1 album of all time.” … “I own it and have listened through it over a hundred times, so I am more than familiar with it, along with the rest of Zeppelin’s music.” … “Four Sticks is heavier, but nothing exceptional.”
On a search for [green cleaners], these quotes were pulled from a couple of different reviews on one page:
“Overall, I feel good about using these products.” … “The other seemed to work ok, but overall I really recommend Clorox brand GreenWorks instead.” … “I am hoping with continued use, it will also assist in eliminating the mold stains in the grout lines.”
Exactly why did Google choose the particular quotes that it shows?
Sentiments by Different Aspects
One recent paper from Google describes some of the thought processes that might explain why certain statements may be included. In Building a Sentiment Summarizer for Local Service Reviews (pdf), we’re shown the following example of quotes from reviews broken out by different aspects, such as “service,” “value,” and “general comments.” Aspects are defined in one of Google’s papers on sentiment analysis as “properties of an object that can be rated by a user.”
The abstract for the paper tells us that:
In this paper, we present a system that summarizes the sentiment of reviews for a local service such as a restaurant or hotel. In particular, we focus on aspect-based summarization models. A summary is built by extracting relevant aspects of a service, such as service or value, aggregating the sentiment per aspect, and selecting aspect-relevant text.
So, when we are shown multiple quotes, one goal that Google may try to reach is to provide sentiment information about different aspects of an item or service.
Other Google papers on sentiment analysis also worth looking over include:
- Comparative Experiments on Sentiment Classification for Online Product Reviews (pdf)
- Sentiment Summarization: Evaluating and Learning User Preferences (pdf)
- A Joint Model of Text and Aspect Ratings for Sentiment Summarization (pdf)
Google’s Patent Filing on Sentiment Analysis
The patent application is interesting because it provides some information about how Google might choose text from reviews to present. The patent filing appears at:
Domain-Specific Sentiment Classification
Invented by Tyler J. Neylon, Kerry L. Hannan, Ryan T. McDonald, Michael Wells, Jeffrey C. Reynar
Assigned to Google
US Patent Application 20090125371
Published May 14, 2009
Filed August 23, 2007
One of the document’s primary focuses is describing how different words or terms that may appear in reviews may have completely different meanings when applied to different products or services. A couple of early examples illustrate this very well:
The word “small” usually indicates positive sentiment when describing a portable electronic device. Still, it can indicate negative sentiment when describing the size of a portion served by a restaurant.
Thus, words that are positive in one domain can be negative in another.
Moreover, words which are relevant in one domain may not be relevant in another domain. For example, “battery life” may be a key concept in the domain of portable music players but be irrelevant in the domain of restaurants.
The abstract from the patent filing provides a pretty high-level overview of what the document contains:
A domain-specific sentiment classifier that can be used to score the polarity and magnitude of sentiment expressed by domain-specific documents is created. A domain-independent sentiment lexicon is established, and a classifier uses the lexicon to score the sentiment of domain-specific documents.
Sets of high-sentiment documents having positive and negative polarities are identified. The n-grams within the high-sentiment documents are filtered to remove extremely common n-grams. The filtered n-grams are saved as a domain-specific sentiment lexicon and are used as features in a model.
The model is trained using a set of training documents which may be manually or automatically labeled as to their overall sentiment to produce sentiment scores for the n-grams in the domain-specific sentiment lexicon. This lexicon is used by the domain-specific sentiment classifier.
If you want to dive deeper into the actual processes behind how different sentiments are identified for different kinds of products or services, you may want to spend some time with this patent filing and the papers I linked to above. I’d also recommend looking at many reviews for products and services in different areas to get an idea of how Google uses sentiment analysis in actual practice.
Hi People Finder,
Very good point. I may have some more on that sometime in the very near future. 🙂
Some sentiments are more equal than others… It will be interesting to see how Google determines which sentiments on which sites are worth using in their sentiment analysis of sites, products and services. With all of the online venues available nowadays for opining, it will be cool to see how Google determines its “Sentiment Rank”
Very interesting.. I wonder what they do with words that could be interpreted as both negative AND positive? Ignore them or look for additional context?
Please forgive my ignorance with this question.
Doesn’t the current reviews scoring rely heavily on the base “5 star” system rather than physically/literally interpretation sentiment?
I am not in any way shape or form saying that a level of interpretation doesn’t take place, I am simply curious to understand where the line blurs between simply taking an average of the “5 star” reviews and calibrating the algorithm to identify syntax/context.
Hi jlbraaten,
I guess that might depend upon how many reviews might be found for a particular product or service, and how many statements are included in those that address different aspects of the products or services. If there aren’t any that address sentiments at either exreme, it’s possible that more neutral reviews might be included. It’s worth spending some time lookings at reviews that are included in snippets and taking a look at where they originated from.
Hi Robin,
Good question. In my search above for [new york seafood restaurants, there were a number of 5 star reviews, yet two of the three statements came from 4 star reviews instead. I don’t know if that means that the number of stars may be less important than some level of semantic analysis, but it might. The best way to answer your question might be to start looking at a number of results that include reviews, and look to see where they are taken from.
Automating such analysis has been done by various companies in the past-a more detailed analysis of the google application probably is required. Based on my past experience, it seems that while some of this can be automated, depending on the degree of accuracy, it is but impossible to eliminate human element. While accuracy may not play a huge factor in less critical areas such as restaurant review, companies have known to use such sentiment analysis to track product pricing patterns etc based on supply-demand patterns etc. In such cases, accuracy is of far greater importance and I’m not sure if an automated solution would really work. After all, the human brain is of course the most sophisticated computer.
@Bill: great post. At Center’d we have been pushing forward on this idea to make local content more consumer friendly. We also consider sentiment into search as @People Finder suggests.
@RAM: agree with you – if you want a deeper analysis it gets tough/expensive to build out the ontologies, training sets etc to get accurate.
Hi Ram,
Thank you for your thoughtful comments. It is an exciting area, and a difficult computing task. I do think that breaking the task of sentiment analysis into different categories, or domains, isn’t a bad idea. But many of the challenges still remain to anyone who would try to automate the proces. I was tempted to break the patent filing out in more detail, but realized after reading it that such an analysis might end up being longer than the patent application itself.
Hi Chandu,
Thank you – a pleasure to see you here. I haven’t had the chance to see centerd before – I’ll be spending some time with it now, and seeing how you use sentiment in searches there. Making local search more consumer friendly is a great goal, and one I hope that you’ll accomplish.
Very interesting. I love your blog, you research out topics thoroughly instead of simply talking about status quo. It’s amazing how in-depth the software is getting, but as I can see there is always room for improvement.
Hi Joel,
Thank you very much for your kind words. At some point we may see Google attempting to use sentiment analysis in areas other than reviews. RAM described some other uses in his comment that are likely much more difficult. I’m looking forward to seeing where they might try to take it.
Joachims, Granka and Pan have done a particularly interesting paper on the topic, titled “Accurately Interpreting Clickthrough Data as Implicit
Feedback”. They did a study using eyetracking on search engine users to do things like sentiment analysis. It also describes how someone with access to user clickthrough logs (such as Google – or anyone, using that leaked AOL data from a few years back) can start to judge user opinions of sites. There’s a PDF at http://seorant.ath.cx/joachims_etal_05a.pdf and the follow-up work is also good, if that paper held your interest.
Hi SEO Ranter,
Thank you. I really like studies like the one that you’ve pointed out. The focus of that one seems to be more on a bias that may come about based upon how search results are shown to searchers, such as a large amount of trust imparted to the first search result. I’ve seen a number of papers from the authors of that PDF, and they’ve all been pretty interesting.
One of the papers that I linked to above, “Sentiment Summarization: Evaluating and Learning User Preferences” does cite a paper by one of the co-authors of the paper that you’ve linked to – Thorsten Joachims. That paper, Optimizing Search Engines using Clickthrough Data is cited in the “Sentiment Summarization” document as describing a way to help improve the sentiment summaries shown to searchers. It’s pretty interesting to see how much of this kind of research can tie together.
Great blog, its fasinating how capable the systems are becoming in such short spaces of time. I would think the next step would be to attempt to cater for all types of searches, using one system, although its easier said than done, clearly.
Hi Rak Design,
Thanks for your kind words. It is pretty exciting to see how search engines and the Web are evolving. Things like “real time” search (or near real time search) combined with social networks like Twitter are spreading information so quickly these days. News outlets are having a hard time keeping up.
I’m not sure that creating a single system that can handle all kinds of searches is always the ideal approach, though.
If I’m narrowly focused upon achieving one type of task, such as finding a local pizza place, I don’t necessarily want to see pizza recipes, or pictures of pizza, or videos of people making pizza, or web pages about pizza. I just want to be able to find a place to grab some slices, and probably quickly too. 🙂
That may be true with some other kinds of tasks as well.
Hiya Bill! I haven’t been to your website / blog for many many moons, and am impressed that you’re still at it, and dishing up the very best information and analysis. I like your choice of band, as Led Zeppelin rate among my all time favourites. Stairway to Heaven is a track that still moves me, even after listening to it 1000’s of times already. On to the important stuff though. How would an internet marketer target reviews in such a way that Google will use your specicific review?
Hi Jacques,
It’s good to see you. The Led Zeppelin reviews were interesting – I couldn’t pass them up.
There’s some new recent stuff from Google that I haven’t had a chance to get to yet, but will be. 🙂
Hopefully it will give us some ideas.
Amazing blog! Thanks to you I learned that you can draw a deep knowledge of the patents, which are publicly available. I do not know where you’re all that knowledge and why is it so keen to share. Do you have Polish roots?
I dont get it. Lets say there are 4 main sentiments expressed on a web page. How will Google represent/rank this and how will this influce the SERPs?
Hi SF,
Thanks.
Hi Dirk,
I’m not sure how connected the ranking of results, and the display of sentiments in snippets or summaries is, but it’s possible that some results in the “review” option that Google provides may take into account how strong the sentiments expressed might be.
Google also shows reviews in local search, and the choice of what to show from different reviews may be influenced by what sentiments are contained in statements about the objects or services written about.
I’ve followed up this post with another – Opinion Summaries in Google Maps Reviews, and I’ll have another on the use of sentiment statements in snippets in the near future.
This patent filing didn’t give us much insight into how sentiment analysis might play a role in the ranking of reviews. The others don’t either. But they do provide some ideas about what might be included in summaries of reviews (possibly from different sources) and snippets from reviews. And we know that people will often choose which results to actually visit based upon what they see in search result snippets.
This is all very interesting. We all know google is rampantly trying to deliver the most relevant results for searches, but how are they going to incorporate sentiment analysis into the already complex algorythm equation? I think this is going to end up being a very interesting year to see how Google and Binghoo adjust to each others changes in search technology.
Hi webheadz1,
At this point, it looks like they are using it for their reviews options, and possibly in their local search reviews as well. It’s hard to tell how they might continue to expand its use further.
It appears this year is going to turn the search engine world upside down. Sentiment Analysis, Rel No Follows, and from what I hear Goog is working on geo targeted results to be a major factor in their new algorythms. This will be a great move for local businesses, but could wreak havoc on large companies competing for a nationwide market. Thanks for the great post and analysis.
Really interesting article, and I’ve also heard talk of the importance of Geo targeting…
Hi Tony and Identity Design,
I’ve seen some geo targeting in the results that Google shows. At the top of the results for a few searches that I’ve performed has been a message that says something along the lines of, “These results have been customized based upon your location in XXXXXX.”
I’m not sure if this will become more common, but I have seen a few.
Also, some searches may show automatic local search results for a location near you. For instance, I search for “pizza,” and I see a local result blended into the web results that includes local pizza places.
How will this geo targeting affect companies selling services and products nationwide or globally? Will the searcher have an option to search locally? this could be detrimental to larger companies expanding nationally. wow, what a head ache this will be for web masters.
Hi JT,
The post isn’t about geo targeting, which is a topic that I discuss in considerable detail in other posts here. Geo targeting can affect companies locally, regionally, and globally, and in some instances provide a searcher the chance to explicitly search locally or globally, while in others geo targeting happens behind the scene.
Hi Bill,
May be I’ll look Stupid, but are people really using this new features that google provides?!
It will be a great help if you can enlighten me with some info or statistics 🙂
Regards~Tonny (BTW the second link in your article is broken “Building a Sentiment Summarizer for Local Service Reviews”)
Hi Tonny,
If you click on the “show options” link at the top of a set of search results at Google, and you click on the “reviews” link in the left navigation, you will see reviews related to your query terms. If you search in Google Maps for a business, you also have the option of looking at reviews for a business location. The reviews don’t look like the screen shot from the patent application, but Google did say in the Searchology presentation that they gave earlier this year that they would be using sentiment analysis with the reviews that they present.
Thanks for the heads up on the broken link. I’ve located that document at a different URL, where it is still available.
I find it amazing how a google algorithym can almost process sentiment and intent like a human to figure out the most relevant results for a search. Im eagerly looking forward to seeing how yahoo and bing will combine to compete with Google. Thanks for an enlightening article.
Hi Lee,
I agree with you. It is amazing. Thanks.
One of Microsoft’s more interesting acquisitions was that of Powerset, which has developed a lot of ideas around understanding semantics of words, including sentiment analysis. See:
Microsoft Bing, with Powerset Inside, and my description of the patent filing “Calculating Valence Of Expressions Within Documents For Searching A Document Index” on that page.
Hi Bill,
I’ve noticed this addition. I think it’s a good idea. I like the way google keep adding things to the search results, I think it makes them pull further away from yahoo and bing. I feel google have moved forward for the last decade while the rest have stood still.
Hi Peter,
I do like the review approach – it does give us a different view of pages that we might find on the Web. I’m not completely convinced that the other search engines are standing still, but it is fun looking for the changes, and comparing them.
Do you think Google is eventually moving towards using “+1 tagged” content as being a sort of broader reviewer system? Anything that helps to improve the quality of search results above and beyond the content farms is a move in the right direction from my perspective!
Hi Wes,
I’m not sure that just a “+1” does enough to provide sentiment, but I think a company could match up a positive upvote like a +1 with information about people leaving those +1’s to get a sense of who different pages or content might appeal to.