New Sentiment Phrase Snippets
When you look at web page search results for a Web search, there are usually three important elements displayed for each page. One is the page title, which also acts as a link to the page. Another is the URL of the page, which sometimes gives you a hint of what you might find at the page if the URL shows a meaningful directory category or two for the page.
The third element is a snippet of text for the page, which describes some type for what you might find on the page. This is sometimes taken from the meta description for the page if it contains the keywords you searched for or possibly a synonym for one or more keywords, or often some text from the page itself if that snippet of text contains the keywords or synonyms for those. Snippets aren’t limited solely to Web search results, and recently a new type of snippet has been showing on Google Place Pages, as highlighted in the image below:
A Google patent granted this week provides a detailed look at short snippets like these related to businesses and products and gives us some insights into what the people at Google may be thinking about when they come up with the snippets. These types of Sentiment Phrase Snippets were the topic of a blog post from Mike Blumenthal titled Google Places Descriptor Snippets. I’m not sure that I like the name “Descriptor” from Mike’s name to describe them – it reminds me of Transformers for some reason (or would that be “descripticon” snippets)?
Mike mentions in his post contacting me to ask about a Google patent I wrote about recently on how Google might categorize queries and websites, and boost web search rankings for those pages which shared categories with the queries. While I responded to Mike, telling him that I wasn’t sure about the relationship between that patent and new Google Places snippets, Google published this new patent after our correspondence, and it seems like a good fit for the new Google Places snippets that appear upon Place Pages. I sent Mike an email this morning, and I’m looking forward to his thoughts.
Using sentiment analysis in snippets to describe a business or product does seem like a good idea, and this patent discusses why Google thinks doing that would be both useful and important:
A snippet is a segment of a document used to summarize an entity or document associated with search results. Snippets allow the users of a search engine to quickly assess the content of the search results to identify the search results that are of greatest interest to them. Snippet text is usually selected based on keywords, word frequencies, and words or phrases that signify summarization such as “in sum” or “overall”. Snippet text is also selected based on many other factors including the length of the snippet as defined by the size of the display.
Users of search engines often perform searches for entities such as hotels, restaurants, and consumer products. These entities are considered “reviewable” as public opinion or sentiment is often expressed about them in websites such as review websites and personal web pages. For reviewable entities, sentiment forms a special type of summarization. Consequently, the sentiment expressed in one or more reviews provides valuable information for inclusion in snippets generated for reviewable entities.
Sentiment information included in snippets should be representative of the opinion expressed about the reviewable entity over several reviews while including non-redundant sentiment information. Further, sentiment information should be readable and easily understandable. Lastly, each piece of sentiment information should be as concise as possible to allow for the inclusion of the maximum amount of sentiment information for each snippet.
The patent tells us that the focus upon these sentiment phrase snippets is to provide several sentiments about specific entities extracted from documents such as reviews, in as concise language as possible. The example that Mike displays in his post for Google’s headquarters doesn’t seem to do that very successfully, but my pizza example above does tell me a lot about the Pizzeria in very few phrases.
Phrase based snippet generation
Invented by Sasha Blair-Goldensohn, Kerry Hannan, Ryan McDonald, Tyler Neylon, and Jeffrey C. Reynar
Assigned to Google
US Patent 8,010,539
Granted August 30, 2011
Filed January 25, 2008
Abstract
Disclosed herein is a method, a system, and a computer product for generating a snippet for an entity, wherein each snippet comprises a plurality of sentiments about the entity. One or more textual reviews associated with the entity is selected. A plurality of sentiment phrases is identified based on the one or more textual reviews, wherein each sentiment phrase comprises a sentiment about the entity. One or more sentiment phrases from the plurality of sentiment phrases are selected to generate a snippet.
If you’re interested in how these sentiment phrase snippets might be selected, it’s worth working through the patent, but I do want to also point to a paper that two of the co-inventors of this patent were co-authors of involving snippet selection for reviews for businesses. The paper is Sentiment Summarization: Evaluating and Learning User Preferences (pdf) by Kevin Lerman, Sasha Blair-Goldensohn, and Ryan McDonald.
A previous post I wrote about reviews and sentiment analysis links to some other Google papers and patent filings about sentiment reviews as well: Google’s New Review Search Option and Sentiment Analysis. It’s interesting seeing how Google is experimenting with expressing sentiment in snippets from reviews for products and businesses, and how their display of that information is evolving or transforming over time.
One of the areas that the patent spends some time with is in distinguishing between structured and unstructured reviews.
A structured review is one that follows a specific format, including a defined rating and/or textual review of a specific business or product:
A structured review will typically have a format such as:, “F-; The pizza was horrible. Never going there again.” In this instance, F- corresponds to the rating, and “The pizza was horrible. Never going there again” corresponds to the Textual Review. Structured reviews are collected through the Network from known reviews web sites such as TripAdvisor, Citysearch, or Yelp. Structured reviews can also be collected from other types of textual documents such as the text of books, newspapers and magazines.
On the other hand, unstructured reviews are text-based pages that include a reference to the product or business being “reviewed”, and also have a high likelihood of including an opinion about those entities. Those aren’t structured like the reviews that might be found at Yelp or Citysearch and might be from webpages, blogs, newsgroup postings, and other sources.
The process behind this patent attempts to find reviews, either structured or unstructured that can be associated with specific people, places, and things about which an opinion is being expressed to extract sentiment phrases about those. In other words, words or text about things like restaurants, hotels, consumer products, books, films and so on, that express some kind of attitude about them such as an opinion.
The patent tells us that these sentiment phrases are “short, easily-readable phrases which provide a synopsis of a Textual Review,” and provide examples such as: “great setting”, “clean rooms”, “fantastic debut”, “an interesting book”.
These phrases are identified through many natural language processing (NLP) techniques, and it’s worth drilling down into the descriptions of those in the patent if you want to get an idea of how they might be selected from different reviews. I will probably be spending some time over the weeks to come exploring different place pages and the rest of this patent to get a sense of how Google is collecting snippet phrases to display with some real-life examples.
If you decide to do the same, please let me know if you come up with something interesting, like the odd examples that Mike found with the place listing for Google’s headquarters.
Is there any indication in the patent that they are using other sources for sentiment besides reviews? Like Link text?
Bill,
Have you seen the Google Mountain View Places Page? Some of the Descriptive Terms are Western Union and quick cash and quick cash!
Hi Mike,
I haven’t seen anything in the patent that states that they would give some type of special emphasis to text found in links because it was found within a link, but as long as the link text is part of a document that might be possibly said to express some type of attitude or opinion about a specific product or business, then it could potentially undergo the kind of extraction and analysis described in the patent like any other text found on the same page..
One of the reasons that I pointed out that structured and unstructured reviews could be used is that not only might we see phrases taken from the usual suspects when it comes to structured reviews such as can be found at sites like yelp or citysearch or elsewhere, but they might also be found on other webpages as well, including newspaper and magazine articles, blog posts, and possibly even user generated content sites (Google Plus reviews of restaurants, businesses, etc.).
If there’s a mention of a business and a link pointing to the business, that might be one way that one of these “unstructured” reviews might be found, though the patent really doesn’t go into how they might locate reviews. But, a review could be a formal one at a review site of some type, or it could be any page on the web that appears to express some type of sentiment about a particular person, place, or thing.
What we are seeing differently than the previous incarnation is more descriptive snippets and not so many sentiment snippets (thus my naming 🙂 ). In the case of Google PLaces listing the use of “Western Union” which seems to come from their help files or in the case of Barbara Oliver’s listing the misspelled word “jewelery” which seems to have come from an inbound link. Neither case is it easy to come to the conclusion that Google is mistaking a web page for an unstructured review.
In the Lat-long Blog they note:
These phrases come from sources all across the web, such as reviews, web pages and other online references, and they can help people quickly identify the characteristics that make a particular place unique.
Your thoughts?
Hi Mike,
From reading the patent, an “unstructured review” doesn’t have to be called a review, or even be a review, and perhaps that’s a failing of the algorithm that things that clearly aren’t reviews might be seen as reviews. From the patent:
The definition seems to be quite simply that it needs to be a text-based document referencing a “reviewable entity” that may contain an opinion about that entity, and that seems to match up very well with the description in the Lat-long blog that you point out.
What really stands out for me on that page (http://www.google.com/adsense/support/bin/answer.py?answer=65789) is that it’s one of the very few pages I think I’ve seen that has such complete address information for google: “Google Inc. 1600 Amphitheatre Parkway, Mountain View, California 94043, USA. Phone Number: 650-253-4000”
While there’s not any “real” sentiment that a human being might say is attached to that page, the level of confidence that page is about the entity that is Google at that specific address is extremely high.
Hi Urban,
I have had a chance to see the Google Mountain View Places Page. Mike posted a screen shot of those in his post. I didn’t write the algorithm explained in the patent, but it’s not hard to imagine that Google’s algorithm would choose to grab snippets from that page after doing some investigation. 🙂 Have you had a chance to read through the patent carefully yet?
Rather than question me about what I’ve seen, it might be more productive to read through the patent that Google wrote, and explore why phrases like those might be appearing, and see if what’s described within that patent might explain how those phrases got there. If it is, great. If it isn’t, also great.
You might be right about the citation as the reason that it is used. No doubt as to who the page is about. Although the Dublin office did not receive a snippet.
I did locate the source of the word “jewelry” for Barbara Oliver & Co… it comes from a Kudzu review – http://www.kudzu.com/m/Barbara-Oliver-And-Co-Inc-6091591/reviews/ . It strikes me as odd that one use of one word, that is no where else would become the snippet.
Ah, the mysteries of the algo…
Actually it comes from a syndicated CityGrid review… that could explain the use of the word as it is likely to appear at a number of sites.
Hi Mike,
The number of appearances of a phrase might definitely make a difference as well.
One possible part of the algorithm to choose phrases seems to involve looking at the textual documents involved and performing an n-gram analysis, where the text is broken down into tokens of certain sizes, and performing a statistical analysis of those. I did a search for [google “western union”] and found over 90,000,000 results, and many of the top results copy information from Google’s help page about receving quick cash payments for Adsense via western union. Many of those also copy the complete address information word for word (over 6,700 on a search for the full phrase in quotes) for Google from the Adsense western union payment help page.
So we not only have a large number of pages that tie the phrase “western union” to Google, but a good number of those contain fairly full business location information for Google, too, including at least one from Google itself.
Here is a link to a site that displays a Sentiment Snippet, but not one that the business would appreciate. http://goo.gl/r1uvH
Prior to this Place Page being claimed someone had been able to essentially post a banner that came from yellowbot.com that stated that the Dr. was convicted of a crime and then had a link going to cc4rvc.com, which is a website dedicated to this issue.
The Places example that Bryan provided is an interesting example and reinforces both the citygrid connection and the quantity idea. Here is a search on the phrases from the Places page that demonstrates the point – http://www.google.com/search?client=safari&rls=en&q=%22dr+langhofer%22++%22google+his+name%22&ie=UTF-8&oe=UTF-8
Hi Bryan and Mike.
Thanks for pointing out tht example, Bryan. Interesting and very unfortunate set of sentiment phrase snippets for the owner of that clinic.
Mike’s search results show that reviews with those phrases do appear within a good number of reviews on different sites, so quantity does seem to be a factor to consider.
Hey Bill,
That’s great! You know just now I was reading a blog where Google+ and Facebook were being compared with each other. I said that it’s impossible to win over Facebook for the next 10 years at least, but now this feature of Google will make a change. Because it will help in a website’s marketing as pictures and images also attracts traffic, along with that the space that the first result is taking to display will also bring a point increment in the CTR than the second one.
Well, may be not in social media but Google is the real king of search.
Sounds to me like there may well be a surge in webmasters incentivising customers to use sites like Yelp and citysearch to come! i certainly see this currently where ecommerce websites encourage reviews that will show up on Base.
Jon
Both the “western union” example and the clinic example point out that while sentiment snippets may be another step in the direction of more relevant search results, there is certainly still room for improvement… and potential pitfalls as reviews can be manipulated.
It’s becoming more and more obvious that Google is now getting more serious about the social internet. What will this do for SEO and for site rankings? Anyone care to take a stab at that?
Morning Bill,
Hope you enjoyed Labour Day.
The sentiment phrase snippet is an interesting patent (i do prefer the phrase “Snippet” over description too – I’m not sure that the amount of information conveyed could be defined as a description.)
This patent does seem to have hall marks of some of the previous patents you’ve covered recently – most notably the search rankings by website category post.
In some embodiments, all Sentiment Phrases 317 with the same Noun Phrase 318 are grouped into bins before selecting 712 the Sentiment Phrases 317 based on Sentiment Scores 312
So it would seem to suggest that the use of a key noun as a defined topic is a core principle before the sentiment of the phrase is taken into account. This is then followed by indicative value of sentiment across each occurrence of the key noun which can then be grouped and compared.
I find this really interesting that they’ve decided to polarise and sort from sentiment and group in this way – at the same time considering the grouping of website by sector and topic – it would seem to make sense. Either that or I still can’t get over the fact that you’ve got a Sushi Bar in your supermarket 🙂
The NLP techniques are another area that really interests me – I think I need to have a “print out and highlighter” session with this patent though – before I can properly digest all the information their conveying.
I had a little play around with the maps tool too and I think it’s quite neat that when browsing for a location using the map – on mouse-over the snippets appear on the map as a summary as the kind of business.
I do get the impression that this is one of those patents that we’ll keep referring back to in the future.
Tom
@Barry
As Tom points out, this patent is very similar in functionality to the recently received patent to categorize keywords and website pages. An in fact, I think we are seeing some use of sentiment in ranking. Essentially if there is enough frequency of a sentiment AND searchers use it in their searches, Google will rank sites based on terms like “bad/cheap/disgusting” + service/product +location type searches.
There is every reason to believe that as Google gathers more information about individual as searchable and rankable entities this could be applied to people as well…
I can see the social searches now… “most disgusting blogger”, “most annoying facebook fan”, “most profilic tweep”. 🙂
HI Kevin,
I believe I ran into the post that you are writing about, where eye tracking was used to compare how people were viewing Facebook pages and Google Plus pages, and the results of that eyetracking was similar in a good number of ways (except for the fact that Google Plus isn’t showing advertisements like Facebook does).
At this point, these sentiment snippets seem limited to Google Plus pages. Google does have a different way of showing sentiment snippets for Google products. For example, see the “summary of reviews” section on this review of a HP Tablet:
http://www.google.com/products/catalog?q=hp+tablet&oe=utf-8&rls=org.mozilla:en-US:official&client=firefox-a&um=1&hl=en&bav=on.2,or.r_gc.r_pw.&biw=1920&bih=961&ie=UTF-8&tbm=shop&cid=6385989111978868049&sa=X&ei=MItnTv6KB86dgQfQ48C7DA&ved=0CHIQ8wIwAQ
Note the green and red bars in front of short snippets, where green is positive and red is negative.
Hi Jon,
Finding ways to encourage people to leave reviews in places like Yelp and Citysearch isn’t a bad idea. It’s something that I wish more people in my community would do, if only to make it easier for me to learn about what people are saying about local businesses when I’m deciding to do things like tune up my car or decide where to shop.
But seen the link in the trackback a couple of comments above yours which starts with “High Ranking Local Results You Just Might Not Want…”
Hi Kevin Hughes,
There do seem to be some kinks in the system of finding snippets for reviews still. The “western union” snippet for Google really isn’t very helpful, or very representative, even though it is in some ways relevant to interactng with Google as a business when working with them to publish Adsense advertisements.
HI Barry,
I’ve written a lot of posts about Google and their social networking activities. Check out my Social Networking and Social Search category posts.
In short though, I think Google may be more likely to use some of their own data from their own social networks to influence rankings than they might from places like Twitter or Facebook since they have more control over information associated with that data, such as the IP addresses that people log in when they use something like Google Plus.
Hi Tom,
Thanks. I had a chance to do some labor and to do some relaxing as well. Hope that you took a chance to take some time out as well.
It does look like they are trying to do some clustering of those phrases based upon the nouns found associated with them, so that they might be able to provide a broad range of sentiments regarding different aspects of a business or product.
For instance, with a restaurant, they might want to show phrases associated with the food served, the service they received, the location of the restaurant, the prices paid, and so on. If they didn’t do that, then the phrases they showed might be too one dimensional.
I haven’t had a chance to explore the map aspect of this, so thanks for bringing that up. It’s definitely worth exploring the patent in more depth.
Hi Mike,
Definitely an interesting aspect of how sentiment can influence search results, and some great examples. Thanks for bringing those up. The use of sentiment in ranking search results seems like a dangerous path to go down, but one that we may start seeing.
@Bill
The keyword –> category mapping doesn’t seem to distinguish between sentiment and product at this point. Perhaps by design? Maybe they want folks to know who the “worst plumber in NYC” is?
Hi Mike,
I definitely have to spend more time looking for examples, and exploring how Google may be using sentiment here. Is it just part of the descriptive snippets that they display on Places pages or does it have a larger role?
The influence of categories of queries and categories of websites, and a boost in ranking of pages for businesses in those categories when there’s a match between the query category and the webpage category seems to make sense. Will sentiment fit in as well?
I did copy one of your searches from your post, [terrible doctor nyc] and the third of the three listings includes a review that starts out with the phrase ‘terrible doctor’ but none of the three listings have the word “terrible” in their semantic phrase snippets. Might they have been displaying reviews in the past that included the word ‘terrible’ in them, and might it be possible that the word was used in some other manner, such as “great doctor, but terrible location”?
It’s definitely something to explore in much further depth, to see why some of the Place results were showing for the queries that you used, and if sentiment is playing a role, or if they appeared because of another reason.
Does anyone have a theory as to why the Google Places reviews are ALL stellar right now? I suspect that everyone who is “up to date” on SEO tactics have asked their best clients to send their reviews in. To this extent, the Google reviews do not reflect the sometimes “real” bad reviews on particular establishments. I wonder how Google will address this issue, if at all.
Interested observation about Google Places reviews. My guess is that most of authentic reviews are done by disgruntled customers, while the stellar ones are planted. Correct me if I’m wrong, but when someone had a great experience at a restaurant or a car repair shop, their first impulse is not to post a favorable review online. However, when customers are steamed, many often feel more compelled to complain. Human nature, I guess.
Hi Joel,
I’m not sure that people are only motivated to leave negative reviews. I’ve written a few that were inspired by great service or great food, as well as a desire to see places that I like do well, especially with the economy the way that it is. I do run across a lot of local businesses that don’t have any reviews at all, and wish that there were more, and that’s inspired me to leave some reviews as well.
I think that easier access to review sites via mobile phones makes it more likely that people will leave positive reviews as well.
I would agree, users are not motived to only leave negative reviews. I recently wrote an article about this. Based on my experience, there are 5 major reasons a user will leave a review:
1. Self promotion; false reviews
2. Very negative experience
3. Very positive experience
4. Friend of business owner
5. Some incentive (value) gained for user leaving the review
@Forrest
You need to add a 6th major reason- if they are politely asked. We find just the asking is enough to get a fairly large number to leave a review.
Hi Forest and Mike,
Thank you both for the list of reasons why people might leave a review for someone. I do think that there’s just as much chance that someone will leave a legitimate positive review as a negative one.
Bill I was reading some of the comments and was surprised to see your opinion that Google would be giving more weight to their own social signals such as Google Plus. While this would seem like the natural thing to do I have been reading otherwise from some pretty reputable sources who did a study on this – I believe it was Rand Fishkin. If I remember correctly Facebook shares was believed to be the #1 social signal for SEO followed by Twitter.
Hi Bill,
Rand and SEOmoz did a “correlation” study on the impact of Facebook status updates with rankings, but that’s something very hard to do because there are so many factors that are either outside of their control or that they might not even know about or anticipate. I don’t believe that their experiment was also intended to compare the impact of other potential social signals like those from Google Plus.
Chances are that the impact of Google Plus posts don’t directly impact search rankings for specific pages either, but rather can influence a user rank or reputation score for authors or contributors to the social network who may have linked their content elsewhere to their Google profiles via authorship markup.
Since Google has much more access to information about Google Plus posts than they do to Facebook status updates, which includes information helpful to creating a “reputation score,” that can help them do things like filter out contributions, endorsements and other actions from fake profiles and sock puppets. Those things can be a little more difficult when it comes to looking at signals from Facebook, for example.