In the early days of Google, when you performed a search, the results you received were just links to pages found on the Web, showing page titles, snippets, and URLs. Google started adding other types of searches to its Web search, such as:
- July, 2001 – Image Search
- September, 2002 – News search
- December, 2002 – Product search (Froogle)
- December, 2003 – Book search (Google Print)
- February, 2005 – Google Maps
- September, 2005 – Blog search
While these launched as separate search repositories, they weren’t going to stay that way, and may never have been planned as solely being standalone data repositories. In 2007, Google introduced Universal Search. At a Google presentation called Searchology in May of 2007, Google announced Universal Search, which included video, news, books, image and local results incorporated into Web search results. According to the Official Google Blog post, the roots of Universal Search can be traced back to 2001, with a lot of effort leading to its launch:
Over several years, with the help of more than 100 people, we’ve built the infrastructure, search algorithms, and presentation mechanisms to provide what we see as just the first step in the evolution toward universal search. Today, we’re making that first step available on google.com by launching the new architecture and using it to blend content from Images, Maps, Books, Video, and News into our web results.
While I could focus upon the original Universal Search patent with this post, there’s another patent filing from Google that possibly describes better how search results from different data repositories are included in search results today. Before we get there, a little more of the history behind how we got to this newer version of Universal Search.
Vertical Creep
A few years before the official announcement of Universal Search in 2007, we started seeing Google incorporating different types of search results into the Web results displayed by the search engine. In the search community, this was referred to as “vertical creep” into those results, since many referred to those separate data repositories as Vertical search results since they were taken from more focused search results or vertical results.
In 2006, I wrote about that in a post titled Vertical creep into regular search results in Google. Often those results would be displayed in a specific location within results, such as images or news at the tops of results, local results possibly starting in a fourth result slot on the page, and so on.
OneBox Results
In addition to experimenting with these other results, Google would sometimes display OneBox results as well when it thought they were appropriate, such as:
- Weather (type in – weather and a city name – such as “weather Cincinnati”)
- Definitions (type in – define:xxxxx to see a definition result, or type in this format – “what is XXXXXXXX” for a vertical search result above regular results)
- Music (type in a band name, for example – “Foo Fighters”)
- Books (type in the name of a book, for example – “Moby Dick”)
I wrote more about how Google might determine which results to display in an OneBox over at Search Engine Land in the post, Google’s OneBox Patent Application. The patent behind that approach (now granted), Determination of a desired repository (US Patent # 7,584,177) involves user behavior data to a large extent to determine what might be shown in a OneBox. As I noted in the Search Engine Land post:
If I read this patent filing correctly, user data about queries in the different vertical searches may influence which documents or objects appear in OneBox results. So, if a lot of people go to Google Image Search and look for pictures of “lionsâ€, then OneBox results may show images of lions. If suddenly, a lot of people are looking for “lions†on Google news searches, then we might also see news results the OneBox area, instead of the images or in addition to them.
The Original Universal Search Patent
When the original Universal Search patent was published in July of 2005, it described search results that actually segmented different types of results into different areas within search results, like in the screenshot below, which shows web results to the left, and boxes to the right starting with product search above, then news results, followed by a segment for image results with a link to see those, and then “groups” results with a link to get to those.
Some of the categories may be seen as being more relevant than others, based upon a reading of the relevance or intent behind the search. The patent filing tells us:
For example, ranking component 402 may generally compare the search query to the contents of the documents in each list and base its ranking values on the closeness of the comparison.
Consider the search query “buy athletic shoes.†For this search query, ranking component 402 may determine that the user is most likely interested in athletic shoes that are for sale.
Accordingly, the ranking component may rank the
‘products” category highly. The links in the list of links that correspond to the products category are likely to be links that correspond to web pages that are offering shoes for sale.
That Universal Search patent was granted in 2008, but it probably isn’t the final word on how Google determines which results to show from which search repositories, and how to display them.
A Yahoo patent application published around six months later also hints at how it might determine which results it might display to searchers based upon “historic selection data,” in search results for different search repositories.
Blended and Universal Search
While user selection might still play a strong role in which results are presented to searchers from which data repositories, a new patent filing was published at the USPTO in June of 2008 that provided considerably more information about how the relevance of different types of data might be considered in that determination as well. I wrote about the patent filing in How Google Universal Search and Blended Results May Work.
Google’s David Bailey, who is one of the named inventors on this newer patent filing described some of the issues with the older version of Universal Search in an Official Google Blog post almost a year earlier, titled Behind the Scenes with Universal Search. There he told us:
Here’s the challenge in a nutshell: Until now, we’ve only been able to show news, books, local and other such results at the top of the page, like this example for [trends in education]. But it’s a tall order to earn placement at the top of our search results, so plenty often we end up not showing these kinds of results even when they might be useful. If only we could smartly place such results elsewhere on the page when they don’t quite deserve the top, we could share the benefits of these great Google features with people much more often.
The idea of determining which additional results should be shown, and where to blend them into Web results was the focus of this patent application. The Interleaving search results patent was granted in December of 2011.
Blending Non Web Results
In a nutshell, when someone sends a query to Google, search results are received for web pages, and from the other data repositories as well. The search quality scores from the other repositories might be based on historical selection data or upon unique scoring features appropriate to those repositories, or both. For instance, a unique scoring feature for Google News results could include the freshness of news articles. Google might also still look at how frequently people might be selecting certain news results for a particular query over a specific period. For image results, Google might include unique scoring features such as the size or resolution of images.
The top results for different categories of results from those other repositories might be selected to be considered against the Web search results, and possibly blended into them. When blending would happen from more than one source, a mixing program might recalculate search quality scores, but decrease the contribution of the “unique scores” for results from the different search engines. So, for example, the size or resolution of an image might not be as strong a part of the ranking of that image. In this way, the final blended results shown to searchers would be less like comparing apples to oranges.
There might be one or more strategic approaches to grouping non Web search results with Web results:
- A non Web result might be inserted at any of several positions within a list of ten Web page search results
- Only the highest-ranked non-web search result might be included among the web page search results
- None of the non-web search results might be included because they don’t have a high enough rank
- Non-web results may be inserted at a position among the web page search results
- Non-web results may be placed at a fixed position, such as the top, bottom, or center of a list of web page search results
- Some restrictions might be placed upon where non-web search results may be placed, such as either the third-ranked result or a lesser ranked result
- A non-web result might be limited, for example to a position in a ranking order that is more than two or more positions away from another type of non Web search result
Takeaways
The idea of providing “universal” search results was in the air at Google before the search engine even started introducing other vertical search repositories such as image search or product search.
Providing a diversity of results seems to be one of the key reasons for presenting Universal Search results, as does matching the intent of searchers for those results.
Non Web results that might potentially be displayed with Web results are initially selected from each of the different types of data repositories, such as image search and product search and news search. Initial rankings for those are in part based upon unique ranking features from those specific repositories, so if someone wants non Web results to possibly appear within Google’s organic results, those results would need to rank well based upon those features unique to each repository.
Once those non Web results are selected, they are ranked again, with less weight given to those features unique to each specific repository, against Web search results, and that ranking (along with some strategic considerations listed above), may determine whether they appear in Web search results and where. User clicks may still play a role as well, so for instance, a news story that gets lots of clicks in news results might stand a better chance of showing up in Web results than one that doesn’t.
All parts of the 10 Most Important SEO Patents series:
Part 1 – The Original PageRank Patent Application
Part 2 – The Original Historical Data Patent Filing and its Children
Part 3 – Classifying Web Blocks with Linguistic Features
Part 4 – PageRank Meets the Reasonable Surfer
Part 5 – Phrase Based Indexing
Part 6 – Named Entity Detection in Queries
Part 7 – Sets, Semantic Closeness, Segmentation, and Webtables
Part 8 – Assigning Geographic Relevance to Web Pages
Part 9 – From Ten Blue Links to Blended and Universal Search
Part 10 – Just the Beginning
Personally, I’m glad that Google started showing non-website search results in listings.
More often than not, when it has come to serious research on non-main-stream subjects, the most important information available has actually not been available in Google’s index, not by a long shot.
Contrary to what most people believe…not all knowledge in online. There are many things that you can’t just “Google”.
At least Google admits this by mixing in these non-web-based results.
Kinda humble of the juggernaut to even do that, I think. That is indeed putting the customer first.
Awesome, as always Bill.
Mark 🙂
From my experience at the Belfast Telegraph I know that the inclusion of news results in universal search was the salvation of news sites, as it allowed them to harvest cheap traffic (& ad impressions) from high volume search terms relatively easily.
Image search pattern interests me most as I have been getting 20 % off traffic from image SERP and having products which got purchased totally on its visual look and design.
But weather and definitions isn’t blended search results, are they? I mean I can’t get my weather data up in there but it’s all Google’s service.
Like calculations which I use a alot. E.g. try searching for: 123/2
Blended search has saved me in several instances and is also the bane of my existence. I am a local business owner and every time someone adds the city name plus one of my keywords it is a huge bonus for me because I pop up in the top 2 or 3 Google Places consistently. But for the inhouse SEO work I do in the travel space it is a giant pain. For example if you Google “hotels in Amsterdam” the majority of the first page is taken up by local results that lead the customer directly to the hotel’s contact details. Not exactly helpful if you’re a middle man or wholesaler. I don’t think it always serves the client that well either. A lot of people will be better served by an American based wholesaler than dealing directly with a foreign business, but that of course is my biased opinion.
great article and i agree in the most part that the blended results give users more “relevant”search results, which is of course the ultimate aim of big G. understand our grievance Dave but you’ll just have to grin and bare it i’m afraid.
I also feel that blended results give users more relevant results.
Hi Mark,
Thanks.
Google’s initial focus was upon showing web page results, and while a lot of the other search repositories they built that they’ve worked on blending into their web results involve specific vertical results, most of those are web based as well, but not all of it is.
By offering a separate local search, for instance, Google provided a way to integrate telecom directory information, satellite information, and other types of data into that repository that wouldn’t be available on the web otherwise. Owners of businesses and other organizations were also able to verify listings and submit additional information as well, and Google could take information from other sources (and have done so), such as reviews, and create data mash-ups that bring together useful data that couldn’t be matched by just paying attention to geographic information listed on Web pages.
By offering this kinds of information in Web results, when it seems that there’s an intent on the part of searchers to see it, Google is doing something that does put their customers first.
Hi Barry,
Thanks for sharing your experience. It’s nice to get an insider’s experience of the impact of Universal Search by news sites.
As a searcher, I really like the integration of news results into Web search results as well. Chances are that I’m not going to turn to Google News to find stories that I’m probably not aware of related to my web query, but it’s really nice to be able to discover those stories as they are blended into results.
Hi Tobias,
I did mention One Box results in the post, and for some types of search results that appear in one box listing, there are a limited number of providers. But I have been able to get definitions into one box results, and Google may look at things like Custom Search Result listings (subscribed link type results) for things like sports scores and information.
Those One Box results are something that Google has been experimenting with for years, and I’ve never seen them technically referred to as part of blended or Universal Search.
Hi Dave,
I agree with you that the inclusion of Universal Search creates both challenges and opportunities. We definitely see Google making some assumptions that we might question, like the “hotels in Amsterdam” approach that you point out. I don’t know if I would prefer to try to talk to the people at those hotels directly, or through someone in the United States who might know more than I do, and could possibly give me more information, and the possibility of discounted rates.
Hi Sean,
Thanks. I think in most cases, Google’s search is better for searchers with the inclusion of Universal search results. As an SEO they do make things more challenging, but they also create the potential to have more opportunities to create things that searchers might be interested in.
Hi Jen,
Thanks. That’s where I think blending results like this is important, and a little tricky – when you mention the word “relevant.” Relevance seems to be increasingly growing into meaning “matching the intent of searchers” rather than just matching the keywords in a query with they keywords found on a web page. If I search for “pizza,” it’s probably because I’m hungry rather than because I want to learn about the history of Pizza. Showing me at least a local map and the locations of nearby pizzerias is a relevant result when we thing of it that way.
Thanks for another great post, Bill.
When Universal Search first rolled out, I remember how many SEOs reacted by creating more optimized videos, images etc. in an attempt to establish a foothold into these new opportunities to rank. And while creating rich, quality content in varied formats is critical to SEO, this “lets make a video” knee-jerk reaction didn’t work all the time.
It certainly makes sense that Google uses aggregate historical search data to rank the non Web results in Universal Search. The reality is that a video is much more likely to be served for a “Justin Beiber” or a “Twilight” search (I can’t believe I just typed that…) than for a more “dry” search like “banking industry”. No one wants to watch a YouTube video about banking but many would think that a YouTube video result for a pop star search is appropriate.
Alternatively, it stands to reason that a “banking industry” search is much more likely to serve news results. This is especially true if certain bank bailout, Occupy Wall Street or similar topics are trending at the time and those same news stories are getting found and clicked elsewhere via related searches.
In my opinion, Universal Search has been one of Google’s most intuitive and elegant updates to date. In contrast, and as we discussed on your “Trusted Agent” post, I feel that the recent “Search Plus Your World” update is clumsy, intrusive and very heavy-handed in comparison. Ultimately, Universal Search is based on perceived searcher intent and that is its saving grace.
Hi Jose,
Thank you.
The patent filings tell us that historical search data might be used, but also that it doesn’t necessarily have to be something that’s considered for some of the results. But you raise a really good point about the types of searches that videos tend to be shown for, and the fact that some topics are likely to show one type of vertical searches over others.
I do really like Universal Search as a searcher and as an SEO. I liked Google’s implementation of real time results and social search results prior to their “Search Plus Your World” update. The user interface for their new social search does have a clunky feel to it. Hopefully they’ll spend some time tweaking it, and tone it down somewhat.
Hi Bill,
I agree. I hope G tones down their latest social update.
Just curious … Do you think that the signals used for Personalized Search are incorporated into Universal Search? For example, if I’ve shown a tendency to click on blended video results when they are served, then will I be served more video results than a different user for the same search?
Yeah, blended results give more relevant results!
BTW – Now I have to read this 8 parts which you upload before this one. 🙂
Indeed there are numerous SEO approaches that can be applied. However, initially knowing if a certain approach really works is very important so that you will not waste your time on aimlessly applying ineffective techniques.
-CJ
Hi Jose,
I suspect that Google’s social search will see some changes based upon how people are using the service.
If you’re logged into Google, and it’s recording your web history and showing you personalized results, then chances are that your choices of what to click upon and view will influence what you see in other searches. Some of that personalization is based upon your own actions, and some of it might be based upon choices made by other people who might seem to have similar interests as you.
Hi John,
Blended result do create the possibility that the results will provide more things that are relevant to what you might be interested in. By including news, maps, books, etc., into search results where they seem most relevant, Google’s providing us with a wider range of options of things that might be most relevant for what we want to see.
Hi Clint,
That’s completely true. Knowing more about how different types of results that might be blended into search results creates the possibility that the pages you might be trying to optimize can appear on those as well. So, if you’re interested in having a Place Page show up, knowing how Google might rank Maps results can really help.
thanks for this blog, and especially this post, which I stumbled on. Like the comment that the ‘news’ search helped news sites survive – weren’t they all screaming at Google a while ago about ‘stealing’ their news.
Also re ‘local’ search – not so helpful for my partners who have street locations. Before ‘local’ they could get high placement on a general search for their area – Chicago. Now they dont appear on the results unless their actual suburb is in the search term.
Hi Martin,
You’re welcome. Many news sites were screaming about Google “stealing” their news via Google News, especially in places like Belgium. But they were also very adamant about wanting to have their news continue to show up in Google’s organic Web results via blended results. Guess they weren’t aware that if those results weren’t in Google News that they would never be blended into the Web search results.
As for the local search results, I understand what you’re saying about people ranking well in organic Web results not getting as much traffic once Google started showing local results, but which is more helpful for searchers? To see local results that might not necessarily be that local to them? There is a part of the local search algorithm known as location sensitivity that tries to adjust Google Map results based upon the query used, user behavior, and other factors. Based upon that, a person searching for [pizza] is more likely to see pizzerias closer to them, and on a search for something like [travel agent] might see a much larger map with travel agencies further away. So Google is trying to take into account how far searchers might actually travel for different types of needs.