When you search on Bing, sometimes instead of seeing an ordered list of search results, you might see search results broken up into categories. For example, if you search for “Virginia,” your search results start off with an image and link to the state web site, as well as a map. You then see a couple of search results that look pretty relevant for the term.
What comes next is a little interesting. Instead of showing you just more links to web pages like you might see at Google or Yahoo, Bing starts showing you groupings of additional web pages organized by category. There’s a Virginia map category, then Virginia Tourism followed by Virginia Facts, then Virginia Jobs, and finally, Virginia History.
This diversification and grouping of search results is a departure from a paradigm commonly followed by many search engines. When a query term might have more than one meaning, or different categories of results might be equally useful to searchers, Bing may decide to present those search results in different categories, like it does on a search for Virginia. Here’s the first category shown in the Bing results on a search for Virginia:
Bing’s method of diversifying search results raises a few questions, such as:
- What queries should receive categorized search results?
- How are the categories decided upon?
- Why are the categories in the order they are in?
- How are specific pages selected for those categories?
The intent behind a query may differ from one searcher to another. As the patent’s authors tell us, someone searching for “Flash” might be interested in Grandmaster Flash, Flash Gordon, the Adobe Flash Player, a camera flash, or a town named Flash (a village with the highest elevation in England). Because there may be a number of different intents behind some queries, Bing takes a different approach in returning results for some searches than Google or Yahoo.
Before showing search results, Bing may attempt to identify categories that a query may belong to, rank those categories in an order, decide how many categories to show, and how many results to show for each category.
The Microsoft patent application is:
Methods and Apparatus for Result Diversification
Invented by Sreenivas Gollapudi, Rakesh Agrawal, and Samuel Ieong
Assigned to Microsoft
US Patent Application 20100153388
Published June 17, 2010
Filed: December 12, 2008
Abstract
Methods, apparatus, and systems directed to receiving search queries, retrieving documents, computing the number of categories to present for a given query, computing the number of results to show in each category, computing an ordering of categories, and for all the result pages beyond the first page employing user interface elements that optionally allow the user to quickly zoom in on a specific category and get more results belonging to that category.
Under the patent filing, this process has a number of steps. The first couple may be to create a taxonomy for categorizing documents and queries, and an authority score for documents in its index. The creation of a taxonomy and authority scores can happen independently of any searches from a searcher.
What happens next is in response to a search.
A query is received by Bing from a searcher, and it is assigned probabilities that it is in one or more categories within the created taxonomy. If the probability that a category is a good match for a query is above a certain threshold, than that category may be included within the search results.
Pages are then retrieved for each of those categories based upon their authority scores. The number of categories that are decided upon for a query may determine how many pages are shown in each category – if more categories, Bing might show less pages for each category. If there aren’t many categories for the query, Bing may show more pages for each of those categories.
A combination of looking at the probability scores for categories and authority scores for pages within those categories will determine the order that categories are shown in search results. The patent application describes a few different approaches to using those scores to determine ordering, and some examples to illustrate how they might work.
What it doesn’t provide, which I would be interested in finding out more about, is how authority scores might be calculated for pages that might appear in each of the categories. That approach may may be similar to the ranking scores used in response to a query when Bing doesn’t break search results up into categories.
Conclusion
I’ve been seeing these kinds of categorized search results from Bing for a while, but couldn’t find anything from Microsoft on how they might select categories to display in response to those queries.
While I’ve found the categories useful, I’ve been wondering how most people react to Bing showing results in categories like this. I imagine most searchers have gotten used to seeing what might be considered the best results at the tops of search results pages.
It’s possible that Google and Yahoo might also attempt to diversify search results they show by associating categories with queries, and ranking some pages higher in order to show a diverse set of pages to meet different intents behind some queries that may have more than one meaning. But those search engines don’t explicitly display categories the way that Bing is for some queries.
How do you feel about Bing’s categorized search results?
Thanks! I am still trying to find out how to master Bing. While the features listed in this article are definitely cool, I just can’t seem to steer traffic to my site.
I bet you that 95% of my search traffic is from Google and Yahoo, and have been disappointed about the lack of SEO finds by Bing……..especially when you consider that they are supposed to be taking over the Yahoo search function.
Does anyone have some Bing pointers on how to better optimize for their search?
Hi Bill,
I enjoyed reading this post, I’ve printed out the patent to examine it in greater detail later.
To answer your question about Bing’s categorized results: I used the author’s example of flash and other keywords to run multiple queries on Bing for “apple, iphone, blackberry, shoes, flash, ibm, google and san francisco” with personalization enabled and disabled.
With regards to the corporation and city specific names, I noticed these categories look eerily like the Google sitelinks. I could be wrong here but the category ordering for this small sample of keywords appears to be based off the order of the categories in the first result from an initial glance.
I wonder if Microsoft is serving these categories on basis of search volume. High search volume for a specific set of documents = high authority scores for those sets of documents and thus the ordering is created.
On a side topic, this has been on my mind for some time. If a search engine xyz has links for a set of documents indexed in its indices and a searcher searches for item abc and travels to website efg through a link/document indexed by search engine xyz, it is quite possible for the engine to know how much traffic passed through that link right and score that document right?
Best,
Jey
Sorry to double post but if you query “flash” in Bing you will notice that the results on the first page are predominantly about “Adobe flash,” but each result from the second page onwards come from different categories which lends credence to what you’ve written in your entry. This makes sense to me, search for “flash and most likely you are searching for “adobe flash,” if not then let’s show the users all the different kinds of “flash” so he/she gets the right result.
Hi Mike,
The methods that the search engines use to rank pages all differ, but their ultimate goals are to deliver people to pages that are the best results that they can deliver based upon the relevance of those pages to queries used by searchers, and some measure of quality of those pages.
A number of whitepapers over the past couple of years from Microsoft have made it clear that Microsoft was using a ranknet algorithm to rank pages in their search results. One of the first papers from Microsoft that described that ranknet approach came out in 2005 – Learning to Rank using Gradient Descent (pdf). The paper gives us some hints at the kinds of things they might look at when creating a machine learning approach to ranking pages:
I wrote about that paper and some other papers and patents from Microsoft, back in 2006, in the post Feature based rankings at MSN.
One of those papers hinted at some potential ranking factors that they might consider in the future, such as:
I suspect that Microsoft does use a number of measures that might surprise people, to attempt to actually understand the quality of webpages, such as reading levels associated with pages. So if they are looking at things like W3C validation, that might not be a surprise.
A couple of posts about Microsoft patents/papers that you might find interesting as well:
How a Search Engine Might Analyze the Linking Structure of a Web Site
Microsoft Granted Patent on Vision-Based Document Segmentation (VIPS)
Hi Jey,
Thank you. Those are some great observations.
Regarding your side topic first, It is very much possible that Microsoft is considering what they call “final destination pages” to determine some level of authority for a page for a certain query topic. Their papers, Studying the Use of Popular Destinations to Enhance Web Search Interaction and Leveraging Popular Destinations to Enhance Web Search Interaction (pdf) hint at such an approach. I wrote about those further in Microsoft Tracking Search and Browsing Behavior to Find Authoritative Pages.
We’re told in this patent on deciding upon pages to show with categories that they are presenting authoritative pages as results. Is the use of the word authority the same as in the “Destination” papers? They could be. From one of those papers:
Interesting idea to compare Google’s sitelinks and Bings Categories. I expect some overlap in the sitelinks for corporations, but you have me wondering how much overlap might be found in Bing’s categories and Googles suggested query refinements. For example:
Google Query Refinements
Searches related to apple
apple fruit
apple rumors
apple federal credit union
apple trailers
apple jobs
shabby apple
apple ipod
apple store locator
Bing Categories for “Apple”
Apple Fruit
Apple Store
Apple Downloads
Apple Customer Service
Apple Reviews
Google Query Refinements
Searches related to iphone
iphone verizon
iphone plans
iphone tmobile
iphone review
iphone antenna
iphone 3g
iphone price
iphone unlocked
Bing Categories for “iPhone”
iPhones Unlocked
iPhones Accessories
Refurbished iPhone
iPhones software
iPhones Problems
Some overlap there.
Search volume is a possibility. It’s probably worth looking at a number of the category type query results.
Thanks again, Jey.
Hi Jey,
There is a great deal of diversity in the Bing search results for Flash after the first page. The topics differ so much from one to the next that it seems like it might be intentional rather than based upon a combination of relevance and importance. This may be closer to the way that Google seems to handle diversity in search results.
I am not using Bing but basing on your description of Bing here on this post regarding how searches come up. It actually sounds interesting and quite organized. So I guess I may have to take a peek at Bing.
Bill,
Very interesting reading, as a recruiter I find myself constantly on the other end of the equation, trying to ferret new information out of similar queries. One side effect of Google’s dominance is everything it optimized accordingly (or seems to be from this users perspective). Hopefully the pendulam will swing a bit back the other way and open up some alternatives….funny that it’s Microsoft looking up the mountain this time…
Thanks for a very informative post.
Dave
I have been using Bing more and more in the past 6 months or so. I love the new features they continue to roll out. The categorical listings are quite interesting. In my opinion, Bing is ahead of the game when it comes to intent-based search. The categorical listings are a step in that direction, and users should expect more relevant search results.
Search engines always put stress on the keyword which is entered on search bar.Diversification and grouping of search results often create confusion to choose the best selection.The user should be careful by selecting the right data.
This article is exactly about why I am not using Bing and probably never will.
I was never a huge fan of Bing, primarily because I never thought it would go anywhere (something like canoe). But now that I’m seeing how much more different and convenient their searching is by putting them into categories I might consider giving it a try. It might also be a more convenient way to get better listings per category than over all.
I’ve been hearing about Bing these past few days but never got interested. I was contented with google and thought that its just another search engine. Now that you mentioned about its special feature, I admire them for differentiating. Now I think I need to try it.
Hi Andrew,
I usually like to try to keep an eye on all the search engines, and to perform the same searches at different search engines just to see how they might treat those differently.
Hi David,
Microsoft seems to have so many resources available to them to create something that could rival Google. Sometimes I feel like they might be the only ones holding themselves back. For example, I’ve seen a number of patents from them on topics like tracking blogs, and on blog search, but I haven’t seen a blog search engine from them. I’m not sure why they haven’t released one.
Hi Bradford,
Microsoft does have some interesting features. For example, what they are doing with wikipedia is pretty interesting.
Hi TomPal,
I’d love to hear you elaborate on that. I’m not sure I understand why Bing’s Category search, and somewhat different way of presenting some search results would turn you off to using the search engine, especially if it provides meaningful results.
Hi vitalsigns786,
I do think that there is some possibility for confusion, especially for searchers who may not understand that Bing is sometimes segmenting results into different categories.
The categories are pretty clearly marked, but there may be some things that Bing can do to make it clearer that the results are presented in categories, at least on the front page of the search results.
Hmmm…
At first this looks like a good idea, but when i think about it, who would want to know about Virginia Jobs, if they type in just the word Virginia :/
Hi Bryan,
People who are new or not very knowledgeable to the internet have a tendency to type one word queries. As one’s internet sophisticatedness grows, the user starts using site operators or more words to help the engine.
as much as i like BING google still holds the key to my heart.
I think that Bing is useful because of this, especially as a web marketing enthusiast. It’s nice to see that they try to break things up, a little bit more like Google, but still retaining the search engine qualities that only Bing provides!
While the search engine may be doing some clever stuff in segmenting the results, the main concern I have is about “usability”. There is limited explanation provided about how or why these particular results have been formatted for the user. The great thing about a simple set of 20 search results (whatever the quality) is that the users have a pretty good understanding on what they are seeing.
Usability studies looking at search results show that best bets, side bars for facets etc. are usually ignored by the user who looks at the first few “real” results on the page. There is a danger that too many features on the search results page will confuse users more. We are all search experts commenting on Bill’s blog, but the majority of search users (I believe) require a simple, intuitive interface.
Interesting post
I didn’t even noticed categorized search results in bing since i don’t use it so much.
I must also say that i receive very little traffic from Bing so i don’t spend too much time optimizing sites for it
Hi Bryan,
That’s a good point. I do wonder how well those categories do match up to what people are actually searching for. You would think that someone looking for a job would type more than just a state name into a search box.
Hi Ted,
I think that may be one of the biggest failings of this category approach – Bing isn’t presenting the categories in a way that makes it easy to understand what they are doing to people who are used to a straightforward list of ranked results.
I’m (as I guess we all are) such a creature of habit. I eat sleep and breath Google when performing online searches. The way Bing presents search results sounds a million times better than what Google’s doing. I think I’ll check it out and try to get used to it.
Plus, competition is always good. Maybe this will give Google something to think about
Yes indeed, it sort of seems like you’re not going to beat Google at their own game, so do something different with search, make a new category of search engine. In the end it’s obviously doing the same thing, but as they say in marketing, if you can’t be best, be first. This seems to be Microsoft’s strategy – position Bing as something different and new. In the process they might actually make something new for real!
But it’s got to work though… In my experience, ‘intuitive’ and maybe even ‘efficient’ are not words that accurately describe Microsoft products (although they maybe fit their business model better…)
Bill, isn’t this kind of what Google did do and wasn’t a similar topic introduced with the Google Caffeine? i like Bing, but it seems to not like me. My site is high up in Yahoo and Google, but its on the second page for my main search term.
Does anyone else get the Yahoo Directory vibe from this, just not static. In the end the directories have become nothing but a way for Yahoo to make money and I suspect the answer for how to focus your entry into categories will end up costing $ too.
In the Netherlands bing has hardly 1% market share, so it doesn’t give a lot of traffic but with a few project i worked on the things did work on Google did good (sometimes) better on Bing. I must say i kind of like Bing but i have to agree with Bryan that if you search for Virginia and get’s Virginia jobs it’s not the most relevant result i would say.
Hi Roschelle,
Competition is good. I think Bing still has some work to do to catch up with many things that Google is doing well, but I’d love to see them become more competitive and provide us with more options.
Hi Warren,
I still haven’t gotten through my mind how a “decision engine” is different than a search engine, but I’m completely with you in hoping to see something innovative and different from Bing. Fingers crossed.
Hi Eric,
Caffeine is more of an infrastructure change than an indexing or user interface change. It provided Google with more storage on each of their servers, as well as a faster way to update its index and to process search requests, but it didn’t offer changes in the way that information is displayed to searchers.
The ranking approach behind Bing is different in a number of ways when compared to Yahoo and Google, but in all honesty, keep on doing the things you are to rank well in Google and Yahoo, and it’s likely that your rankings will increase in Bing as well, as long as a good part of your efforts focus upon creating a good user experience on your pages.
Hi Duncan,
I’ve always sort of looked at Yahoo as a portal attempting to be a search engine, and Google as a search engine attempting in many ways to be more portal like. Microsoft has always been a little more like Yahoo than Google in its efforts to provide search results as well, so to see them start arraigning some search results by categories isn’t a surprise.
One aspect of indexing the web is to be able to classify and categorize web pages in an automated fashion. That can help in areas like deciding which ads to show with which pages, in building local search directories, and in working towards a search engine that is less based upon links and exact keyword matching to one that is more based upon user behavior and indexing based upon the intent of people searching.
The old web directories like the Yahoo Directory and DMOZ require people to perform those categorization tasks manually, even though the numbers of web pages have increased dramatically, and that’s probably been the biggest factor in their beginning to fade in usefulness. There is value in being able to present some topics in a directory type fashion, especially when people searching don’t know too much about a topic. I’m guessing we will see more automated categorization at all of the major search engines in the future.
Hi Daniel,
I do think that Bing is getting a number of things right, which is encouraging. With this classification approach, it’s possible that they may make some missteps, like the “jobs” category showing for Virginia. Hopefully they’ll work on finding ways to improve those.
“This diversification and grouping of search results is a departure from a paradigm commonly followed by many search engines. When a query term might have more than one meaning, or different categories of results might be equally useful to searchers, Bing may decide to present those search results in different categories, like it does on a search for Virginia.”
So this is over all a good thing, right?
Hey Bill, its weird though. Before my site started to rank high in google and yahoo, it was ranked within the top 3 places everytime. Then I started making changes to my site and it switched- Google & Yahoo both started ranking my site within the first 3 spots, while Bing flipped and started ranking it way lower.
Its kind of strange, but for the better really, considering that 2 search engines is better than one and that they both send way more traffic.
-Eric
Hi Mike,
That diversification might be helpful if it truly addresses the information needs that searchers have.
Hi Eric,
Then again, we’re likely going to see Bing results appearing at Yahoo sometime in the very near future.
Bill, your right. I have heard about that before. I wonder why bing decided to rank my site lower.
well, thanks for the reminder.
-Eric
bing gives exactly 0 useful information on this site about how to get reesults for the exact topic searched for.all results given are unrelated or redirected or both.For some reason bing refuses to allow results for the exact search topic entered.Bing will only give results if they are allowed to be redirected to unsearched for topics!
Hi catmankent,
I suspect that if you asked most search engineers working on search engines, they would tell you that search is still in its infancy, and that it’s a difficult problem when someone types in 2-3 words as a query to give them exactly what they want in search results. Not only are search engines limited by how well they might understand the intent of a searcher, but they are also limited by what actually appears on the Web itself, and for many queries, there may not be very good search results.
Bing has been advertising itself as a “decision engine,” which is kind of interesting. A “decision” engine would not only give you the search results that it might think you meant with your query, but would also provide you with a way to interact more fully with the search engine by offering possible query revisions and suggestions and refinements to help you find what it is that you’re looking for. Interestingly, Google has stayed away from the “decision engine” lable, but it also attempts to offer suggestions and refinements as well. Google did launch their “verbatim” search not long ago, and I agree that it wouldn’t be a bad idea for Bing to do the same as well.