While Google’s social search activities have been fairly well publicized and discussed, and in some areas criticized, Bing appears to be fairly busy working on their own social search approach a little more quietly…
When Google introduced their Search Plus Your World social search results this past January, they were to a degree following in the footsteps of Bing, who announced last May that they would deliver Facebook based social search results when you’re logged into Facebook and searching at Bing, in their blog post Facebook Friends Now Fueling Faster Decisions on Bing.
People often share a fair amount of information about themselves on social networking sites such as Twitter and Facebook including what they are presently doing, how they feel and more. An interview by Eric Enge of Stone Temple Consulting, Paul Yiu, who runs Bing Social Search, uncovers some aspects of how Bing’s personalized social search works.
The post, titled Author Authority and Social Media with Bing’s Paul Yiu uncovers some aspects of Bing’s approach to social search that you might not have been aware of if you haven’t been paying attention. The post provides a convenient overview before the start of the interview on how Bing might be calculating things like authorship authority from Twitter (and likely Facebook as well).
A patent application filed by Microsoft published this past week describes how they might look at social activity, including indications of your mood and the topics you and your connections write about, to help deliver personalized results to searchers at Bing.
The pending patent tells us that it might customize search results based upon “inferring a mood and/or interests of the user at least in part from an analysis of the user’s posts on one or more social media sites.” In part, this would be done by incorporating a user-following engine that would follow a person’s activities on social networking sites. The kinds of inferences that it might make could include:
- Emotional state
- Current trending interests
- Future plans
- Likes/Dislikes
- Aspirations
Those inferences might then be used to customize search results.
The patent filing is:
Following Online Social Behavior to Enhance Search Experience
Invented by Douglas C. Hebenthal; Douglas C., Cesare J. Saretto, Kathleen P. Mulcahy, and James E. Allard
Assigned to Microsoft
US Patent Application 20120095976
Published April 19, 2012
Filed: October 13, 2010
Abstract
Systems and methods are disclosed for customizing a user’s experience with an application such as a search engine. The user’s experience is customized based on inferring a mood and/or interests of the user at least in part from an analysis of the user’s posts on one or more social media sites.
The search engine is configured to include a user-following engine which follows a user’s activities on social media websites. By tracking a user’s posts and other activity on social media websites, and possibly those of his or her friends, the user-following engine is able to draw inferences about a user, including for example the user’s emotional state, current trending interests, future plans, likes/dislikes and aspirations.
The user-following engine may then customize the user’s search experience based on these drawn inferences.
Customizing a User’s Experience
The user-following engine might follow a person to social media sites and copy their posts and other activities to a data store at the search engine though an alternative approach might not include copying of that information. Whether copied or not, that activity might be analyzed to infer both a mood and interests of the user based upon posts made. In addition to looking at that person’s activities, the activities of their connections might also be tracked and analyzed.
Customization might be done in a couple of ways. One would be to personalize the search interface, by doing things like showing you which of your connections might have “liked” a particular results. The other would be to personalize the results you see by including documents that you might otherwise not have seen based upon your social activities, and those of your friends.
The information monitored by the user-following engine might also derive inferences on mood and interests for a group as a whole to use to “gauge public opinion on one or more topics,” and to identify users who might be “prolific and/or influential in their postings with respect to one or more topics.”
In addition to tracking activities on social networking sites like Facebook or Twitter, it’s possible that activities on blogs might also be included as well. The patent filing includes an example list of social networking sites that might be included in a system like this, including: Facebook, MySpace, Twitter, Linkedin, Ning, Tagged, Classmates, Hi5, MyYearbook, Meetup, Bebo, Mylife, Friendster, MyHeritage, Multiply, Orkut, Foursquare, Digg, Match, and the Xbox Live gaming service.
Some specific blogs are also mentioned as examples as well, such as TMZ, Huffington Post, Engadget, Gizmodo, Mashable, TechCrunch, Gawker, and FanHouse. Facebook comments aren’t mentioned specifically within the patent filing, but that would be something that would make it easier for Bing to track activities on blogs that use them.
The patent filing also notes that while some social activity might be public, other social interactions might only be available to people whom have been granted permission to see that information. Presumably, since you only see personalized and customized results if you’re logged into Facebook, you may be able to see both public and private social information from those social services.
Setting Up Bing To “Listen” to Your Social Activities (and those of your connections)
For a system like this to work for you, you would need to give Bing explicit permission to access information from different social networking sites. The patent filing describes a couple of possible ways that might be done.
The first method of delegating authority to Bing to access social media sites and act on behalf of the user would be through authentication and authorization protocols, such as OAuth WRAP or OAuth 2.0, allowing a user “to set up the user-following engine to act on behalf of the user on the social media sites.” If you’ve signed into a service like Klout, or a site like inbound.org, or signed on to leave a comment through your Twitter or Facebook log-in, you’ve used an authentication service like that.
The user-following engine described in the patent application might work with an application programming interfaces (APIs) of those social media sites to gain access to the sites as the user, in accordance with permissions set by that user:
Once on the site, the user-following engine may track posts of the user, and may have access to posts of the user’s friends. In particular, where a friend posts and sets privileges for the user (and for example none others) to view that post, the user-following engine would have access to that confidential friend post when connected via delegated authority.
Alternatively, a user of this system could set up the user-following engine with permissions so it becomes a connection, and have access to what they post, and possibly also to the posts of the user’s connections.
All users of under this approach might have a user profile service which would store information relating to that user.
There could also be a setup and user preferences module within the search engine that would allow someone to opt into this service, and indicate which social media sites the system would track for that user. The user might provide their name and/or ID on the different social media sites. Upon doing that, a friend request might be sent out from Bing’s system to social media sites that have private user profiles, so that the user can allow access to those profiles if they choose to do so. As I noted above, instead of this friend approach, a user might use a delegation approach to allow access to that kind of information.
Bing might receive a notification when a user of one of the social media site makes a post, or it might monitor the stream of all data from a site.
In addition to tracking the kinds of content presented on these social media sites, this system might also gather collateral data, or meta data, about those activities, such as the time and date of the activity, the type of device used to connect to the social media site, and the location of the user when the connection was made via GPS or some other means for indicating location.
Information about activities conducted by friends or connections on different social media sites, where available, may also be tracked.
The information collected could then be used to draw different inferences about the user, and the user’s connections, based upon their activities. A person using this system might have the option to look at all of the inferences made about their actitives, and could modify or delete them. They could also possibly set up preference rules to prevent the re-occurrence of inferences that they’ve removed.
Other user settings might include rules like “what social media sites, posts or topics the gathering engine is to ignore.” A person could also set preferences regarding friends, so that posts from certain friends might be weighted higher or lower, or posts from some connections might be ignored completely.
How Bing Might Tell if You’re Happy or Sad or Bored or Mad 🙂
One approach that might be followed is to look at the words used in posts. Different words might indicate different emotions being experienced by someone. The patent filing cites the technical report Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings, which contains emotional ratings for a number of words. Those types of rankings might be used to get a sense of a mood behind social activities.
Another approach could be to look for posts that contain emoticons.
A third step could be to look for changes in status indicators, and changes in status, such as “whether the user is in a relationship or single, employed or unemployed, etc.” and if there is a change in that status.
We are also told that a number of other rules, or heuristic tests, might be used to determine emotional states.
The moods of others might be looked at as well, from people who are connected to the a searcher, to people who might be unrelated. For instance, if an event of some type has happened that a large percentage of people using a social network might post about, such as “the death of a popular figure, or the winning of a big game by a team,” that might be considered as well.
This system might find little to support influencing search results based upon moods, or it might find some information that would cause it to look at more posts from the searcher or from others to get a sense of how strongly that emotional state might be. A certain confidence level might need to be reached before search results would be influenced.
The patent application also notes that moods tend to be time dependent, and often transitory, so the influence of different emotional signals might be limited.
Reading through the patent filing, I’m not sure at this point how search results might be modified based upon mood. How would they differ if you seemed to be happy or sad or bored?
The Influence of Trends on Search Results
In addition to looking at social activity to try to gauge moods, they might also be analyzed to try to detect trending interests of searchers. These could include things like new songs or albums or artists or current events, or many other topics.
A keyword search of the searcher’s posts, with information about frequencies about terms appearing within their posts might indicate trending topics. If the searcher has performed queries at sites, such as a Facebook or Twitter search, those queries might also be analyzed as well to see if they are part of a trend.
Other similar heuristics, or rules, might be looked at to identify trends from an individual and possibly from their connections as well.
Identifying Geographic Places of Interest
Someone might be planning a trip, or returning from somewhere, and their social activities might indicate some interest in that location. Search queries at the search engines and at social media sites may also evidence an interest in that location as well. Like the identification of moods and trends above, activity from connections on social media sites might also be looked at:
The posts from the user’s friends may also be analyzed in the same manner in an attempt to discern or confirm a geographic place of interest of the user. It may happen that a user asks for help in finding a vacation spot or other place of interest. The replies from the user’s friends may then help in determining if the user has found a particular place of interest.
Other Items That Might be Monitored
In addition to moods, trends, and locations of interest, this system could look for other things to personalize search results based upon social activities, looking at their social activities and searches at social media sites.
These could include “events of interest” to a user, such as the celebration of a birthday, or attendance at a concert, or an upcoming test at school.
This system could also look for specific likes and dislikes of a searcher, such as “an affinity or aversion to certain topics, people, brands, stores, organizations, places and/or things.” In addition to looking at specific topics, this approach could also look at terms included within a post that might distinguish whether there’s a positive or negative feeling associated with those topics. Other signals, such as becoming a “fan” of something on a social network could also indicate a positive feeling about it.
Another aspect of social activities to be tracked might be goals or aspirations of a user. Signals that might be looked for could include statements like “‘I want . . . ‘ or ‘I wish . . . ‘ or ‘I hope . . . ‘ or ‘It would be great if . . . ‘”.
Identifying Interests of Groups
The patent filing also describes a number of ways that the interests of groups of users might be monitored, and some additional ways that information could be used.
These could be used to learn about public opinions on different topics or entities, and could be broken down into different geographic regions.
Another application could be a “prolific fan identification application.” Here’s a description from the patent filing as to how that might work:
As one example of the prolific fan identification application, a music recording artist may release a new album. The artist wants to tap into his most prolific fans, though he does not know who they are, and reward them for promoting his album. The user-following engine would identify the users who repeatedly post about the artist, which information would be stored in the all users index of the service. An intelligent processing module may be provided which scores the selected users’ posts for positive opinions of the artist.
The number of readers who see, and possibly repeat, the users’ posts may also be factored into the score. The highest scoring users could be given the opportunity to participate in a promotion. The artist would buy a promotion campaign, through the search processing environment or elsewhere, giving each prolific fan a unique purchase URL for his album. The artist asks the fans to include their unique URL in their postings about the new album. The fan with the highest performing URL (measured via sales) would get a cash prize and the opportunity to meet the artist. It is understood that the prolific fan identification application may have a wide variety of other uses in further embodiments.
Another application described might be enabling people to perform “people like you”-type searches.
Takeaways
One thing made very clear by this patent filing is how much information Bing might be collecting from the social activities of people who enable the search engine to gather that information.
What isn’t as clear is how they might customize search results based upon things like identified moods, or how they might use trend information or some of the other types of information they describe as creating from their different kinds of analysis.
We do have a screenshot that shows a couple of interesting additions. In addition to showing custom results at the top of search results (no guarantee that Bing will separate results that way), there’s also two links at the bottom left of the browser window. One enables you to access your individual preferences, and the other allows you to search for “like inferences,” presumably for “people like you” results.
The patent does provide some indications of commercial uses for this kind of information, such as helping advertisers identify people who might be influential and interested in specific topics.
What impact will the approach described in the patent have on individuals or businesses using social media sites? That’s hard to gauge at this stage in the development of Bing’s social and personalized search. The same could be said of Google’s Search Plus Your World.
Good article Bill. There is no doubt the Microsoft needs to take social signals into account in their SERPs as they are too important to ignore. But given Google with 90% market share (at least here down under) can’t seem to get penetration with Google+ it remains to be seen whether Microsoft, with even less market control, can make inroads. But they need to do something, or they risk falling behind like they have so many other times.
With Bing’s relationship with Facebook it would seem they’d have more data on users then anyone. The information related to moods, however, almost seems too farfetched. Would they be giving us a better search experience just because they anticipate our mood. Good luck trying to do that with teenagers. Interesting post Bill and look forward to seeing how this plays out.
Great data, Bill. I’m still skeptical about the relevance of a searcher’s mood as it relates to SERP’s, but maybe they know best. Seems like they’re trying to put too fine a point on it to me.
I don’t like the intuitive searches that are being instituted. I turn off the Google one. I would rather see things for myself, step outside the box I know and see what else is there. It’s so easy to only view what you already know, but it’s better to get a variety of things that make you think and expand your horizons.
That being said, I’m as guilty as anyone of choosing Google searches by default. I often wonder what Bing is like, what they are up to, yet I never use them… I should make myself get used to them and see what they have to offer me.
Google is big and getting bigger. Have you heard about their latest endeavor? Going into space exploration and mining minerals from asteroids. Sound like I made it up? Bing it! LOL
I hope all this data harvesting will lead to better search, although am a bit skeptical. When I search for cat pictures I don’t want to see pictures that my friends posted on FB, I want to see the best cat pictures.
This is really gonna hit hard in the searches ….m loving this…:)
Bing’s No. 1 result on average gets just under 10 percent of clicks, while Google’s top spot accounts for just over 18 percent of clicks according to a Slingshot SEO study last year.Dany
Hi jlawrence,
I don’t think that social search is an area that Bing can afford to ignore at this point either. Their partnership with Facebook seems to be fairly strong, and is definitely an asset that they should be taking advantage of as much as possible.
I’m not quite sure that it’s a good idea to think of Google Plus only as a social network, since Google is also tying it in so closely to things like authorship markup and social search. There have also been rumors of Google comments coming out to be used on other sites, which will be most definitely tied into Google Plus as well.
Hi Scott,
Thanks. Facebook does have a lot of information about many of its users. Google does have a lot of information about the searching and browsing and page level interactions of people who use it while signed into their Google Accounts, as well as the connections and interactions they may have had with others. I’m not sure that I’d take a bet at this point on which side had more data about their users.
Many of the other signals or factors in the patent other than moods seem like things that might provide better search results. It will be interesting to see how much of Bing’s social search is unique and how much of it ends up looking like what Google has done.
Hi Phil,
I’m still trying to wrap my mind around mood as well, and how it could be used to influence or filter the results that someone might be served in some manner. I’s possible maybe that when there are lots of social signals that show some type of very strong sentiment that might also be relevant to a query performed that those might show up as social results in a search. The patent really just doesn’t spell that out well, which makes me wonder if there’s a companion patent filing that hasn’t been published yet that might.
Hi Adam,
I’ve seen the idea of “filter bubbles” being discussed in a number of places on the Web, which involve the possibility that a search engine customizes so much of your search experience to what it thinks you might want to see that you don’t see things that you would otherwise. Here’s a web page from the author of a book that’s focused on the concept:
The asteroid thing was interesting. Don’t know if it was Google that was backing it as much as the founders of Google as individuals along with James Cameron and others.
John Stewart’s take is fun:
http://blogs.wsj.com/speakeasy/2012/04/26/mining-asteroids-jon-stewart-doesnt-want-to-miss-a-thing/?mod=google_news_blog
Hi Bob,
Don’t let your friends know that you don’t think their pictures of their cats aren’t among the best of the Web. 🙂
I agree with you though. I’d rather see things in search results because they belong there rather than because one of my friends liked them or shared them.
Hi Shony,
Right now when I log into Facebook then search at Bing, I see a few results that have some annotations under them that someone I’m connected to at Facebook has liked them. I don’t know what kind of impact it might have.
Hi Dany,
Not sure at this point what kind of influence social results might have upon percentages of clicks based upon positions within search results. I’d love it if Google came out with some figures on the topic though, since they have both authorship profile badges and social sharing images appearing in search results now.
This is some interesting stuff. I dont know if I would want Bing to customize my search results like this.
At present we have got personalized google ads already, but didn’t expect them to go even further. This is definitely something to do with cookies, so I hope that clearing the Cache and Cookies history will solve the problem or am I wrong and they will use different aproaches to gain the data?
Hi Ryan,
I’m a little wary about customization from the search engines, too. I’d rather be the one calling the shots as to what gets included, and possibly excluded from the search results that I see.
Hi Martin,
The post is about Bing, and not Google.
Hi Bill,
The reason why I mentioned about Google is because these two companies (Google and Bing) compete in a race to create a better search engine and the winning company will score bigger clientele – daily users who in order to find something on the internet normally use search engines. Many people do not know how their search results work/how they are calculated and perhaps because of personalized search a typical user would be able to find something faster – we don’t know yet. If Bing wants to win the race it must introduce a new technology. I think it’s only a matter of time when personalized search will knock to our computers. I will do as much as I can to prevent from being categorized and tracked – cleaning cache and cookies is a sort of way, but not only one.
Hi Martin.
Thanks for providing more details on your question.
The thing is, the search engines have been attempting to personalize our search results for us for a number of years now. One of the very first detailed posts I wrote about Google’s personalization methods was back in 2006:
https://www.seobythesea.com/2006/10/google-personalization-methods/
Providing social results may customize those results for us, but it really isn’t “personalization” in the way that both Google and Bing are trying to provide personalized results.
If you want to try to keep personalized ads and search results from showing up on your computer, it does help to clean your cache and cookies, as well as to not log into your Google Account when searching Google, and to not log into Facebook when searching Bing, and so on. You could even turn on the Incognito mode in Google Chrome if you use it to try to avoid personalized results.
The social results that both Google and Bing present to us are personalized in that they are based upon our social activities and social connections, but they aren’t “personalized” in the ways that both search engines consider how personalized search works. Those personalized results are based upon our past search and browsing activities, and other possible interactions with services from the search engines.
Not sure why bing not crawling my website “http://www.addenda.us/adidas/adidas_GSG_9.2.html” , pages are not updated in search result, website is regularly modified
I saw a related article about how Facebook was changing its privacy policy agreement and users would not be able to opt out. The new policy is not focused on protecting users privacy but rather an outline of they are going to use information to provide more targeted ads. Its getting crazy how much knowledge and information these companies have about us. While this is “good” for search results information sharing like this only benefits the large companies and their advertising budgets.