Timing is everything. On Monday, I was asked if I would give a presentation at the Internet Summit 2011 in Raleigh, NC on November 15th and 16th on an advanced SEO topic. I thought about it, and agreed, and decided to give a presentation on how social media has been transforming search on Tuesday night. On Thursday morning, Google delivered a present in the form of a Blog post at the Official Google Blog titled Giving you fresher, more recent search results.
The title for my presentation is “Next Level SEO: Social Media Integration” and the basic premise is that social media has changed the expectation of searchers. Searchers want fresher content, they want to see what their friends and contacts have to say, and they want access to experts and authority figures and their thoughts on timely events and news. Search engines have no recourse but to respond.
I didn’t see the Google blog post until yesterday afternoon, and quickly wrote up some of my thoughts at Google Plus regarding Fresher Results at Google? There are a number of other very thoughtful reactions to the change, and I thought I might point those out, and maybe expand upon my thoughts from yesterday.
Here are a few of the posts that captured my attention:
Gianluca Fiorelli explored the reasons for the freshness update and some of the history behind it in Completing the Google Puzzle, almost. This snippet from the post puts the change into perspective:
Google is an substantially an editor (even it will never recognize it) that sells ad spaces, and Search is still its main product to sell toward the advertiser. So Google needs to have the best product to continue selling ad space. That product was ironically endangered by Caffeine. With Panda Google tries to solve the content issue and with the Social Signals linked inextricably to the Authors and Publishers tries to solve the obsolescence of the Link Graph.
Justin Briggs digs deeply into some of the methodolgies that Google may be using to deliver fresher results in Methods for Evaluating Freshness, and his analysis of how Google looks at a document inception date sounds spot on, as does his description of the use of microblog data to uncover fresh and bursty topics and pages using social media.
I also like the SEOmoz presentation from Rand Fishkin and Mike King, Google’s “Freshness” Update – Whiteboard Friday, which covered a number of the implications of the update for search marketers.
In my Google Plus post, I mentioned three alternative approaches that Google might follow either independently or in combination to deliver fresher results.
Using Social Media to Find Bursty Topics and Associated Pages
One of them is to grab information from social streams such as Google Plus, Twitter, and others as quickly as possible to find recently popular topics that people might search for. I wrote about a Yahoo patent that explores this approach in the post Do Search Engines Use Social Media to Discover New Topics? It’s not hard to imagine that Google would do something similar with Google Plus posts at the very least, if not also from crawling sites like Twitter.
As I wrote in that post:
The patent filing explores “recency-sensitive†queries, where searchers are looking for resources that are both topically relevant as well as fresh, such as novel information about an earthquake. If you’ve been watching twitter streams, Facebook updates, and other social media, you’ve seen that sometimes these sources are the best and fastest places on the Web to find that kind of information.
It’s possible that a search engine that ignores sources like those isn’t going to be able to return any relevant results for those types of queries – what the patent’s inventors call a “zero recall†problem.
If I want to find information about very recent events, such as the things going on at occupy wallstreet or occupy oakland, I’ve been finding it much easier to type the hashtags for those into twitter than search at Google or Bing or Yahoo. At this point, Twitter still seems to be very much ahead of Google’s new freshness approach in pointing out recent pages about those protest movements.
Using Distributed Crawling Through Browsers to Find Bursty Topics and Associated Pages
Another approach that Google could follow to get very recent information to show to searchers, and one that circumvents social media itself and looks at browsing information directly. The search engine WOWD developed this approach over the past couple of years without the use of a web crawler. Instead, it used information gathered from a browser plugin to see what links people are clicking upon. If a lot of people started clicking upon certain new links, those might indicate hot topics and pages that should be shown to searchers.
Google acquired the patents from WOWD in late June of this year, and I wrote about the acquisition last month in Wow! Google Acquires Wowd Search Patents. As I wrote in that post:
This is an interesting group of patents, from their approach to distributing crawling of web pages by people with the application installed, to ranking “fresh†or “popular†recent pages, to the recommendation system Wowd has developed. It’s possible that Google may have purchased these patents to protect some of the things that they may have already been doing or were planning on doing, or that they may implement some of the technologies described within the acquired patents.
If Google were using the WOWD approach, I might still expect to see fresher results for the occupy protests that I am right now when I search at Google.
Caffeine and an Updated Change Analysis to Find Fresher Content
The third alternative is a change to the way that Google handles fresh results under the patents that came from Google’s Historical Data patent, which are intended to both address the problem of stale results and some spam results in response to queries.
Justin’s post discusses a number of the issues that are described by these patents, such as how freshness might be determined by things such as when a page was first crawled. Google did recently file for a divisional patent on one of the patents from the historial data patent family which described a change in the way that Google might determine whether important changes might have taken place on a page over time.
I wrote about that change in the post Revisiting Google’s Information Retrieval Based Upon Historical Data, which looked at changes to a patent titled Document Scoring Based on Document Content Update. How does the updated version differ from the original version?
The original claims for the patent told us that Google might ignore the “boilerplate†language it finds on pages, and the changes to those. In the newer version, instead of mentioning the word boilerplate, the patent tells us that it might calculate the frequency with which words appear on a page (excluding stop words), and look at changes to the section of a page that contains the most frequently used words.
So, for a page that has been around for a while which has updated information included in the main content area of that page – which often is the section of a page which usually contains that most frequently used words – the changes to that page might make it appear more fresh than its original document inception date, or when Google first crawled the page. This change, which doesn’t require a boilerplate analysis, but rather may just look at changes to a main content area is probably something that can be processed much faster under the incremental update approach from Google Caffeine.
The section on Google Caffeine in my post Son of SEO is Undead (Google Caffeine and New Product Refinements) describes some of the changes brought about by the Caffeine update:
In a nutshell, those changes involved:
- Reducing the default sizes of blocks within which files are stored in the Google File System, from 64mb to 1mb, which enables hard drives to hold considerable more information when they contain small files.
- Allowing metadata on a master server to be distributed to multiple master servers, so that searches can more easily be split into parts.
- Placing information about specific pages on multiple smaller files instead of one larger file, and only updating the parts that change for a page instead of all of the information about that page.
Instead of a more complicated analysis to identify changes in content, the new approach may just look at the main content area of a page to see if the content contained within it is fresher.
For queries where the average age of top results are fairly fresh, as influenced by substantial changes to the main content area of those pages, freshness of certain results may move those results up higher in rankings. Those may have to compete against very relevant new results as well.
Under that approach, if we perform a search for a bursty topic like [occupy oakland], we should see within search results some pages that may have been around for a while but which have some fairly new fresh main content as well as newer pages. They might not be as fresh as new pages pointed to by lots of tweets or Google Plus posts, but they should work towards solving the “zero recall†problem mentioned in my quote above for recency-sensitive queries.
Conclusion
Some topics demand very fresh content, and social media has raised the expectations of searchers by providing them with a way of finding fresh information on natural disasters, breaking news events, and other topics that are recency sensitive. If you search at one of the search engines and get no results for those topics, that might seem like a failure on their part. Google’s move towards providing fresher results was pretty much demanded by the expectations of searchers who want fresher content.
The three approaches that I’ve described that Google might use to display fresher content directly into their search results may be methods that the search engine is using, or could be using in the future. Of course, they could be looking at other information and following other approaches as well.
If you get the chance to see my presentation in Raleigh in a couple of weeks, I’ll probably be discussing this topic more and other things involving how the search engines may be integrating social signals into search, and I’ll look forward to talking with you about them there. If you can’t make it, let me know what your thoughts are on the topic below.
Great post Bill. I really hope Google can get this down and do it well. As you pointed out, Twitter is still a lot better for finding fresh content. That being said, it sucks for finding older content. I can’t tell you how many times I’ve tried to find older content that I wanted to reference on Twitter and the system can’t find it. If Google can give the best of old and new they’ll definitely be top of the totem pole. The key is getting to that point.
Hi Bill,
I just wondered why you don’t open links in new pages on the blog?
Best regards,
Mike
PS thanks for being awesome
Gadzooks. Thanks for this – it’s amazing how much information and analysis is already available on this update (just looking at your various links here).
Lots to think through as (for example) I am in a family of media websites with a range of different search traffic profiles – some terms more evergreen and others more likely to be affected by freshness requirements.
I agree with you when you said that it was a failure for search engines if they can’t produce the up-to-date information that users are looking for.
Search engine crawlers must really work faster on up-to-date and breaking news.
Search engine users are getting more familiar on how to use proper keywords and they rely so much to this that’s why social media really took part in changing expectations of users.
Three months is Three years in SEO so chances are, upgrades will be more dynamic.
This is interesting. When I read your stuff, I’m always left with a lot to think about. Mainly, how can I use this to benefit a client? I have a 20 something guy who I was thinking of having do Tweets to build trust and such; a little snippet, a photo and a link. Regularly done, will provide benefits. Now Google+ may provide another avenue for this type of integration that he may enjoy. The freshness has to somehow point back to his blog where customers come.
Kris
Fresh and up to date information it’s good, but I don’t want to see lots of tweets or Google Plus posts in my search results
I meant to say just NOT have the commercial attractiveness.
why do you say that Timothy? I think at least the choice to see tweets would be great!
I have looked for research on this one before but have so far failed to obtain any useful metrics. It is a curly one.
In the same way that some people do not want to see twitter post in their search, like Timothy, some of us want new tabs and some do not. Some probably even want a mixture but that is of course not going to happen (anytime soon) until we get a search engine that is truly personalised, not like google where they base it on personal trends but user defined with the ability to say that on this search I want information, but on the next I am looking for reviews so the friend comments are more relevant or whatever. Then I do a news search and get just that fresh breaking stuff. Perhaps something like that just would have the commercial attractiveness of Google et al.
Mike
Hi Derek,
You’re welcome. I’m a big fan of evergreen content and blog posts and pages that act as useful resources which retain their value over time. I definitely think people do need to consider how this increased demand for freshness on the part of searchers, and increased interest by Google in showing fresh results for some queries might impact what they create on the Web.
In your situation, with sites that focus upon both fresh results and upon more timeless pages, it sounds like this change might impact traffic to some of your pages. I’d be interested in hearing back from you to see how much it might, if you’d be inclined to share.
Hi Mike
Thanks. I have been using the sidebar time restricted searches that Google provides, as well as the custom date searches a lot, and wishing that Bing/Yahoo had something like that as well.
When Google was showing instant results, it was possible to find very recent information from Twitter and other sources there, and I hope we see that come back sometimes soon (perhaps powered by Google Plus), even with these fresher results from Google. If Google can manage that, I agree with you completely that it would be the best of both worlds.
Hi Mike,
I’ve never liked it personally when a site owner decides that it’s best for me to open a new window when I click upon a link on one of their pages, and I don’t want to do that to people who visit my pages if I can help it.
I know that opening new windows makes it a little more likely that people will stay upon my pages, but I figure that if I offer enough value to them that they would return if they go off to somewhere else that I might link to. The better the quality of my content, the better the links that I might curate, the more likely that people will return.
Thanks for your kind postscript. 🙂
Hi Ashley,
There are times when I pick up the morning newspaper and see articles on stories that happened yesterday, and I find myself wondering what took them so long to report upon it, and why they don’t have the latest information that I saw on twitter or online news sites. Then I remember that the event only happened yesterday.
Social media has changed our expectation. During the Virginia earthquake earlier this year, I was tweeting about it instead of running outside. I’m seriously second guessing that in hindsite, but a tweet can carry across the internet faster than the media can report a story like that, and people might see a tweet and expect to be able to find something more about it online. I know I hit Google almost immediately after everything stopped shaking to find out more. I had to return to twitter to get the latest information.
I do expect Google to start showing even fresher pages in the future. One problem that they have with showing very recent content is that those types of pages don’t have links/pagerank to help the search engine know how to rank them. That’s one of the reasons why social signals, like a user rank, might be important and essential in deciding what to display in search results.
Hi Kris,
One of the reasons why I like looking at patents and whitepapers from the search engines isn’t necessarily their description of a specific event, but rather what it might tell me about the assumptions they might make behind certain inventions or patents. And the more questions they raise, often the better. And many of those do involve how that information might be helpful in helping my clients as well.
Google has described a “user rank” in papers about Google Confucius and the Open Social Graph that they were exploring that would rank content found in a social network based upon a combination of contributions someone might make to the social network and the value of their meaningful interactions with others there as well. See my post and the papers linked to within it: Early Google Circles and the Google Social Site You Might Not Know About for more on that.
Things like Google’s authorship markup can help associate social activities and user rank at Google Plus with the content that might be created by those authors in articles and blog posts as well.
As I noted in my response to Ashley, for very fresh results Google doesn’t have the kind of ranking signals to use that it might for older pages, such as PageRank type information. So social signals may have to help fill those gaps, and things that might have been linked to by people with higher user ranks in specific topics might be the kind of thing that Google can look at to help them with those rankings.
Hi Timothy,
I think we might start seeing more Google Plus results when Google may not be able to find relevant web pages, and the queries being searched for are very recency-sensitive. When there are webpages that may adequately address those queries, I suspect that Google would rather point to those.
I also believe that we will start seeing real time results return sometime soon, once Google is satisfied that Google Plus posts can be helpful in showing meaningful content for very recent events and news. It’s possible that may not happen until there are enough people on Google Plus who might write about those things to fill the demand for them.
Hi Ron,
I did really like seeing tweets in Google’s real time results. Would love for Google and Twitter to renew their deal to have Google be able to access that information again.
Hi Bill,
I could not agree for more.
Google and other search engines are in the business of pleasing their customers – the searchers and those who use their search engine services.
Google panda and all other algorithms are needed to bring more relevant and better websites…
And this website of yours is one of those! Cheers!
JP Montero
Hello Bill
As always, a stellar examination of Google’s direction on incorporating social into search. I will hope to see your Raleigh presentation on Slideshare.
I agree that Google is looking for ways to capitalize on other channels to provide relevant, as in current, information for current topics in addition to results from a more complex set of calculations. The social channels are a like a flash mob and are short-term visitors in any results set making it harder for SEO firms that rely on keyword placement as an indicator of success. This would also support the underlying conspiracy theory behind Panda that the transition from off the page to on the page factors for relevance is to reduce the opportunity for “Tin Foil Hat SEO” (to quote Google Webmaster forum). Social channels become the wild cards that force the SERP issue back to having a strategy, good content and realistic expectations.
Or so it seems to me. m
Hi Mike,
Good points. A couple of quick thoughts in response to your comment.
Google does seem to have some features that they’ve made more commercial by adding advertisements to them, and others that they’ve kept advertisements away from, such as Google Earth, Google News, Streetviews, and some others. I’m hoping that’s something that they will continue to do.
There has been research that people don’t click on tabs for some of the specialized searches that Google provides, but still want to see that kind of information, which is why Google introduced Universal Search – so that we could see results for images or videos or maps or more when we just perform a Web search. I don’t think we’re going to get away from them doing that anytime soon. Having realtime search and fresher results seems to feed this desire, without requiring people click on additional search tabs. You can still click on those tabs, but you don’t always have to unless you want more focus in your search.
Personalized search is even more in its infancy than websearch, and it’s likely that we are going to see the search engines doing a lot of experimentation with it. Google news does allow you to specify the kinds of things that you might like to see more of, and that’s a concession I think to your point about allowing some user defined element into personalization.
Hi JP Montero,
Thanks for your kinds words.
Google Panda and this new freshness update seem like some very major changes at Google, but I think the authorship markup change will have an even bigger impact, even though it might not have been written about as much.
Hi Bill,
I saw a lot of news about this update. Do you have some information about the location of the update. Is it worldwide or not ? I don’t see anything in France (no trafic drop) in my websites. It has been viewed as a big update, so I was wondering what sites are earning/losing trafic. Do you have this information ?
Thanks
Damien.
It seems like Google has implemented so many changes in 2011. I agree Google will try to improve its search result for its own products and to get the trust of its users. It may possible that from two main changes of the year which is panda update and Fresher result, It will get new Adword customers as people will get frustrated with their fluctuating ranking due to these changes. What you think?
I think there may be a little Japanese in me. Change is great but not too much and not all the time. I like stability and sameness as long as it works. In my mind, Google wasn’t broken. All this tinkering may end up with some backlash with users who like stability.
This freshnesss issue and the need for fresh content is going to add a ton of work to the plate of SEOs if they can’t build true social media communities (where the users provide the content).
Hi Damien,
I’ve seen some additional coverage about the update, but no information about how wide the rollout of it might have been, and whether or not it only affected sites in certain countries or languages.
A search metrics post indicates sites that they beleive had benefited and sites that may have been harmed because of the update:
http://blog.searchmetrics.com/us/2011/11/06/google-freshness-update-many-winners-few-losers/
Hi marianne,
Thank you. I am thinking about publishing my ppt presentation to slideshare after the event.
Ideally a site ower will be the one to engage in social activity on behalf of his or her site, with some guidance and help and coaching from people with more experience, and that should involve a thoughtful strategy as well as the content and expectations that you mention.
Hi Alex,
I think the focus of Google’s updates and changes isn’t upon getting more paid search customers, but rather improving the quality of results that is shows for searchers. As I mentioned as a main thesis of this post, the expectations that people have for search are changing because of social activity, and they want fresher and higher quality results when they perform a search.
As site owners, I think we also need to be aware of this change in searchers’ expectations.
Hi Kris,
There have been a lot of changes at Google over the past year, and the number of experiments reported taking place at the search engine numbers in the thousands. I agree with you that so many changes may find an impact upon searchers who like stability, but I suspect that Google believes those changes have been necessary, especially with Facebook at Twitter as possible competitors when it comes to being places where people find very recent information.
Hi Greg,
This kind of change could definitely impact the workload of SEOs, but I believe a smart approach might be in acting as consultants who help put strategies in place to make it easier for site owners to provide fresher content, to engage in social activities on the Web, and not to try to do that work for them.
Very interesting read. Googles actions continue to make sense though, in the way they are tailered to cater for the people doing the searching. When I search for things, I almost always want fresh content. If it’s something really obscure, I don’t mind if the content is old. I just hope it encourages publishers on all levels to improve quality, rather than quantity of content output.
is real time search really a need?
g+ search is quite real time – how will that influence search?
It seems the seo community is using this to its spammy effect and there are so many seo articles appearing with terms in them like Xfactor or Kim Khardashian, I wonder if Googles filter will not just work on relevance of a subject but how that site is becoming spammy towards a subject just to take advantage o QDF?
Hi Spinner,
Thank you.
There are times when you do want to find content that is older, and I believe Google tries to understand when that might happen, but for queries that do evidence in some way that fresh content is desired, it does make sense for the search engine to try to serve it.
Hi Pradeep,
Real time search seems like something that searcher desire increasingly. Google has access to query logs that likely show things like the volume of people searching for news about recent events and recency sensitive queries as they happen, and if they are focusing upon showing fresher results, they likely have the data to show that people want to see that kind of information.
Google Plus search may be realtime, but if people are searching at Google’s Web results, they aren’t necessarily seeing that information unless Google starts incorporating Google Plus results into Web search. There have been some mentions of Google showing real time search results in the future with Google Plus content.
Hi Alan,
There are many SEOs who publish on the Web, as well as many people who spam the Web who might call themselves SEOs, but there is no “SEO community” per se that performs spamdexing activities as a coordinated effort. There are many spammers and webmasters who also may be responsible for manipulative activities who have nothing to do with the practice of SEO and don’t claim to be SEOs.
There are spammers who might try to take advantage of articles or social media that might focus upon terms that many people are searching for, and Google likely will be taking advantage of things like authority signals to try to filter out content created by disreputable or thin social or author profiles. That’s one of the reasons for Google to do things like set up authorship markup that can tie Google Accounts to content published on the Web.
Excellent post – a great summary. I’ve seen lots of changes in the UK SERP’s, although I’m not sure that this is all down to the fresher update as I’ve seen a lot of big, predominately offline brands start moving up within rankings for generic terms.
I find the role of social signals within the fresher update very interesting, this could be a big step for personalised search.
Great summary and explanation of what’s coming. I wonder how they will filter out all of the useless conversations that will occur in Google plus from a general search for a particular topic?
How did the presentation go, and will we get an online version?
Hi Paul,
Thank you. Google does do so many updates and changes (an average of 2 a day or so), that it is hard to pinpoint exactly why something might have changed.
I do think we will continue to see more changes come about in search results based upon reputation signals associated with specific authors based in part on their social media activities, and in the use of social signals to help identify trending topics to determine when to show fresh results for many queries. I think we will be seeing more of a role for those in personalizing results as well.
Hi Josh,
Thanks. It does seem likely that Google will use a combination of relevancy ranking and some kind of topic-based reputation score/user rank to filter out some conversations from Google Plus. It’s probably worth doing some searches on Google Plus itself to see what kind of content, and from whom, shows up in response to different queries, to see whether the rankings they use there might be somewhat similar to what shows up in web searches.
Hi Vince,
Thanks for asking.
I like Raleigh a lot, and I enjoyed visiting.
The presentation went pretty well, and I wrote a post which includes my powerpoint from the presentation here:
https://www.seobythesea.com/2011/11/the-integration-of-social-media-into-search-results-and-rankings-internet-summit-2011/
Great post! Social media is next wave for SEO that’s exploding all over!
Hi Carlos,
Yes, but practices like buying facebook friends and likes and twitter followers, etc., is another form of snakeoil. The search engines are looking for authenticity and reputation in social media, and real interaction rather than just spam.