Back in 2005, Google filed a patent application on Ranking video articles which gives some insights into Google’s future plans for what they might do with video, and some of the possible ranking algorithms they would consider using.
Google Video search went live in January of 2005. Its focus was on helping people find videos on the Web and on television, and providing a place for people to upload videos that could be watched from the service or embedded on other sites. The patent shows a screen that would allow you to enter your local TV provider:
When the patent was filed, it appears that Google Video would be the central hub of all things video-related for Google. But those plans seem to have changed with the acquisition of YouTube, the development of Google TV, and the growth of Android.
Google acquired YouTube in October of 2006, and only recently announced that they would be stopping the uploading and playback of videos at Google Video. Within a week, they rescinded that move (playbacks from Google Video will continue). Still, they announced that they would make it easier for people to migrate their videos from Google Video to YouTube.
If you watch movies on your phone or surf the Web on your TV, the chances are that you’ve seen something about Google TV or Google Android. Google has been focusing upon an android powered device that can make it easier for people to find television programming they want to watch, and visit websites on their televisions. Both Google TV and Google Apps for Android contemplate a much larger role for video on both TVs and telephones.
If you want to develop and optimize web content for television and for television applications on mobile devices, Google has provided some optimization advice for televisions, and will be sharing information in the future about building Android applications for Google TV.
Video Rankings
Back in February, I wrote about How a Search Engine Might Rank Videos Based Upon Video Content, describing a recently published Google patent application telling us that the search engine might capture many frames from videos and use a “similar image algorithm” to tag frames in those videos with relevant keywords.
The approach in the patent filing also included a sound fingerprinting technique that would capture sounds as electronic images and match those sounds from the videos with known sounds from other sources to tag those with keywords as well.
The keywords association with images and sounds from videos could be used in association with text and metadata about the videos to index the content of those videos.
This older granted patent from Google anticipates the use of similarity algorithms like that and a wide range of other signals that might be used together to rank videos in a video search.
The video search would include videos found across the Web, videos hosted by Google Video, and videos that might be presented on television.
Queries used in a video search might be text-based, with searchers entering keywords in a search box.
Queries could also be image-based, with searches of videos based upon an image that a searcher might upload.
A video search might also be started by someone selecting content associated with another video, such as highlighting text from closed captioning data to use as a query to find other videos.
In the Google Video help page, one page titled How are videos ranked in the Google Video search results? tells us that:
Our technology examines dozens of aspects of the video’s content (including the number of hits and rating) to determine if it’s a good match for your query.
So just what kinds of things might Google Video be looking at to rank videos?
The patent provides some clues:
Ranking video articles
Invented by Shahid Choudhry, John Piscitello, Christopher Richard Uhlik, Monika Hildegard Henzinger, Matthew Vosburgh, Aaron Lee, David Marwood, Peter Chane, and Steve Okamoto
Assigned to Google
US Patent 7,933,338
Granted April 26, 2011
Filed: November 10, 2005
Abstract
An information retrieval system is provided for processing queries for video content. A server receives a query for video content and returns video articles, as received from broadcast systems or other content providers. Queries are formulated using text, video images, and/or visual content associated with a video article.
Various video-oriented characteristics associated with the results of the queries are determined, and a rank score is calculated for each. The ranked video articles are displayed in a representation to the user, from which the user can play the video article either within the representation or independent of it.
If you have uploaded a video to the Web in the hopes that people find it, and view it, it doesn’t hurt to think about how the different ranking signals listed in this patent might influence how easy it is to find that video. Keep in mind that some of the videos that might have been found through Google Video were intended to be viewable only on television, and some of the ranking signals included are aimed specifically at that content.
Video Ranking Signals
Here are a number of the signals that Google might consider in ranking videos for a video search.
- PageRank based upon links pointing to the video or the page that it is hosted upon.
- A title associated with the video
- A title for a program associated with the video
- The size of the video
- The date that the video was aired
- A category associated with the video
- A snippet (which may be a portion of the dialog relating to a thumbnail image)
- Broadcast source associated with a video
- Broadcast time period data associated with a video
- Related links
- Program guide information
- Key terms and words from audio in a video
- Closed captioning text
- Key terms or words associated with the video
- Key terms or words associated with a matching or similar image previously stored in a database
- Third-party ratings data associated with the video
- Third-party audience viewing data for a particular video
- User data associated with a video
- Textual data associated with the video
- A name associated with a video
- A name associated with a content provider
- Number of times the video has been broadcast
- Number of content providers that broadcast a particular video in a defined time period
- Particular time period a video article has been broadcast
- Particular date a video article has been broadcast
- Particular geographical location a video article has been broadcast
- Number of times other users have selected a particular video article in response to different queries
- Number of times other users have selected a particular video article in response to the same query
- Clickthrough data associated with a video
- Queries associated with a video
- Nielsen ratings data
- Thumbs up viewer data
- Thumbs down viewer data
- Replay viewer data
- Record viewer data
- Fast forward viewer data
- Review viewer data
- Production costs associated with a video
- Advertising costs associated with a video
- Advertising production costs associated with the video
- Speech-to-text conversion of dialogue associated with the video article
- Text associated with articles referencing the video
- Text associated with other videos referencing the video
- Text associated with documents referencing the video
- Length of the video article (e.g., longer equals better rank score)
- Quality of the video article (e.g., higher quality is higher ranked)
- Sound quality (e.g., higher quality is higher ranked)
- The type of video article
- Time period (e.g., time of day, or day of the week)
- User location (e.g., to more heavily weight video articles broadcast in close proximity to the user)
- Selection of the video article by previous users (e.g., users querying for the same or similar terms)
- Whether the video article is available for playback (e.g., video articles available for playback are weighted more heavily)
As I mentioned above, a number of these signals are more appropriate for a search of broadcast TV (or Cable TV) content than for videos uploaded to web pages, and could potentially be included in a search for programming through Google TV. It’s possible that many of these may or could be included in the search function at YouTube and at Google Video.
It shouldn’t be a surprise that Google is looking at the text associated with a video, such as titles or descriptions, as well as user data, such as the number of plays and ratings. The signals that may be used for television-based content such as Nielsen Ratings or broadcast source or broadcast time (prime time or daylight shows or Sunday morning news) likely won’t play much of a role in the ranking of web-based videos.
But one thing I found interesting was that pages or other documents that reference a video might also be included as a ranking signal for that video.
We aren’t given much in the way of details on how these different factors might be weighted against each other, and it’s possible that in the five years since this patent was originally filed, other signals might have been considered as well. But if you’re considering how Videos might be ranked presently and in the future by Google, this isn’t a bad list to start with.
I bet the list of factors/rank signals would be pretty saturated with social media influences right now (Facebook likes, Twitter mentions/hashtags).
Wow! The list in my head was substantially lower that this one. #Timeless #Resource #Thanks
Hi Kentaro,
I suspect that there’s a role for social media influences in ranking videos, but I suspect that Google might place more importance on data that they collect themselves directly than they might from other sites.
A thumbs-up on YouTube might just carry more weight than a tweet or a like on Facebook, if for no other reason than the data is easier for Google to collect and to rely upon. In other words, Google may be able to collect more information about the source of that thumbs-up and the person who liked a video than they can about the person who liked a video at Facebook, and may feel it to be a more reliable signal because of that.
Hi Brent,
Thanks. I was somewhat surprised to see all of the signals that the patent pointed towards as well. As I mentioned in my post, some of those are more relevant for a search that includes a program guide for television than for videos that are uploaded to the Web, but those may give us a hint about what we might see in searches on a program guide interface for Google TV (which would include both television programming and Web videos).
Video ranking is still too much text based and can be easily manipulated just via title, description, keywords and by sending some thumbs up via fake accounts, which can be seen on youtube on a daily basis, for example for real time sports footage with a date in the title that appears in search results but actually leads to a video promoting a website and not to the actual content.
I was only just reading about YouView in the UK – the TV aspects of the patent suggest Google will be more than ready for the full integration of TV and web!
Hi Kelly,
I wasn’t aware of YouView since it seems to be UK only at this point, but it’s an interesting idea. Don’t know that Google Video will include a programming guide for television in the future, but it’s quite possible that a program guide for Google TV will allow searches of television programming and online videos.
Andreas – totaly agree. The same as will happen with google +1 button!
Awesome post. What I find fascinating, looking through the possible video ranking factors, are the large number of factors based on third-party metrics: third-party ratings, third party audience-viewing data, thumbs up and down, Nielsen ratings and so on. For HTML pages, of course, the primary popularity metrics influencing rank are related to hyperlinks; the video ranking factors incorporate a much broader range of popularity signals.
Things like broadcast time period and production costs are really intriguing. Would a video of a TV show broadcast Sunday at 8:00 PM rank better than a similar TV show broadcast Monday at 2:00 PM? Does an expensive video inherently outrank a cheaply-made video?
The patent deals with video “as received from broadcast systems or other content providers” but one wonders – traditional inbound links aside – if this doesn’t put web-only videos at something of a disadvantage compared to videos produced in a broadcast environment, because of the large number of broadcast-based factors that wouldn’t carry any weight (even if this absence isn’t interpreted negatively).
It might also put web-based non-YouTube videos at a further disadvantage, or at least videos not hosted in a enterprise environment, as a number of web-based popularity metrics may not be available on the video host site – ratings, thumbs-up and down, review data and so on.
Finally, this 2005 patent makes me think there might be clues here about contemporary web ranking factors as well. While Google has been notoriously close-lipped about whether such factors as ratings, “likes” and Tweets impact web rankings, this patent embraces a large number possible ranking factors that could be regarded as “social.” If Google so forthrightly shows states an interest in these factors for video search, might that interest not also extend to other forms of search?
Bill Slawski – I stubled apon your website today. Wish I would have found this a LONG TIME AGO! You have some of the best information on the greatest topics on my mind. I subscribed to your site and I look forward to your future writings. Thanks so much!
Agree with Kelly Felix. Integration of TV on web will be implied as Google video to YouTube.
thanks for doing the digging. If they use all of those factors I cant imagine what people are going to try to do to game the new system. People think they have a handle on website algorithms, this is a totally new ballgame.
I read a book a fews years ago that suggested that google was going to really integrate it all so that what you watched brought up the ads that were most appropriate to your viewing in total as well as provide automated checkout options etc. They want your total dollar and once they get into tv, they may very well have it.
Thanks for sharing this comprehensive list. Video optimization is just as important as text content optimization since videos rank in the search engines too. This is especially true for businesses that rely heavily on promotional and how-to videos as a part of their marketing mix.
Bill, brilliant as usual. Thank you so much for the concise summary of the value points from this patent!
I see that there are some “easily gamed” factors (some emphasized in the other comments). I suspect that these are confounding variables that, in practice, won’t matter. But rather, might be red herrings for SEOs. I know that I would include red-herring trails in my public documents if I were the big G, in order to draw out the webmasters that may attempt to manipulate my SE (without paying my company).
Have you seen that sort of behavior before? Agree with my conspiracy theory?
Yeah agree with Brent…wow. Very comprehensive list of factors most of which I’ve never considered. Thanks Bill.
Hi Andreas,
I agree, video ranking is still too much based upon text associated with a video rather than on the content of the video itself. That might not change today, or tomorrow, but I think it will change sometime in the future.
Hi John,
We have yet to see how Google will use the +1 button when it comes to ranking websites. It may measure the number of people who clicked on a button, but it doesn’t necessarily mean that one search result is better than another in response to a query.
Hi Tony,
Thank you very much for your kind words.
Hi Aaron,
Great observations and questions. Comparing broadcast videos to web only videos when ranking them does have the feel of comparing apples to oranges. Would it be a smart move for a search engine to allow those searches to be independent of each other – so that for instance you could search for television viewing only, or web videos?
One of the help pages at Google Video insists that special preference isn’t given to videos found on Google Video or at YouTube, and I think it would be a mistake if it was. But, Google probably has more user-behavior data about those videos than videos that might be found elsewhere, so there might be a possibility that something like that could cause a bias.
There may be some assumptions behind how social signals might be used for video that could have transferred over to how social signals might be used for other types of content as well. Since it’s difficult to view and rank the actual content of a video, more weight might have been given to social signals than might be given to the contents of things like Web pages. But, the use of social signals for videos may have taught the search engineers something about how they might be used for other content as well.
Hi Tessa,
I’m not sure if we will see Google Video start showing us television programming results anytime in the future, but it’s quite likely that searches on Google TV will include both television programming and web video results.
Hi Bruce,
Interesting thoughts. I suspect that analytics for advertisers and site owners might be a comp;letely different animal when it comes to TV ads than it was in the past. Google would be able to provide a lot more information about actual viewership of specific programs, and information about who it was who was watching those programs. Ads could definitely become a lot more interactive as well, including as you note, the ability to purchase something immediately while watching.
Hi Nick,
I think universal search did a lot to raise the value of videos as a marketing tool on the Web. A smarter and more useful video search could improve that as well, and would likely filter into organic searches with even more relevant results. It’s probably not a bad time to think about video marketing as a site or business owner, if you haven’t started already.
Hi Glenn,
Thank you. I do agree in part with your conspiracy theory.
Patent descriptions are illustrations for the patent office of how something may work, and they may include examples that aren’t used in the final version of some process or invention, and may leave out possible others. Some other features might be added or removed during the development and implementation of a process or after some testing.
A patent like this might provide a number of hints of what we might see out of something like a video search, but chances are that the finished product would be different in a number of ways. Some of those might be intentional, to lead competitors or other interested parties down some false paths, but chances are that many examples that might not be used weren’t necessarily intentionally misleading.
With possibly a few exceptions, I try not to rely too much upon the desciptions of how something might work as described in a patent without some thought and testing and observation.
Hi Steve
Thanks. I was a little surprised myself once I looked at the length of the list of possible ranking signals that I pulled out of the patent.
Right on. (& Mad props for responding to the folks in your comments)
Hi Glenn,
I really enjoy receiving comments and responding to them. The discussion is part of the fun of blogging to me. Thanks. 🙂