Chances are that when you search for a video on Google or at YouTube, the results that you receive are based upon text about the video rather than the content of the video itself. The search algorithm involved might look at the title of the video, as well as a description and tags entered by the person who uploaded the video as well. Annotations on the video may also play a role in determining what terms and phrases the video may be determined to be relevant for as well.
For example, the video below announces Google’s new food recipe search option, and provides a detailed description about the new feature. But none of the text accompanying the video mentions that the person providing details about Google’s added functionality is one of Google’s executive chefs, Scott Giambastianai. If you search for [Google executive chef], you wouldn’t see this video appear in YouTube’s search results and you probably should.
There have been a number of patent filings and whitepapers from the major search engines over the past 5 or 6 years that describe how they might use Web page blocks or segments to understand things like the main topic or topics on a page, which block might be the most important for a page, what to show on smaller screens for mobile devices, and to apply different weights for links depending upon which block they are located within.
I’ve written about a number of those in the past in posts such as:
If Google decided to include a facial recognition search as part of the Visual Search described in a Google patent application a couple of weeks ago, a couple of questions need to be addressed by the search engine.
One is, where would they get the pictures to power that facial recognition software (hint in the image above)? The other is, how would they best avoid privacy concerns?
A patent filing from last week provides some possible answers.
A month or so after that patent was granted and I wrote my post, Google researcher Steven Baker published a blog post at the Official Google Blog titled Helping computers understand language, where he announced that Google would start including synonyms for query terms in search results when the search engine thought that the synonym was a good match for a query term.
Google Goggles lets you search by taking a picture of landmarks, books, business cards, artwork, product labels, logos, and text. It can use Optical Character Recognition to transform text in an image to searchable text on the Web, reads barcodes, finds similar images in databases of artwork and landmarks and other databases. But, we’re only seeing the surface of the capabilities that a phone based visual search can offer with Google Goggles.
A Google patent application published this week shows us what Google’s visual Search for phones might evolve into. When you take a picture of a city street, your picture may include buildings, street signs, people’s faces, cars, and many other objects. If you send that picture as a query, the search engine might break the image into parts and search for many of the objects in the image, and give you a mix of search results based upon all of those parts.