When you walk into the lobby of Building 42 at the Googleplex, you can see a display that shows you queries entered into the search engine at any one time. It’s a mesmerizing sight, and I found myself wondering about the people and motivations behind some of the search terms I saw flowing down the screen.
Imagine that instead of seeing one query at a time, that search information was analyzed, and queries were bundled together, to maybe provide us with more meaning.
Can search engines be used to tell us what the world is thinking at any one time? Would looking at the most popular search queries that people type into a search engine provide us with some insights?
Most Popular Search Queries Information from Search Engines
Popular search query information is available from the major search engines, though some services created to provide us with a look at what people are searching for the most.
Google gives us quite a lot of information with a number of services, including Google Trends, Google Trends for Websites, Google Insights for Search, and Google Hot Trends.
Yahoo also lets us look at the words people are searching for with a service that is updated daily – the Yahoo Buzz Index.
Ask shows off weekly search information on their IQ – Interesting Queries page.
None of the information from any of those services is updated in real-time, or near real-time.
Presenting Trends About the Most Popular Search Queries Visually
Imagine being able to see the most popular search queries from different parts of the world as they are searched for, and being able to break those down into their geographic origins or by demographics for searchers, and being able to look at the pages that people clicked upon for those queries.
A new patent filing from Microsoft describes a way of visualizing search activity, using log file information from the search engines, and capturing search keywords as they are used, or very close to a real-time capture.
The screenshots from the patent filing aren’t of the highest quality, but the one above is a variation that could be used which shows a map of the world, with bubbles floating from the bottom from different parts of the world, standing for specific popular queries from those areas, with the sizes of the bubbles indicating how popular the queries are.
Those bubbles could be clicked upon, to find out more information, such as which pages people clicked upon to see information about those queries. The bubbles can also change in size as they float up to the top, to show changes in the popularity of each query.
The information presented is taken from Microsoft’s search log data, and it represents what Microsoft calls a real-time “pulse of the Internet.”
The patent application is:
Internet Visualization System and Related User Interfaces
Invented by Min Wang, Weizhu Chen, Benyu Zhang, Zheng Chen, Jian Wang
Assigned to Microsoft
US Patent Application 20080256444
Published October 16, 2008
Filed January 10, 2008
Different types of information that could be shown might include:
- Geographic origins of queries,
- Demographics of Internet users submitting each query,
- Most popular queries for a given geographical region or demographic category,
- Clicked-on links associated with each query, and;
- Displayed pages of the clicked-on links.
This system may also allow viewers with multiple views of the information presented, by allowing them to filter it based upon geographical and demographic information. I’m not sure how they might capture some of this information, but they tell us they they could filter by:
- Gender,
- Age,
- Nationality,
- Organization,
- Interests,
- Income,
- Profession,
- Hobbies,
- Shopping habits, or;
- Web pages linked to.
Privacy and Removal of Some Information.
We are given some information about the kinds of things that might not show up in this visualization of Microsoft’s searches.
The patent application doesn’t go into much detail on privacy, but it does touch upon the topic a little.
We are told that an anonymizer might strip some queries of private and personal information, and other filters might be used to remove:
- noise or repetitiveness in the queries,
- Unintelligible queries, or;
- Undesirable queries, such as search queries that include illegal or undesirable requests.
Conclusion
I’d love to see more detailed information from all of the search engines on what they consider to be the most popular search queries. I suspect that all of them may provide more detailed looks in the future.
I’m with you, I’d love to see more detailed information from all of the search engines on this topic. I use hot trends the most, but sometimes it appears a bit slow. Imagine watching one of those bubbles grow and/or rise as a term became more popular. Seeing something like that in real time would be very cool.
While not a search engine, another place I’ve found trending topics is search.twitter.com. The list is small but I compare terms there regularly to those on hot trends.
Nice post Bill. It reminded me to update my keyword research and competitive intelligence resources on my website. I had to add “Google Insights for Search” hehe.
I also list a bunch of Microsoft Labs tools btw. that are related to this topic among tons of other tools and services that provide interesting figures and trends.
You might want to take a look, because you might learn about some new ones that you didn’t know of before.
http://www.cumbrowski.com/CarstenC/seosem_keyword_research.asp
http://www.cumbrowski.com/CarstenC/seo_competitive_intelligence.asp
Cheers!
Carsten
Hi Michael,
I love the idea of being able to watch those queries in real time, and see changes take place. I found it interesting that the “hot” topics in Google, Yahoo, and Ask all seemed to be different, and I’m wondering why.
I hadn’t paid much attention to the topic trends at search.twitter.com., but that is a pretty interesting application – instead of search behavior, it’s looking at topics that show up in conversations. Great stuff.
Hi Carsten,
Thanks. Those are a great couple of pages that you’ve put together. Thanks for pointing them out.
I probably should have included the Microsoft Labs tools in my post – I guess I was engaging in some wishful thinking, hoping that they would make this new application available to the public sometime soon. We don’t know if it’s something that they have developed yet, but I’m guessing that the biggest impediment might be in collecting and analyzing all of that information in real time.
Bill, it would probably make its way into some tools at adLabs first, so we have to keep an eye on what they do there. 🙂
Some of it makes it then into their official products (sometimes) or becomes a product of its own… like their PhotoSynth, which is somewhat related to search… image search to be specific. Did you ever have a look at that? Pretty fascinating tool.
PhotoSynth was a Live Labs project for a long time. Now its out of beta and moved to its own website. See PhotoSynth.net (or .com)
Hi Carsten,
Thanks.
Photosynth is pretty nice. I knew about it, but hadn’t heard that it had come out of beta.
Starting programs out at places like Live Labs first makes sense. I haven’t been there in a while – the Political Streams project looks interesting. I had missed that.
I actually worry that this could have a “chilling effect” on the use of search engines by people if they knew this type of detailed information was being broadcast about search engine use and search queries.
Especially since you noted that the patent application doesn’t go into much detail on the issue of privacy.
Hi Peoplefinder,
That kind of chilling effect is a possibility. I don’t think that it would hurt for a search engine to stress as much as possible that any time they might display information like this that they would try to protect the privacy of people, and that they would only display information in an aggregated form that works to protect the privacy of individuals.
I do wish that the patent application went into more details about privacy, but like most patent filings, it provides an overview rather than a detailed look at how the processes involved might actually work. The difficult part in bringing something from idea to actuality is in the details. Hopefully, if Microsoft pursues this idea, they will take privacy into consideration.
Hi Bill, Bing xRank uses Live Search to count popular web searches, and tell you who or what the world is searching for most:
http://www.bing.com/xrank
Thanks, SEO Practices. 🙂
I probably should have included the Microsoft tools in my post, too.
Hi Bill,
Your blog is very nice. Good style of writing.
Have you seen WeFeelFine.org? It is sort of what you are asking, a window into the way the world is feeling. I found it after watching a TED lecture/video (something else I only discovered a couple of weeks ago). Great stuff. Wefeelfine.org is mesmerizing at first, and after a while you (well, I) really start to feel like you are listening to millions of voices calling out to the world.
Hi Jon,
Thanks. That is a pretty interesting visualization. It does provide something of the same feeling as the display I wrote about in the lobby of Google. Some really great videos on TED as well – I’ve been visiting there for a few years, and found the vast majority of presentations always brought me some new way to think about the world around me.