Earlier this month I wrote about a granted Google patent, and a continuation of that patent filed earlier this year, that describe How Google Might Suggest Topics for You to Write About, by providing information to web publishers on queries and topics that are either under-represented in search results or where there’s more demand for information about those topics or queries than there are search results to meet that demand.
The topic struck home with several people, especially journalists, and I had a chance to have a conversion with Financial Times (FT.com) reporter Kenneth Li about Google’s patents. The Financial Times ran with two different stories on the topic (Google shadow over new media groups, and Google eyes Demand Media’s way with words), focusing primarily on how the technology involved in the patents could bring Google into competition with companies such as Demand Media, Associated Content, and AOL.
While searching through patent filings this morning, I came across an interesting newly published patent application from Demand Media. In the FT.com article on Demand Media, we’re told that:
Demand’s secret ingredient is a set of algorithms that analyses search engine data and traffic logs for topics to identify keywords for which advertisers are likely to pay.
The new patent filing may just give us a glimpse at some of the algorithms that are used to identify keywords selected that Demand Media might use on sites like eHow and Youtube.
The patent application is:
Method and System for Ranking of Keywords for Profitability
Invented by Byron William Reese
Assigned to Demand Media, Inc.
US Patent Application 20100153391
Published June 17, 2010
Filed December 17, 2008
Abstract
A physical computing device receives information regarding the total number of people who are searching on the search term. Information is received regarding an amount advertisers pay for the search term. Information is received regarding a click-through rate of the search term. A traffic estimate of the search term is determined. The longevity of the search term is determined.
What’s interesting about it is that many of the factors that are used to identify keywords are the kind of signals that people performing SEO and Internet Marketing are also likely to use as well.
The patent filing goes into a fair amount of detail on which factors they might consider to determine how profitable specific keyword phrases might be, such as:
- What kind of search volume there might be for specific keywords
- How much advertisers might be willing to pay for those search terms
- What the likely click rate might be for those terms
- How much competition there might be for the terms
- How much long term value, or longevity, there might be for the terms
- How reusable is the content created around the terms
- How likely will that content help attract or create inbound links
Many of the processes described in the patent filing make some sense in determining idea keywords to target, though there are some interesting assumptions made about a few of the factors involved, such as competitiveness.
For instance, we’re told that Alexa ranking numbers might be viewed for websites that appear for certain queries, as part of the process to gauge competitiveness.
There may be a couple of issues with using that approach. One is that Alexa rankings are taken from tracking self-selected toolbar users, and those rankings can widely misrepresent how popular a site might be. Another is that the rankings of pages for particular terms and phrases at the major search engines are based upon a mix of how relevant and how important a page might be for a particular term, with the possibility of additional filters and reranking methods boosting the rankings of some pages. An Alexa ranking isn’t a very good indication of how well a particular page might rank for a particular term.
Another generalization in the patent is that shorter words are more competitive than longer words. If that were true, then it would be much harder to compete to rank well for a word like “xeric” than it would be to rank highly for “ice cream.” An additional one states that keyword phrases with one or two words are more competitive than search terms with more than two words. While that may often be true, it isn’t always and there are ways to gauge that competitiveness that go beyond such an assumption. I would never draw that conclusion in comparing terms without doing more actual research.
The processes described in the patent filing to determine the potential profitability of a keyword for drawing search traffic have some similarities with how someone doing SEO might perform keyword research to find words for a particular site. Though an SEO would try to identify keyword phrases that are both more likely to be used in a search by the audience members for that site, and which those audience members might expect to see on the pages of that site.
Demand Media may use either additional factors or other factors to determine which keyword terms to target when creating content to attract advertisers and people to view those advertisements, but the patent filing does provide a glimpse at some of the approaches that the digital media company may take in identifying terms to target.
I don’t trust alexa rankings too because you’ll only rank high on alexa rankings if you use alexa. I think that’s all there is to it so it’s not really reliable.
Hi Bill,
These days everybody use Keyword research tool to find out best possible keywords for their website. With Demand Media Internet presence they can easily make good revenues from advertising from this patent.
Thanks bill, its natural that short keyword is more competitive. And if we try to write our post for google by writing keywordrich content, most of the time the content be less attractive than as usual content.
Hi Andrew,
I don’t think that you need to actually use Alexa to have one of your sites rank well there, but there is some selection bias involved in that the people using the toolbar choose themselves. The Alexa toolbar doesn’t measure a random sampling of people visiting websites, but rather a sampling of people who have chosen to use the Alexa toolbar.
Regardless of how accurate or inaccurate the Alexa data is, the Alexa ranking of a site that a page is on probably isn’t a very good indication of how a search engine might perceive the relevance or importance of a page for a specific query term.
Hi Max,
I don’t know if everyone uses keyword research tools when creating content for their website, but some do.
If Demand Media is paying $2 for someone to create three or four paragraphs around a particular term or topic, it’s possible that the quality of the articles they are creating is likely to be fairly low in most cases. There’s room for other website publishers to compete with them, and succeed.
I think that google will go somehow against this trend. So many people today try to create endless seo content with the hope of making easy money. Not that I will stop reasearching. And about Alexa , Alexa is very good in comparing. If I compare my website with a few competitors websites I get a good idea. If somebody is visiting my site, so it should be visiting my competitor’s website as well.
Hi Alamin,
I don’t think it’s a good assumption to make that the shorter a keyword is, the more competitive it will be. There are better ways to gauge how competitive a term is than a broad sweeping assumption like that. How would you go about deciding whether “shoe” is more competitive than “shoes” or “scissor” is more competitive than “scissors”? I suspect that there’s a very good chance that the longer versions of each of those words is more competitive than the shorter versions, so just going by the length of words may be more harmful than helpful.
Doing keyword research before we write content for web pages or blog posts doesn’t automatically lead to writing stilted prose that provides a bad experience for visitors, but rather means that we are making intelligent choices about the words that we choose to use, where we try to include within our content words and phrases that our intended audiences are more likely to search for and expect to see on our pages.
In my gut, what Demand Media is doing just feels so wrong. Google can’t possibly like this kind of manipulation.
I have to admit that after I read an article about them (I forget now where, maybe Time) and learned how little writers are paid for their eHow articles I lost all faith in any content I find on eHow. And, unfortunately (I guess) those eHow articles come up all the time in Google searches for all sorts of obscure topics.
Thanks bill. Nice point but one word like more competitive than then phrase. And we can write our post as usual then we can replace words by our keyword.
This kind of information would be very valuable, especially considering keyword competition and all. As usual Google could charge any mount for these tools, as well they could on webmaster tools, analytics, etc. Great read Bill.
There are easy ways of checking competition in SEs for terms you select from Googs keyword research tool.. notably allintitle / allinurl etc..
I was recently blocked overnight by google for running allintitle on keywords to fast (checking through results pages).. was ok next day again tho 🙂 So I guess this is to stop all the automated tools online that use their engine, as it’s the first time it has happened to me, and I have been doing that for a couple of years now.
“Another generalization in the patent is that shorter words are more competitive than longer words.”
this is not always correct. canon is more popular (and so competitive) than digital camera. using google’s own keyword suggestion tool may easily find out such. not sure why they are assuming that…
Hi Bill,
Great post…I have to agree with Lee, the allintitle and allinurl are great ways to gauge the competition for any given keyword. I too have been blocked in the past by Google but persistance is the key. I also agree that research does not lead to ‘stilted prose’. Research is merely the start – if you write with the reader in mind first and the search engine second you will avoid this issue completely. Internet users are getting more sophisticated and to assume that the shorter word is more competitive than longer ones seems a bit strange!
I hope Google adds a ‘suggest’ tool into Gnol and encourages more revenue sharing.
i was sent a tool today that uses googles suggested terms that drop in the search box as keyword suggestions.. but it gets loads more than what they offer in the adwords tool. also covers yahoo and bing.. (not that those matter much haha).. but that is just one of many tools that piggyback Googles engines to find targetted keywords.. they must be getting sick lol
I’ve heard about Google sometimes shutting off the allintitle and related searches. Do people expect that they will shut them off completely some time? If so, what can we use instead?
Hi Modern Heinrich,
I do hope that if Google provides this kind of information, that it will open it up to everyone on the web, much in the way that they make other tools available to everyone, such as their keywords suggestions tools.
Alexa can give you some idea of how much traffic sites receive, but I would caution people to understand its limitations before relying upon it. The main limitation is that the people who choose to use it are self-selected, and we don’t have any idea how well they represent people who use the web.
Hi Andrea,
I’ve been paying a little more attention to Demand Media articles on places like eHow, and I have to say that there’s a real mixed bag in terms of the quality of results. Some of the artcles I see are pretty well done, but many introduce concepts and idea in a limited manner. I guess that shouldn’t be a surprise.
I’d prefer to see high quality content pages show up well in search results, then something that is just a broad overview. I do think that most viewers would as well.
Hi Alamin,
I find it’s really helpful to have an idea of what keywords you want to use before you write an article or blog post instead of trying to insert those after the fact.
Thank you, Keith
I think Google ultimately benefits more by having quality results to show people then it might by charging for access to information about how much search volume there is for different keywords, or how competitive those might be. Hopefully they will keep on feeling that way as well.
Hi Lee,
There are ways that you can use to check for competitiveness that involve collecting information from the search engines themselves, like the ones you mention. I don’t know if I was surprised or not by those not being included in this patent filing from Demand Media.
I’ve manually checked things like allintitle and allinurl, and had Google block me from doing so, or ask me to enter something found in a Captcha to see if I was an automated system. I was able to return later in the same day after that happened.
Hi Eric,
I’m not sure why they are making assumptions like that either, rather than doing some actual competitive analysis.
Hi SEO Nottingham,
Good points. Starting with the reader in mind is ideal and essential, and can make a page much richer, with a particular audience in mind and a reason why them might find a page or article about a particular topic useful or important to them.
Hi Shea,
A suggest tool in Knol might not be a bad idea, but I hope that they make those kinds of suggestions available for people interested in publishing on their own pages and sites as well.
Hi lee,
That sounds like an interesting tool. I’m not so sure that Google might be against tools like that then we think they might be.
Hi Andrea,
I haven’t seen that happen, but I don’t think it’s a bad idea to search for alternatives. I don’t expect them to turn those off anytime soon, but things on the Web do sometime change. If it does happen, it’s possible that someone might develop a tool that provides that kind of information.
This patent filing could threaten the SEO world in the same way attempts to patent “SEO” threatened it. All keyword research tools may have to pay a fee to Demand Media for doing what they were doing long before Demand Media existed.
I think it is always wise to target keywords where your ROI is likely to be high. Other often times it’s nice to rank for less competitive keywords as well. You might not get the value as higher keywords but your site will be visible which is a plus.
The USPTO never ceases to amaze me with how they allow people to patent processes that are super vague or processes which people have been using for a long time in the wild. They are clueless in a lot of cases.
There is nothing innovative about what Demand Media is doing. Almost everything they listed is available through Google’s external Adword tool and other tools already available on the web. People have been using similar combinations of data to do keyword research since the data was available. I do this daily in a very similar way… but include even more factors like organic competitiveness.
As others have mentioned, I use several operators to determine organic competitiveness like:
intitle:”my keyword phrase”
allintitle:my keyword phrase
inurl:”my keyword phrase”
allinurl:”my keyword phrase”
intitle:”my keyword phrase” inurl:”my keyword phrase”
allinanchor:my keyword phrase
inanchor:”my keyword phrase”
etc.
Dividing both the local and global search volume for [my keyword phrase] (exact match) by the number of results returned for each of the above operators gives me a quick way to determine competitiveness and opportunities where there is moderate to high search volume but low competition.
What is sad is that Demand Media will likely get such a patent because the people at the USPTO handing out patents typically have very little experience with the technologies being patented. So they can’t tell a true innovation from someone taking a common practice and patenting it. It’s a sad state of affairs.
Hi Michael,
It sounds like you’re referring to the attempts to trademark the term “SEO”.
I’m not sure that I really see anything new or non-obvious in this patent filing, and it describes the kind of analysis that many people performing SEO have been doing for many years. Creating information rich pages to attract people to click on advertisements is a business model that’s been on the Web for much longer than Demand Media, and the kind of analysis described in the patent is more common sense than innovation.
Hi James,
Good points. There are a few things that you want to consider when optimizing a page for a specific term, such as ROI or even potential ROI if you can identify an emerging trend or technology or product that might have value in the future.
It’s not a bad plan, either, as you wrote, to optimize pages for less competitive terms. One approach that can work well is to choose more competitive keywords for pages like your home page, which may accrue more pagerank and have more visits, and choose less competitive keyword phrases for pages deeper within the hierarchy of your site.
Hi Canonical SEO,
Thanks for sharing your thoughts about the patent office and how you determine competitiveness. I use some similar processes as well, and also spend a lot of time looking at search results pages, and looking at sites that show up in the search results for terms as well.
The patent office covers a very broad range of inventions and topics – I think part of the filtering process that they use is the actual cost of getting a patent. The examination process is supposed to filter out patents of the kind you describe, which are overly broad or cover common practices within an industry. They sometimes don’t, though. There really isn’t anything new in this patent filing.
It also misses out on looking at some other things, such as:
1. The impact of singular vs. plurals
2. What to look at when a term or phrase many have more than one meaning
3. Compound words and decompounded words
4. Search query refinement suggestions from the search engines
5. Blending of results through universal search
6. When a search engine might include synonyms
I think you should also be focusing on conversions when targeting keywords to specific pages. Page rank and site visits while important metrics are not as important in my mind as conversion. Trial and error will always help you get an accurate gauge as to what keywords are working and which are not.
I could not agree with you more, Randy! Ultimately, it’s the only thing that matters.
Hi Randy and Andrea,
I agree with you that tracking conversions is extremely important to a large number of web sites, but what is described in the Demand Media patent filing doesn’t go that far. It is a prediction model for identifying and creating very large quantities of very low cost content pages to host advertising upon, rather than a complete system that includes the feedback that conversion data could provide that could be used for tweaks and improvements to that content.
The process attempts to estimate profitability for specific terms by looking at search volume, bid prices for the terms in paid search, potential click rates, competition, potential long term value, possible likelihod of the resuse of that content, and the potential for attracting links to that content.
But the patent application doesn’t include conversion data or site visits or other information about the performance of pages created using the process in the patent. It’s possible that Demand Media may view that information on its sites, such as eHow, but you can’t collect conversion data until you’ve created and published the content, and this patent filing only goes to the point of keyword suggestions for the creation of content.
While conversion is very important, it’s not always the end all, be all when doing keyword research for paid or organic phrases to target.
For example, while I was Sr. SEO Manager at LendingTree, I watched our paid search team buy the phrase “mortgage calculator” month after month… 10s of thousands of clicks per month… and it’s one of the worst converting mortgage related phrases out there. Why would they waste so much money on that phrase you might ask. Well the answer is simple.
Buying a house and getting a mortgage is a long process. It’s not something someone decides to accomplish in a day. It has a long sales cycle. People searching for “mortgage calculator” are in the early stages of buying a home. They are likely trying to figure out how much they can afford, whether they should rent or buy, what their house payment might be like for a particular rate, term, and amount combination.
It’s very important in the online mortgage business to get in front of the customer early in the process, to be useful to them during the early phases of the buying cycle, so that when they are actually ready to get a loan that they think of your site first.
So even though the phrase is a VERY poor converter, it is a VERY desireable phrase to buy PPC ads for and to rank for organically. It is actually a very competitive terms in the mortgage vertical for the reasons above.
So it depends on the nature of the business for which you are researching keywords. Sometimes it’s just as important to go after keywords that don’t convert as it is to go after those that do. Of course, the “ultimate” goal of all marketing is to convert, but there are reasons to target non-converting keywords as well.
It also may depend on whether you’re researching for PPC or organic keywords. For example, I might decide to target “mortgage calculator” using SEO as a marketing strategy even though it’s a poor converting phrase so that I do not have to spend so much on that phrase for PPC and can spend my PPC budget elsewhere.
That would be amazingly useful to have google tell me what kind of articles would perform well for search results. I hope to see this coming to my webmaster dashboard.
Hi Canonical SEO,
I think those are some very important points regarding the value of keywords when it comes to conversions, sales funnels, and the different kinds of tasks undertaken by searchers, and the varying information needs that they might have. Thank you for sharing your experiences and what you’ve learned from them.
In some ways, part of the business model that Demand Media has undertaken may point towards the creation of low quality content. Their purpose isn’t to sell some product or service directly, but rather to generate clicks on ads that may accompany their pages. If the content that appears on one of their pages is high enough quality to have the pages rank in search results, but not high enough to satisfy visitors, then it becomes more likely that people will click upon ads on those pages.
If they were selling the goods or services themselves, they would probably focus upon higher quality, pay more for people to write articles, and spend time rewriting content to improve conversions.
Hi Matt,
Interesting idea. The patent filing from Google on how they might suggest topics and query terms doesn’t include mentions of making those suggestions through Google’s webmaster tools, but it might not be a bad idea for them to do that.
I would like to see some marketing firms analysis the trends on this keyword way. I remember Google saying a while back they would not release what people search for. Looks like they were still in development for all these tools. Of course this has pop up so many firms that now are “internet marketing gurusâ€. Five years from now what will be the hot thing? What about ranking for txt keywords the more people use phones to search online….Wait I just had a million dollar idea.
All great tools, especially Insight. On Google releasing information, is it true that Google released Adsense percentages a while back? I know thats a little off-topic, sorry. I read that on a blog and just wondered if it was true. They seem to be wanting to add to their “transparency” as far as their tools go.
Hi Rick,
Google has been supplying some information about what people have been searching for on the Web for a while now, with offerings like their keyword research tools, Insights for Search, and Google Trends.
As the Google Mobile Help page tells us, web results through Google Mobile are:
Hi Jared,
That information was released on the official Google Adsense blog. You can read about it directly here:
The Adsense Revenue Share
Transparency is what they state their reason is for providing this information:
It’s possible that they might believe providing that information might lead to more people participating in the Adsense programs.
Cool, thanks. I think it’s good that they made that information official. People had a pretty good estimate of what the payout was anyway, so I don’t think it made a difference in the first place. I do like though, how they continue to only release the information that we need, as it pertains to formulas and percentages. Now all we need are the algorithms that will enable us to be at #1 right away!
Very true about possibly wanting more people to participate in Adsense.
Hi Jared,
I’m not sure that Google’s going to be sharing too much about how they rank pages anytime soon.
Google does ask for transparency on the sites that advertise through adwords – those can affect the price of using adwords. Their efforts to be more transparent is only reasonable in regards to their expectations of their advertisers.
I’m more inclined to use adsense now that I know what the split is between users of Adsense and Google.
I really wonder how accurate the google keyword tool is anyways. I have noticed that it is only accurate on some keywords but not others. I remember one SEO consultant telling me that you cannot base it on the search volume in general, but on the volume to each other. If that makes sense lol.
Hi Dan,
The Google keyword tool can be helpful as a guideline, but using it carries some risks.
We really have no way to gauge the accuracy of the tool as it is.
One of those risks is that you are using it to predict potential traffic based upon past performance. You’re also being given an average based upon the past 12 months, with little indication of whether it’s a term or phrase that might be seasonal, or that had a temporary burst of popularity, or whether it might be trending upward or downward.
As a prediction tool, using the numbers for different keyword phrases as a comparison point isn’t the worst step that someone could take, but you may not want to fall into thinking that just because the tool says that a certain term averaged 1,000 searches a month that you might stand a chance of seeing some percentage of that traffic if you rank in one of the top spots for that term.