Microsoft was granted a new patent today, Search ranger system and double-funnel model for search spam analyses and browser protection (US Patent 7,873,635), which provides a detailed look at how Bing might attempt to identify search spammers who redirect traffic from search results pages to pages filled with advertising or other content intended to earn the spammers some money.
The patent uses Google’s Adsense as an example of the kind of advertising that these spammers might use in one of these cloaking schemes.
Ironically, Google’s Matt Cutts also uncovered an interesting Bing affiliate scheme today, from a company that Ad Age calls FaceBook’s third largest advertiser in the third quarter of last year.
When you visit the advertiser’s site, it immediately prompts you to install a browser plugin/toolbar that will, according to a terms and conditions page, change your default homepage to another page and search provider, and the advertiser will get a percentage of the revenue from Bing ads displayed.
The following snippet from the patent is worth quoting “as is” from the patent:
Traffic-Affiliate Spammers
Some merchant websites provide affiliate programs that pay for traffic drawn to their sites. Many spammers abuse such programs by creating and promoting doorway pages that redirect to these merchant sites. There is a major difference between traffic-affiliate spam and syndication-based spam: while the latter displays a list of ads and requires an additional ad-click to initiate the redirection chain leading to the advertiser’s website, the former directly brings the browser to the merchant site.
As a result, while the final destination for clicking on a syndication-based doorway is often a spammer-operated ads-serving domain, it is the intermediate redirection domains in the traffic-affiliate scenario that are responsible for the spam.
While I applaud Microsoft for coming up with ways to help remove web spam from search results, I’m concerned about the affiliate program that Matt uncovered which changes a searcher’s homepage and search provider without the searcher’s knowledge or approval.
While this affiliate isn’t using cloaking or redirection to bring traffic to click on Bing ads, they are doing something that might be considered even worse.
How responsible is Microsoft for the actions of their affiliate in this instance?
The patent itself is very detailed, and the methods described within it were partly the result of an intensive six month study into search spam that uses cloaking and redirection methods.
One aspect behind the patent includes comparing what a potential web spam page might show searchers who arrive at the page through a search engine results page, and what that page might show someone who arrives at the page directly by typing the URL of the page into the address bar of their browser.
Added: 1/18/2011 1:28pm, Many of the ideas presented in this patent are similar to those from the Microsoft whitepaper Spam Double-Funnel: Connecting Web Spammers with Advertisers, co-authored by Yi-Min Wang, Ming Ma, Yuan Niu, and Hao Chen. Yi-Min Wang and Ming Ma are the inventors listed on the Microsoft patent.
Added: 1/18/2011 3:22 it appears that Microsoft has terminated their relationship with the publisher that was urging people to install the toolbar that would change their default homepage and search provider. See: Bing Terminates Relationship With Publisher Doing Tricky Home Page Switch.
If it “prompts you to install a browser plugin/toolbar that will […] change your default homepage to another page and search provider,” then why a couple of paragraphs later do you say it “changes a searcher’s homepage and search provider without the searcher’s knowledge or approval.”
These appear to be very different things. Does it prompt the user or does it do it without the user’s knowledge or approval? Care to explain?
That’s crazy that a company that large can be involved in a scheme most would consider “black hat”. Although companies exploit sheep all the time, it’s sad that ones that clearly don’t need the funds are the ones going after it. I never install those tool bars…
Hi Schram,
I don’t think my post is being inconsistent.
When you arrive at that page, it prompts you to install the plugin/toolbar, but it doesn’t tell you that doing so will result in a change of your default homepage or search provider. There is a “terms of service” link which does say that, but if you don’t go to that page and read it carefully, you won’t be aware that your homepage and browser default search provide has changed, or that the toolbar provider will now get a percentage of the revenues of any bing ad that you click upon.
Hi Chrissy,
I’m not sure which “company” you are referring to with your comment. It’s Microsoft’s affiliate rather than Microsoft that’s involved in the toolbar downloads that change default browser settings. The question is though, what responsibility does Microsoft have in policing the actions of their affiliates?
Correct me if I am wrong, but isn’t Google hiring tons of manual reviewers to do just that, type in URL’s manually to see if something different shows up other than what the script show the Google or Bing bot? I am not if Microsoft is doing the same, but it seems like that would be something they would want to do. In the end, the best algorithms for detecting SPAM directed towards human eyes is the algorithm directly tied to those eye…i.e. through the optic nerve. Black Hat techniques seem to be matching the algo in my opinion, hence the reason the algo is half wet-wired now.
Sorry I didn’t mean to be vague, I guess I was simply holding Microsoft responsible for the actions of their affiliates. Was that your point too?
Lets see how cool this patent is and does it really help to remove the web spam??
Still very much yesterday’s news and well behind the curve in terms of innovation, giving something for free to the public for downstream benefits, ala. Google. They are technology’s equivalent of rollerskating.
At least Microsoft terminated their relationship with the publisher already. I wonder if it’s because they read your post? 😀
I’m pretty certain Microsoft and Google will start turning up the heat in the debate over who handles spam the best/worst. Microsoft has a tradition of serving rather broken search results with their many instances of search engines. I do believe the Googlers are feeling some heat with the Bing/Yahoo combination and pointing out problems with spam is a good way to keep users away from them. It’s only reasonable Microsoft says the same thing about Google as we do get some spam there and we are getting tired of it, we are not already tired of Bing-spam as we still haven’t tried the SE.
Bill – I think the only reason Microsoft ended their relationship with the publisher is because they got caught. I’m sure Microsoft knew about the Bing affiliate scheme way before Matt Cutts brought it to their attention. It’s hard to believe that Microsoft had no clue about FaceBook’s third largest advertiser in the third quarter of last year.
@Shailender
I highly doubt that.
@Magnus Brath
Google is miles ahead when compared to Bing and Yahoo in every way imaginable. Sometimes I feel that Microsoft is fighting for a lost cause.
Hi Chrissy,
I just wasn’t sure if you were talking about Microsoft, or the affiliate/publisher.
The point of my post was that there was a certain amount of irony that Microsoft would be granted a patent on identifying affiliate spam by looking at advertisers on the same day that a Microsoft/Bing affiliate would be outed as using a very questionable tactic to earn Bing advertising dollars. It may not be an easy to track how your affiliates are attracting customers, but there probably should be some level of monitoring.
Hi Mark,
I believe that the focus of Google’s human evaluators is primarily upon assessing the quality of search results more than identifying web spam. They may do that as a part of the evaluation process, but I’m not sure that it’s the primary focus.
Hi Andrew,
I know that a number of other people wrote about the toolbar changing people’s homepage and search provider to earn Bing advertising money. I’m probably the only one amongst those who wrote about the Microsoft web spam patent. It’s possible that someone at Microsoft read my post – I know I get some visitors here from the major search engines. But I don’t think that I can claim responsibility for Microsoft terminating their relationship with that advertiser.
Hi Shailender,
If you had a chance to look at the whitepaper that I linked to at the end of the post, the process they describe within it isn’t a bad approach. But it’s hard to tell how much of an impact it has had, or might have in the future.
Hi Matthew,
Microsoft has a tremendous amount of resources and some very smart and talented people working on search. Given that, they are behind when it comes to competing with Google, and they are having problems catching up. I’d love to see them offer more competion – I think we’d all benefit.
Hi Magnus,
Google has been the target of a number of claims lately that there’s too much web spam in their search results. On the other hand, I don’t always find some of the great results from Bing that I see from Google. Hopefully both can get a better handle on spam.
Hi Jimmy,
I have seen some things from Bing that I like. In an ideal world, hopefully both Google and Bing develop along slightly different paths as they compete, and each develop their own strengths.
Hi John,
I wonder if Microsoft will be making some changes to the way that they monitor their advertisers after this.
Maybe I’m being cynical, but perhaps the point is not to cut down on questionable advertisers trying to get themselves a profit, but to cut down on questionable advertisers trying to get a profit that Microsoft don’t get a cut of.
Hi Thomas,
I’m not so sure that the problem is about advertisers as it us about publishers who use questionable tactics to deliver people to advertisements, such as cloaking and redirects and invasive toolbars and other software,
As I work a lot with non english markets I see a lot more spam (irony?). It seems as if you can get away with a lot more in the smaller markets. This in it’s turn would imply that a lot more than we think is done manually – Sweden for example that doesn’t have a quality guy for itself, shared with Norway, Finland and Denmark has major problems with spam in Googles results. A good example of this is the main keyword for SEO in Swedish – Sökmotoroptimering, 4 out of 10 in the result got there by old fashion, drug and sex-style comment spamming.
Hi Magnus,
Appreciate that you’re sharing your experience with non-English markets.
I suspect that Google tends to use English markets as a place to experiment at, and implement new approaches at first, as both a proof of concept for those methods, and as a baseline before they attempt to adopt those approaches in other places which often have unique needs to be addressed. It would make a lot of sense for Google to have more search quality team members in more markets, to avoid problems like the situation that you describe. Maybe that’s partially why Google announced a major hiring phase recently.
I’m with Magnus on this one as I’m Swedish myself.
The search result for the keyword SEO consists of about 5 spammers which is sad to see. And no matter what you tell Google – they don’t do anything. I hate it!
Google will lose a lot of their customers sooner or later.
Hi Kim,
Google really does need to get more people out there. I know that after they announced that they would be hiring a good number of positions that they received about 75.000 applications over the course of a week. Hopefully some of those were from people in Sweden.