Yahoo Robots Combat Spam and Phishing

How much of a responsibility do search engines have to police the internet, and protect their users from email and instant message and web spam, phishing fraud, and misuse of chat rooms? How can search engines be socially responsible and work towards keeping consumers from harm?

How effective can they be in pro-actively identifying and fighting phishing attempts on the Web as they index the Web?

A newly published patent application from Yahoo tells us that people using the Web have an expectation that their services providers will help protect them from such abuses, and Yahoo describes how they might use robot programs to help identify emails, IMs, chat rooms, and Web sites where inappropriate activities take place.

The document goes into detail on how a Yahoo robot might interact with others in a chat room, and through email and instant messaging programs to identify spam and fraud, through phishing attacks aimed at obtaining passwords, credit card details, social security numbers and other confidential information.

These robots could passively or actively seek out abusive activity on the Web, and collect information about the abuse, such as “a sender’s network address, a sender’s account name/address, Universal Resource Locators (URLs) associated with the abusive communication or web page, or the like.”

Under this approach, a robot could log in to web pages, sign up for news letters, sign in to chat rooms and initiate conversations, receive emails and respond to them, and proactively take other steps to identify spam and phishing attacks.

Depending upon where and how the abuse takes place, Yahoo could take different responses to spam and phishing, such as blocking or filtering messages and web pages, warning the sender, or disabling accounts.

Dynamic Combatting of Spam and Phishing Attacks
Invented by Richard Sinn and Miles Libbey
US Patent Application 20080177841
Published July 24, 2008
Filed January 19, 2007


A self training set of robots are configured to proactively search for selective communication abuses over a network. Robots may enter a chat room to proactively send messages. The robots then analyze patterns and/or content of a received message for potential abuse.

Robots may also passively reside on/off line without publishing their network address. If a message is received, the message may be interpreted to be SPAM/SPIM.

Robots may also perform a variety of other actions, such as access websites, and analyze received messages to determine if the messages indicate abuse. If abuse is detected, information may also be obtained to enable blocking or filtering of future messages from the sender, or access to/from an abusive website.

The information also may be used to retrain robots, so that the robots may learn from and share their collective knowledge of abusive actions.

Fraud run loose on the internet makes the Web much less safe for all of us. It’s great to see Yahoo attempting to find ways to identify spam, phishing, and misuse of chat environments.

Resources on Phishing

In researching this post, I found a number of articles, blog posts, organizations, and academic papers about phishing, and about some of the technology being developed to address phishing. It’s by no means a complete list – there are many more resources on the web about phishing, but hopefully it’s a good introduction to the topic.

The Ocean is Full of Phish – An introduction to phishing, and phishing attacks, with some thoughts on technical solutions to stopping phishing.

The Anti-Phishing Working Group – A world wide organization “focused on eliminating the fraud and identity theft that result from phishing, pharming and email spoofing of all types.” Members include some of the leading technology service providers in the world, including Microsoft, Yahoo, MySpace, and eBay.

The Messaging Anti-Abuse Working Group (MAAWG) – Aimed at preventing online messaging abuse, another organization sponsored by very large technology groups. The press releases section provides many details of activities undertaken by members of this group.

Federal Trade Commission – Spam Summit: The Next Generation of Threats and Solutions – The staff from the Federal Trade Commission’s Division of Marketing Practices put out this report in November 2007, providing a look at spam and phishing from the perspective of the FTC.

Spam, Phishing, and the Looming Challenge of Big Botnets (pdf) – What dangers might be around the corner for consumers as spammers start utilizing huge networks of virus-compromised computers, called botnets? What does this mean to technologies intended to fight sphishing attacks, such as SSL certificates and DomainKeys/DKIM.

Combating the Botnet Scourge (pdf) – Information about botnets, and some basic and advanced defense approaches to address them.

A Forensic Framework for Tracing Phishers (pdf) – Some ideas on using phishing methods against those engaged in phishing, to help investigators and service providers to “pro-actively fight phishing online fraud by identifying and tracing the involved actors.”

CANTINA: A Content-Based Approach to Detecting Phishing Web Sites (pdf) – an approach to examining the content of web sites to identify ones used in phishing fraud activities.

One small step for email, one giant leap for Internet safety – Yahoo’s May, 2007, announcement of the use of Domain Keys Identified Mail to authenticate the original domain of emails from organizations that are using the technology.

Fighting phishing with eBay and PayPal – Google’s July, 2008, announcement of the use of Domain Keys Identified Mail in Gmail for messages from eBay and PayPal.

Phinding Phish: Evaluating Anti-Phishing Tools (pdf) – A paper from later 2006, which compares tools that help web surfers identify phishing activity, focusing upon the effectiveness of 10 popular antiphishing toolbars.

You’ve Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings (pdf) – A paper written by Carnegie Mellon University researchers for CHI 2008, held in April 2008, which explores the effectiveness of the warnings that anti-phishing programs provide, and ways to improve upon those warnings.


Author: Bill Slawski

Share This Post On