A couple of interesting patent applications surfaced at Google recently, involving the use of photography to identify whether or not businesses exist, or might be closed, or might be Web Spam.
The first of these looks at Street Views images, and is:
Systems and Methods of Correlating Business Information to Determine Spam, Closed Businesses, and Ranking Signals
Inventors: Andrea Frome, Howard Wellington Trickey, Melanie Clements, Ethan G. Russell, Paul Eastlund, Diego Ariel Gertzenstein, Douglas Richard Grundman, Baris Yuksel
Assigned to: Google, Inc.
US Patent Application 20150154607
Published June 4, 2015
Filed: February 24, 2011
Abstract
Systems and methods are provided for determining whether given business listings are spam or are closed businesses. In one example, a system receives a business listing from a user. The business listing contains geolocation and the name of a business. The system also receives one or more street-level images related to a geographic object that is at or near the same geolocation as the business listing. The system determines if the business listing is spam or is closed based on the received street-level images by determining if the geographic object is the business or by analyzing whether the business associated with the listing is shown in any of the street-level images.
This patent is aimed at detecting business listing spam and closed businesses in Google Maps.
Fake business listings can impact people using Google because business listings can appear in Google Maps results, in Google Search Results, and location-based services.
These listings may include input from business owners that can include the business’s name, address, contact numbers, web site and geolocation data (latitude/longitude), and other information. Those listings might include unwanted spam (fake listings for businesses that do not exist) as well as closed businesses, which may mislead searchers.
Google has been sending out cars with cameras on top of them, filming streets across the country. Reports have surfaced recently that Apple is Now sending out vehicles to film businesses in a similar manner.
The street Level images from Google are shown in Google Maps, and usually display photos of buildings, neighborhoods, and other features to enable people to take in a geographical location from the perspective of a person rather than from a satellite view or aerial view from above. These can include places such as “a museum, a gourmet shop, a restaurant or other point of interest” You could also see storefronts and the signs of businesses. Google shows off other sites at their Views pages.
This patent uses something it calls view codes, to “rank business listings relative to one another and to decide which businesses to present for prominent display on a map.”
How Google Determines if a Business Listing is Valid Under this Patent
1) It starts with Google receiving a business listing containing geolocation information and the name of a business
2) An image related to the geographic object initially associated with the business listing may also be received, from a street views vehicle
3) The geographic object may be verified to see if it is associated with the business, by comparing the geolocation information of the business listing with location information of the image
4) If the business listing is not associated with the location information of the image, The business listing may be considered to be spam or a closed business.
The image may contain timing information, which indicates the time when the image was captured.
The originating time of a business might be identified that says when business was set up at an address indicated by the geolocation information of the business listing, and that may be compared to the time the image was captured.
If other business listings were submitted by the same account, and they have been determined to be spam or closed businesses, that may indicate a need for further scrutiny of this particular account, and possibly the shutting down or locking for the account that submitted this business listing.
A view code may be received that shows a different business in the same geographic location, along with timing information that indicates when the image was taken. That might be used to tell if the other business that was submitted as being at that location might be a spam business or a closed business.
Other Photographs and Spam Businesses
Another patent application published today at the United States Patent at Trademark Office seems to be very related but doesn’t rely upon Street Views images.
That patent shares one inventor with the patent listed above, Baris Yuksel.
Verifying a Business Listing Based on Photographic Business Listing Obtained through Image Recognition
Inventors: Baris Yuksel
US Patent Application 20150161619
Published June 11, 2015
Filed: January 5, 2012
Abstract
An apparatus for verifying the authenticity of a potentially false business listing is provided. The apparatus may include a memory that stores a plurality of business types and a plurality of business listings. The business types may identify one or more spam business types. The apparatus may further include a processor in communication with the memory that identifies a selected one of the business listings as a potentially false business listing when the business type of the selected business listing is identified as one of the spam business listing types.
The apparatus may then communicate a request for a photograph of a business corresponding to the identified business listing and extract photographic business listing information from the requested photograph. When the extracted photographic business listing information does not match the identified business listing, the apparatus may remove the business listing from the plurality of business listings.
This patent identifies a problem that it has been created to address, involving fake locations:
Because an online search provider can present many different businesses at once to a user, there may be a high degree of competition among businesses, especially where there is a high concentration of businesses of the same type in a geographic region. As businesses have learned that a user may typically seek out businesses that are geographically closer to him or her, a business may exploit an online search provider to take advantage of this fact. For example, a business may provide a false address to the online search provider such that when the business listing for the business appears in the search result webpage, the business listing may include the false address. The false address may entice the user into contacting the business and thereby may mislead the user away from choosing a business that is geographically closer to him or her. An online search provider has an interest in preventing these types of misleading business listings.
This patent tells us that it may attempt to request photographs of a business, corresponding to a potentially false business listing, and if it receives one and that doesn’t match the geographic location where the business is located, it may identify the business as being a false business listing.
That request for a photograph would be sent to the person listed as the owner of a business listed in the submission of a business.
A photograph returned in response to such a request would be submitted to an image recognition system to determine if the photographic business listing information matches images at the geographic image location submitted by the business owner. This patent doesn’t mention Street Views, and yet I could see how it might be used with Google’s Street Views system to determine if a photo submitted by a business owner might match any images taken for the Street Views project at Google.
This patent tells us that it might look at any business signs or postal address information within received photographs for businesses to see if the business name and the postal address shown to match up with the business name and geographical information submitted about a business. That kind of information would be recognized through an image recognition process.
This patent has a section in the description part where it discusses a spam business type database, where it might store information about business types that are known to be spam business types:
As discussed above, a spam business type is a business type that is usually associated with a business listing that a user creates to drive more consumers to his or her business, where the business listing contains purposefully false or inaccurate information. A typical spam business listing may include a false address such that users in proximity to the false address are drawn to the spam business listing and away from business listings that have correct addresses but may be slightly further away. Other business information that a spam business listing may include maybe a false business phone number, a false or misleading business name (e.g. a business named “Joe’s” may be listed in the business listing as “Joe’s Compare Best Jewelry Prices Lowest”), or any other type of false or misleading business.
It appears that under this patent, Google may only be sending requests for photographs of businesses to check up upon to business types listed in the Business Spam Type Database.
Sort of funny, sort of sad in my point of view.
I periodically speak with a person who was an active reviewer in mapmaker…a higher up. He used to kill spammy businesses quickly using the following methods:
1. Look at street view and see if there is a real business at that address. A lot of times he would find parking lots.
2. Pull the state or jurisdiction web documents on licensing of businesses. Was there a business there???
He suggested these methods to Google several years ago. They didn’t act on them.
I tried it. It works. BTW: His name isn’t on the patent application. 😉
Years ago I recall trying to help a real business fight a spammer. We used street maps on the location that showed the address of this so called business showing in Google local.
It was a field. That was years ago.
Glad to see google is trying this method now. 😉
Thanks for sharing that information, David.
I guess that shows the value in talking to people who handle situations like that every day.
It would be nice if Google were able to use methods like these to clean up Mapspam.
Oh man – I hope all these scammy businesses with PO boxes (not all of them are bad but mainly the locksmiths / taxi services that get 5 boxes and try to claim a whole 5 pack) gets DROPPED LIKE A BAD HABBIT. It is not fair to other businesses that try hard and do work.
It just so happens that most of these businesses are also the ones that do the shoddiest work.
I know I won’t be one of the people giving the Google car the finger, I will be saluting it 🙂
Hi Patrick, funny but the second patent providing examples of business types involved in spamming, use locksmiths as an example business type, so that is one industry that is directly targeted by this patent.These patents are asking for a little more certainty and honesty in the locations of businesses.
Thanks for sharing the information.
If this come into practical there would be less spam in internet. So many people are falling in the traps rights now. This should stop it.
I see one potential problem with this, when it comes to local businesses. Many ONLINE businesses who do not have the real street address, are forced to “invent” address when they enter their business data in local directories. Simply directories will not accept new business entry if they not fill the field with the street address. And if Google in the future will go so far as to take those addresses from the local directories and compare them with Street View, that can be a problem. This can be a big problem for those businesses that do not have bad intentions, but are trying to promote their business on the local level.
I’m not talking here about business listing in Google Maps or Google business pages, but only on local directories.
What about all those people who run a business (often web based) from home?
Thanks for sharing that information..But not clear explanation…………
Google is best to use this method to clean up spam.
if this applies to all countries, or only in certain countries?
Thank you for the information Bill
What about businesses ‘that dont serve customers at the physical address’? Verifying a business address that has no ‘storefront’ would be impossible no?
Interesting concept, seems to me that it would be really hard for Google to reliably base any decisions on this data.
Hi James,
If the photograph indicated the business name in a sign, and the postal address well, what would be difficult to rely upon that? And the street view images capture that kind of information, and have records of their locations when doing so. Again, that would be fairly reliable.
Hi Yuffy,
Google’s rules about submitting your business to Google Maps ask if you serve customers at a physical address, and Google does have service area business listings:
https://support.google.com/business/answer/3038163?hl=en
Google does try to verify the locations of those businesses, too.
Hi Reo.
The patent provides some approaches that Google may use, but doesn’t tell us if Google will use those, or where they might. It does sound like it will help clean up spam.
Hi Sekhar,
I’m sorry if the information is not clear enough for you. I did provide links to the patents that you can click through, and use to try to make sense of them directly.
Hi Steve,
Google does have procedures for submitting businesses from home, in that they won’t show your business location in your listing. Some more information about area of service listings here:
https://support.google.com/business/answer/3038163?hl=en
Hi Milan,
I don’t think Google wants to list businesses that have invented addresses, which is why they seem to have come out with these patents.
Hi Nijinsha,
I think this would help cut down on spam in Google Maps. I’ve heard from people stating tht methods like the ones described in the patents have been used successfully to do so.
Google are definitely looking to clamp down on businesses that have false addresses. I’ve had a few clients receive calls from the Google My Business department asking them to verify their business address. In one case where they had moved location this was immediately updated on the 7 pack listing.
Hey Bill,
Surely not with the same accuracy as a business storefront then?
Not surprised. Re: the second patent, I had an instance last summer/fall where there were multiple businesses in the same building, and there were issues with the Google My Business account. Google Support requested photographs of the business I was working with to resolve the issue. Re: the first patent, I guess this means businesses need to ensure they have something on the outside of the storefront that clearly indicates who they are. Curious how Google would use this in relation to office buildings with many business tenants…
Hi Yuffy. Google may do other things to attempt to verify a business location that doesn’t involve photographs, like look up the zoning laws at a location to see if a business can legally operate there. I am just writing about Google’s patents, and not Google’s actual practices; I don’t work at Google and have no actual insights into those.
Thanks Paul,
Appreciate your sharing your experiences with verification like this.
Hey Bill,
Haha, I know you don’t!
Just wondering if you had any more insights…
Thanks for replying.
Hi Kerry,
I’m guessing that Google may have asked for shots that showed signs and postal information in them, which would make sense. The Streetviews patent likely applies to many businesses, so the fact that it might not apply well to businesses inside of large office buildings really doesn’t matter; as long as the information can be useful for many businesses that register is really the important thing here.
Makes sense that they would incorporate the data sets – thanks Bill 🙂
You’re welcome, Andy.
It does make sense for them to try to handle Mapspam as intelligently as possible.
Interesting article, thanks for posting this.
Seems industrial, shopping centers and office buildings might be hard to have an actual picture from a google car of every business. So there will be gaps in their database.
Maybe just have a few photoshop templates to mockup with addresses and business names and then just add the address and geo data on the pic data when they request you send them a pic of your storefront or office suite.
Also when you call them just make sure the phone number is forwarding to someone that independently (lol) verifies with G over the phone that the business is really located there.
Spy vs. Spy…LOL
In the legal industry many “firms” and “offices” have remote locations they use just to get more listing on the 7 pack of search results. There are cases when lawyers will get a Mail Box just so they can have an address to receive a PIN card form Google My Business.
How will this affect the legal industry and lawyers that are doing this? Will all law firms and law offices need to prove a location rather than just having a mailing address and phone number?
Thank you!
Hi Andy,
I suspect that Google has considered the potential impact upon professionals such as lawyers who might be in remote offices, however, it’s good to have an actual office that clients can come visit, and Google is trying to list businesses that people can come visit. With those remote lawyers, it’s possible for them to set their listings up as area of service businesses, they could do that. Google is trying to avoid having 7 packs of search results spammed, too.
Hi James,
The two patents seem to work together, so that businesses that might not appear prominently in StreetViews can send in photographs of their businesses that do show off their Signs and postal information. Google is trying to provide a service that doesn’t get spammed by people trying to create extra listings that don’t actually exist. I appreciate their efforts to do that.
I thought sending out postcards was supposed to verify listings. If you don’t have a business at a certain address it’s hard to receive a postcard there, so is this really necessary?. Also sending out cars to film businesses will have trouble with businesses in highrises.
Hi Cathie,
Sending out postcards can help to verify listings can be useful, except when the address used is a mail drop, in which case, it’s operated to receive mail like that at a certain address and simulate that a certain company exists at that address. Google is going to send out the cars to film businesses anyway, to provide street views to searchers who want to see them, so as long as Google is filming those businesses, they might as well look at those pictures.
This process sounds too manual and time taking. What if an image of a business does not already exists, will Google go to the lengths of identifying and taking picture of each business and match them, considering the facts that there are too many of them.
Hi Bill Slawski,
Google Maps Using Photos to Identify Spam is relay a good information for a businessman and all of people. Thanks brother for sharing this with us.
It is quite interesting to know what Google is doing. This can be the best method for doing the same.. Nice article..
Hi Durab,
It was interesting seeing Google taking advantage of photos like this, and their ability to read signs and postal information in Photos seems to make this a realistic approach.
It is quite interesting to know what Google is doing. This can be the best method for doing the same.Nice article.
I thought sending out postcards was supposed to verify listings. If you don’t have a business at a certain address it’s hard to receive a postcard there, so is this really necessary?. Also sending out cars to film businesses will have trouble with businesses in highrises.
Hi Cuci Sofa,
People do pay for mail drop services at places that receive mail for them, in locations where they do not run a business, so there’s a limit to how much power such mailings have as a verification method. Google uses Street view informataion as content that they share with searchers, so even if the Streetview images are just helpful for some businesses, and not all businesses, it’s still helpful for those.
Hey Bill,
I was out of town that is why I never had the chance to read your 2 latest posts until today.
Setting the excuse aside, it seems to me that this patent can go a long way. It will surely bring a ton of help if it is done right. However, the method of verifying what is a spam or close business seems time and resource consuming. Imagine sending a view vehicle to verify if the given business address is legit needs a great amount of effort as well as budget. Also, there are places where these vehicles can’t go due to the terrain and other external factors. With this being said, the greatness of this proposal is held off by such certain factors.
Nevertheless, even with the presence of different hindrances, it doesn’t change the fact that this patent brings a lot of promise in countering pesky spam and fake business listings. Kudos to the patent creator and also to you Bill for sharing this interesting post. Cheers!
Thanks for the useful information. i think Google maps are doing a fantastic job
I’m amaze and glad that Google is doing this effort to find spammers. It’s great that they utilize Google map for this.
Thanks for sharing the information.
If this come into practical there would be less spam in internet. I think this would help cut down on spam in Google Maps. I’ve heard from people stating tht methods like the ones described in the patents have been used successfully to do so.
So we are finally ‘living’ in the future, haha! Great stuff! Thanks for sharing the information.
Do you mind if share this article on my blog? I will give the credit. Cheers from Ireland