XML Sitemaps Help Index Fresh Content Quicker

Using an XML Sitemap to Index Fresh Content Quicker

It’s nice to find support documents from Google that point out the benefits of using certain approaches, and Google published one on the benefits of using XML sitemaps.

Search engines use programs to crawl the web and identify new pages and newly updated pages to include in their index. These are often referred to as robots, or crawlers, or spiders. But there are other ways that the search engine gets information about pages that it might include in search results.

A whitepaper from Google, Sitemaps: Above and Beyond the Crawl of Duty (pdf), examines the effectiveness of XML sitemaps, which Google announced as an experiment called Google Sitemaps in 2005. The experiment seems to have been a success.

XML sitemaps are a way for web site owners to help the search engine index pages on their web sites, through the use of XML Sitemaps. Yahoo and Microsoft joined Google in adding support for XML sitemaps not long after, and a set of pages explaining the sitemaps protocol was launched.

Continue reading “XML Sitemaps Help Index Fresh Content Quicker”

Is Game Playing the Future Ranking System for Search Results?

Many tasks are trivial for humans but continue to challenge even the most sophisticated computer programs. Traditional computational approaches to solving such problems focus on improving artificial intelligence algorithms. Here, we advocate a different approach: the constructive channeling of human brainpower through computer games. Toward this goal, we present general design principles for the development and evaluation of a class of games we call “games with a purpose,” or GWAPs, in which people, as a side effect of playing, perform tasks computers are unable to perform.

Designing Games With A Purpose

A paper from Yahoo researchers, Thumbs-Up: A Game for Playing to Rank Search Results, describes a game that they developed and tested internally at Yahoo to allow participants to compete against each other in ranking how relevant pages are for specific search queries.

Continue reading “Is Game Playing the Future Ranking System for Search Results?”

Do Search Engines Look at Keywords in URLs?

Does Having keywords in URLs Make a Difference to Rankings?

Is there any value in using keywords in URLs of web pages? Would a search engine look at keywords that you might include in the addresses of your pages, and associate those keywords with the content of your pages in the search engine’s index?

If so, how would a search engine go about looking at the web addresses indicated in the URLs to your pages, and break them down into meaningful parts to identify keywords?

Breaking URLs down into parts may also play a role in how the pages of a web site might be crawled by a search engine.

A newly published Yahoo patent application gives us some ideas on how it might extract keywords in URLs of pages, and rank them, as well as using information uncovered in the process to determine which pages to crawl first from a web site.

Continue reading “Do Search Engines Look at Keywords in URLs?”

Google Patent Granted on Web Link Spam

When a search engine indexes pages and other documents on the web, hoping to provide meaningful and relevant results to searchers, it doesn’t just rely upon the content found on web pages, but also considers the quality and quantity of of links pointing to those pages.

examples of link farms and clique attacks

A search engine like Google might determine that a page is relevant to a specific query based upon the content found on that page, and the anchor text found in links pointing to the page.

It might also look at what it considers “relationships” between pages by looking at how pages are linked to each other. PageRank is one method of viewing those links that Google states that it uses, and assigning a measure of importance to pages that are linked to from other pages. This measure, or rank might be simplified as a probability that someone might arrive at a certain page if they are arbitrarily and randomly clicking on links on pages that they’ve surfed.

Continue reading “Google Patent Granted on Web Link Spam”

How Searchers’ Queries Might Influence Customized Google Search Results

Why does Google customize some search results based upon a previous query that you’ve performed? Is there a special relationship between those query terms, and if so, how did Google define that relationship?

Imagine searching for “luxury car” at Google, and then performing another search for “infiniti.” On the second search, you find a page in the search results that looks like it will provide you with information that you are looking for, and you select a page.

Now imagine that a number of other people perform the same series of searches and select the same page.

It’s possible that Google might start considering the search for “luxury car” and the search for “infiniti” to be related queries. It’s also possible that the page selected in the second search for “infiniti” might start ranking more highly for the query “luxury car.”

Continue reading “How Searchers’ Queries Might Influence Customized Google Search Results”