While we often talk about relevance when it comes to writing webpages, but another aspect of the content for a page that needs to be considered is how engaging and persuasive what we write might be. What makes one result appear to be more relevant, and more trustworthy than another?
A paper from researchers at A9 and Yahoo, Summary Attributes, and Perceived Search Quality shows some experimentation on how people might perceive search results based upon what a search engine displays from Web pages on its search results pages.
A list of search results will usually contain the title, URL, and an abstract or snippet from pages. If the words within the meta description contain terms from the search query, part or all of the meta description may be shown to searchers. If the page is returned as relevant for a term, and the words used aren’t in the meta description, other words from the page may be shown instead.
What is it that searchers are looking for that might cause them to choose one search result over another to click through and investigate further?
In a recent post on snippets from reviews that Google might display during product searches, I wrote about some of the considerations that Google might be concerned with on choosing which part of a review to display to a searcher. Things like sentence length and grammar play a role in those determinations.
The A9/Yahoo paper looked at some other areas:
Text Choppiness. Conditions tested: (1) all snippets complete sentences; (2) incomplete sentences, but with well-chosen breakpoints; (3) incomplete sentences, but with deliberately bad breakpoints.
Snippet Truncation. Conditions tested: (1) complete sentences; (2) beginning of sentence removed; (3) end of sentence removed. In the latter two cases, good breakpoints were chosen.
Query Term Presence. Conditions tested: (1) both query terms present in the first snippet, both in the second snippet, and one in the third; (2) both query terms in the first snippet, one in the second, and none in the third.
Query Term Density (“Spamminess”). Conditions tested: (1) for a two-snippet abstract, both query terms shown once in the first snippet only; (2) both terms are shown in the first snippet and one in the second; (3) both terms are shown repeatedly, for a total of eight occurrences.
Abstract Length. Conditions tested: (1) Approximately four lines of text (“long”), assuming typical font size and window dimensions; (2) three lines of text (“medium”); (3) two lines of text (“short”).
Genre. Conditions tested: (1) abstracts contained genre cues (such as “official site”); (2) abstracts did not contain genre cues.
Some interesting results, like the positive impact of genre cues in abstracts. This is only a short “poster” presented at the WWW 2007 conference last week, but it’s good to see that the search engines are exploring topics like these.
Good Evening Bill,
I found it very interesting to see the mention of Query Term Density in this. Do you have an opinion as to why one continues to see new web pages that are keyword ‘stuffed’, as in: if you like dogs, this dog is the best dog a dog-lover looking for dogs could ever hope for?
Is this just a hold-over from the dark ages of the Internet when writing like this could actually yield results for the author?
Your extract, above, seems to indicate that search engines are well aware that repeated use of a keyword could indicate a spammy document. Do you believe that this will become more apparent to web page creators in the future and could abolish silly stuffed documents from the Internet, eventually?
I’d be interested to hear your thoughts.
Kind Regards,
Miriam
Hi Miriam,
There’s a fine line between redundancy and spam, as measured by an algorithm. For example, one of my favorite pieces of writing is a speech from Dr. Martin Luther King – I Have A Dream. As a speech, the use of repetition of certain phrases within the document give it strength.
Also, it isn’t uncommon, during a discussion of a topic, to repeat certain phrases over. So, any algorithm that might consider repetition to be an indication of intentional web spam needs to exercise care.
Usually, many more signals than just how many times a word or phrase appears in a document will be reviewed before a determination is made that something is spam.
The paper looks at abstracts or snippets from the view of how people might react to the snippet, and whether or not something like the repeating of a term in that snippet might make search results appear to be less relevant than they are. So, while we get a glimpse at one assumption (that repetition of a term might be a marker for spam), the paper really isn’t focusing upon that.
Why do people stuff pages with keywords? Unfortunately, I think that many do because they think that they have to. And they don’t.
Bill I’m glad to see someone looking at this. It’s interesting since search snippets certainly favor query term presence and density over choppiness and truncation. I know I’d sooner click on something that was more coherent even if it didn’t include my query terms, but bolding query terms has been shown to result in higher click throughs.
I wonder how well an algorithm can really measure grammar. Most spell and grammar check programs are pretty bad and I sometimes wonder how important it is. You can write stylistic sentences which purposely violate rules of grammar, but result in more impact and even better communication.
Still it’s great to see this kind of research. While our own descriptions aren’t guaranteed to appear in search results it’s still a good idea to improve them. I would think these findings would also translate to writing ads.
I appreciated that response, Bill. Your ‘I have a Dream’ example is a fine one.
Like Steven Bradley, above, I have serious doubts and concerns about any algorithm that tries to make judgements based on grammar.
Few modern people write or speak textbook-perfect grammar.
You are far more likely to read or hear a statement like this:
“This is what I was thinking of.”
as opposed to the correct:
“It was of this that I was thinking.”
Additionally, is it colour, optimisation and centre, or color, optimization and center?
Unless Google is prepared to put out a new grammar and dictionary that all users will adhere to in order to ‘level’ language use, I fear they may be biting off more than they can effectively chew.
Miriam
Good points, Miriam.
Google did mention a new algorithm today that they are working on introducing that may handle some of those spelling differences in a meaningful manner. Not so sure about the grammar, though. 🙂