Answering Featured Snippets Timely, Using Sentence Compression on News

Sharing is caring!

Featured Snippets and Sentence Compression

A couple of Augusts ago, I went to a Semantic Business and Technology conference where the head of Yahoo’s Knowledge Graph, Nicolas Torzec, discussed how updates took place to the knowledge graph when some earth-shaking event took place. He told us that they were manually editing information in that knowledge graph. Upon hearing that, I thought it seemed like an area that could have used a machine learning element to it, to automate it to keep it up to date.

Another place that would benefit from machine learning would be generating featured snippets that answer questions people might ask at Google, and it appears that they thought it might be useful there, too. A Wired Magazine article from Monday describes how the sentence compression algorithms behind these featured snippets might be used:

Google’s Hand-Fed AI Now Gives Answers, Not Just Search Results

At the heart of this approach is the crawling of a data store of news articles and other sources, with the help of a “massive team of PhD linguists it calls Pygmalion”, and the use of algorithms that are referred to as “sentence compression” algorithms that might generate answers to questions from sources such as that news source.

Curious and hopeful, I went in search of patents from Google that used “sentence compression” algorithms, and I happended to find one:

Methods and apparatus related to sentence compression
Inventors: Ekaterina Filippova and Yasemin Altun
Assigned to: Google
US Patent 9,336,186
Granted: May 10, 2016
Filed: October 10, 2013

Abstract

Methods and apparatus related to sentence compression. Some implementations are generally directed toward generating a corpus of extractive compressions and associated sentences based on a set of headline, sentence pairs from documents. Some implementations are generally directed toward utilizing a corpus of sentences and associated sentence compressions in training a supervised compression system. Some implementations are generally directed toward determining a compression of a sentence based on edge weights for edges of the sentence that are determined based on weights of features associated with the edges.

The patent doesn’t mention featured snippets, but it does mention paraphrasing sentences in a data store of titles and sentences from a news source:

The documents from which the set of headline, sentence pairs is determined may be news story documents. In some of those implementations, for each of the headline, sentence pairs the sentence is a first sentence of the respective document.

Determining the set of headline, sentence pairs of the set may include: determining non-conforming headline, sentence pairs from a larger set of headline, sentence pairs; and omitting the non-conforming headline, sentence pairs from the set of headline, sentence pairs. Determining non-conforming headline, sentence pairs may include determining the non-conforming sentence pairs as those that satisfy one or more of the following conditions: the headline is less than a headline threshold number of terms, the sentence is less than a sentence threshold number of terms, the headline does not include a verb, and the headline includes one or more of a noun, verb, adjective, and adverb whose lemma does not appear in the sentence.

A Related Sentence Compression Whitepaper

I had hoped to find more that discussed how this algorithm might be used in the generation of featured snippets, but it didn’t provide many details on that aspect of how these algorithms might be used. It does appear to be based on natural language processing. and I went looking at Google whitepapers to see if I could find more. I found a paper that looked related. On a Research at Google page for the paper Overcoming the Lack of Parallel Data in Sentence Compression they tell us is “A subset of the described data (10,000 sentence & extracted headlines pairs, with source URL and annotations) is available for download.”

That data for download includes sentences from news articles that have been tagged as different parts of speech. It looks like a lot of work, but it appears to be done in a way that does take advantage of automating processes that can keep such information up to date.

This appears to be how terms such as “sentence compression” become relevant to what SEOs do.

Sharing is caring!

7 thoughts on “Answering Featured Snippets Timely, Using Sentence Compression on News”

  1. Another excellent post Bill. Thanks for sharing these insights. I haven’t been looking into Yahoo as much as I used to. Very interesting insights.
    P.s. Love the line, “Curious, I went in search of patents from Google that used “sentence compression” algorithms” 😀

  2. Hi David,

    I do see a lot of people asking how featured snippets are created; and when I saw the Wired Article, it seemed to make sense to see if I could find out more about the algorithms mentioned in it. I’m happy I tried to find it. 🙂

  3. Hi Sonia,

    By “answering” I meant having featured snippets show up as answer to questions in queries, rather than something that might be known as “answering featured snippets. When Someone queries something like. “has the Dakota Pipeline been stopped?”, Goog might answer that with a featured snippet quickly because they are using a news source to create those featured snippets. Let’s see if there is one yet. There are Top News stories insead of a featured snippet that answer that query:

    https://www.google.com/search?sourceid=chrome-psyapi2&ion=1&espv=2&ie=UTF-8&q=has%20the%20dakota%20pipeline%20been%20stopped&oq=has%20the%20dakota&aqs=chrome.0.0j69i57j0l4.6879j0j7

  4. Hi Patrick,

    I hadn’t heard of this sentence compression approach either, so when I saw something about it, it made sense to add more details about it.

  5. Thanks for sharing such a wonderful post. It’s very useful and please keep posting such articles.
    Expecting more post from you.

Comments are closed.