Google and Spoken Queries: Understanding Stressed Pronouns

Sharing is caring!

Spoken Queries and Stressed Pronouns

The future of searches on the Web will likely involve searches by voice, as more and more people are connecting to the web with phones and Google has added voice search interfaces to its search on desktop computers.

I thought it was interesting when I ran across a patent that focused on a problem that might arise with spoken searches, and thought it was worth writing about because it’s something that we will need to become acquainted with as it becomes more commonplace.

When Amit Singhal showed off Google’s hummingbird update, he gave a presentation that showed Google handling searches involving pronouns. It’s worth watching for the information about Hummingbird, but also about how Google is becoming more conversational, and can handle things like stressed pronouns. The video is at:

I remembered the presentation about hummingbird and a more conversational Google, when I saw this patent come out from Google, which explains some of the technology behind aspects of conversational search:

Resolving pronoun ambiguity in voice queries
Inventors: Gabriel Taubman and John J. Lee;
Assignee: Google
US Patent 9,529,793
Granted: December 27, 2016
Filed: February 22, 2013

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for resolving ambiguity in received voice queries. An original voice query is received following one or more earlier voice queries, wherein the original voice query includes a pronoun or phrase. In one implementation, a plurality of acoustic parameters is identified for one or more words in the original voice query. A concept represented by the pronoun is identified based on the plurality of acoustic parameters, wherein the concept is associated with a particular query of the one or more earlier queries. The concept is associated with the pronoun. Alternatively, a concept may be associated with a phrase by using grammatical analysis of the query to relate the phrase to a concept derived from a prior query.

I did write about some papers that Google researchers had written about pronouns in the post Searching with Pronouns: What are they? Coreferences in Followup Queries

But, the granted patent from this week had an example that was worth sharing about an aspect of conversational search that wasn’t covered in one of those papers, involving stressed pronouns. Here is the example:

A voice query asks: “Who was Alexander Graham Bell’s father?”
The answer: “Alexander Melville Bell”
A followup voice query: “What is HIS birthday?”
The answer to the followup query: “Alexander Melville Bell’s birthday is 3/1/1819”

Spoken Queries and Stressed pronouns

The point behind this patent is that the search engine decided that it should tell the searcher the birthdate for the inventor’s father. This was done based upon the fact that the “HIS” in that second query was stressed, to indicate that it was about the father, and not the son mentioned in that first query.

The patent tells us of a “stress score” for spoken words or phrases in a voice query that could include “volume, pitch, frequency, duration between each spoken words, and spoken duration of words or phrases,” and it tells us that “By comparing the stress score for the pronoun to a threshold, an implementation may determine that the stress score indicates that the pronoun is stressed or not.”

The impact of a stressed query? The patent says, “For example, if a pronoun is stressed, it may indicate that it refers to a concept from an immediately preceding query, while a pronoun that is not stressed may refer to a concept from a query that occurred earlier in a series of received queries.” It’s an interesting assumption that does sound like it uses how people actually convey information during inquiries when they are having conversations. The patent does tell us about some of the science behind this determination about stressed pronouns:

For example, if the absolute measure for the volume of the pronoun is 80 dB and the average volume for the other words in voice query is 60 db, the ratio of the volumes is 1.33. This relative volume measure for the pronoun indicates that the volume of the pronoun is 33% greater than the volume of the rest of voice query. Alternatively, the relative measures can be a difference between the acoustic parameters for the pronoun and the acoustic parameters for the other words in voice query. For example, if the absolute measure for the time duration of the pronoun is 80 ms and the average time duration of the other words in voice query is 50 ms, the difference in the time duration is 30 ms. This relative time duration measure for the pronoun indicates that the time duration of the pronoun is 30 ms more than the average time duration for the words in voice query. Alternatively, the relative measures of the acoustic parameters for the pronoun can be relative to the acoustic parameters for only the words that immediately precede and follow the pronoun.

The patent provides some other examples of how stresses might be understood, including how grammatical differences may play a role.

It is interesting that these types of things may influence spoken queries. If you’ve been wondering about how Google might understand pronouns, now you have an idea of how it could understand stressed pronouns.

Sharing is caring!

27 thoughts on “Google and Spoken Queries: Understanding Stressed Pronouns”

  1. This is a very interesting It is a great article. You will surely like this also because it is a great stuff, yeah it’s give us lots of interest and pleasure. Their opportunities are so fantastic and working style so speedy.

  2. Hi David,

    Conversational search does seem to make a lot of sense as a path for Google to follow. It’s interesting seeing Google dotting all the i’s and crossing all the t’s.

    I see the following things which look like they are unique to spoken queries:

    1. Queries may refer to previous spoken queries, and entities within those
    2. Queries may be longer and contain more words than written queries
    3. Spoken queries may vary in loudness and in speed spoken, and may be considered “stressed” which may indicate importance.
    4. I remember reading something from Google (a patent or a paper) that talked about how an accent might be identified and used to personalize search results based upon that accent.

  3. Really interesting! No many blogger cover the subject as you doing here! I just wonder why is Instagram so underrated? In my case I found it as the most effective.

  4. Hi Lagertha,

    I’ve seen some posts about conversational search, but I don’t think there is a lot of data to base many blog posts upon. As for instagram, I have seen some articles about instagram, but it’s possible that people who are finding it effective need to blog about it to make it more respected.

  5. Hi Ant,

    The patent was focused upon queries themselves, and how Google might interpret spoken queries, rather than how a webmaster might optimize a page for spoken queries – this patent doesn’t give any insight optimizing for those. I thought it was worth writing about because it gives us some insight into how Google is working, rather than how we can better optimize our pages.

  6. I like seeing that Google is working hard at tweaking the finer things when it comes to speech recognition. I wish Siri could understand when I ask it to “call Stephan”, and it responds with all kinds of things other than just calling Stephan – ack!!!

  7. Hi Andy,

    It was good seeing this level of understanding from Google with pronouns. I hope it is that sophisticated in other aspects of speech interaction.

  8. Google is working hard, really nice article. i never see this type of post really interesting..

  9. Great article and very informative things you discuss. I think you done a lot research. thanks a lot for sharing these points,

  10. This spoken queries thing will take a lot of pros and cons but it’s good to hear that google is working hard about this. It is really a good idea when it comes to searching.Hope it will going to be a success.

  11. Some facts I didn’t know, surprise how google is great. thanks for sharing.can you give more post.??? .really impressed..

  12. Thanks for sharing this great informative article. I really like it as you done a lot research. Keep sharing such useful posts.

  13. There is lot of talk about google voice search feature and learning how google handles these voice queries will hopefully give me some better understanding about how to optimize them. Thanks for this really helpful piece of information.

  14. I am really happy with articles quality and presentation.Thanks a lot for keeping great stuff. I am very much thankful for this site.

  15. Thank you so much for writing this great blog with very much helpful information.

  16. Absolutely amazing work by Google, albeit in baby steps. It will likely be a long time before interpreting verbal intent beyond simply understanding word pronunciation is a flawless process, due to the seemingly endless multitude of variables, but kudos to Google for continuing to explore and develop. This is definitely a valuable step in that discovery process. Thank you for sharing.

  17. Hi Lester,

    I thought it was interesting how Google was interpreting spoken queries as well. It does seem like it potentially has risks, but it’s good hearing examples of how it might be done.

  18. Hi there, thanks for a very interesting article. The example with Bell and the fact that how “his” is stressed may influence how Google understands the query really show how Google is getting smarter. However, voice search (at least not yet) isn’t the “core” search as I believe written (or written keyword-based) search is still the major form of how people do searches. I am wondering how Google could or can detect what we mean if we ask the same two questions but using the traditional methods. How can Google know what we want to stress? Will it ever be possible that it can draw such conclusions based on what we type? Will it measure how fast we type? At what word we stop and for how long?

  19. Hi Olga,

    The patent focuses upon spoken queries, and stresses upon words in those, so it doesn’t appear to have a written comparison. I suspect the amount of spoken queries is growing as are the numbers of searches that take place on mobile devices. I do seem to recall that Google as paying attention to how we type; and we know that Google was trying to provide search results as we were typing:

    https://googleblog.blogspot.com/2010/09/search-now-faster-than-speed-of-type.html

    Maybe they can see how we are typing when we perform written queries, and those might be influenced by how we type. 🙂

Comments are closed.