Unsolicited Content in Human to Computer Dialog

Why Add Unsolicited Content in A Human to Computer Dialog?

Humans may engage in human-to-computer dialogs with “automated assistants.”

For example, humans may provide commands and requests using spoken natural language input (i.e. utterances) which may in some cases be converted into text and then processed, and by providing textual (e.g., typed) natural language input.

Automated assistants are typically reactive, as opposed to proactive.

At the onset of a human-to-computer dialog session between a user and an automated assistant, the automated assistant may provide at most a generic greeting, such as “Hi,” “Good morning,” etc. The automated assistant does not pro-actively obtain and provide specific information of potential interest to the user.

Consequently, the user must provide initial natural language input (e.g., spoken or typed) before the automated assistant will respond with substantive information and initiate tasks on behalf of the user.

Techniques are described herein for configuring automated assistants to proactively incorporate, into existing or newly-initiated human-to-computer dialog sessions, unsolicited content of potential interest to a user.

An automated assistant configured with this patented process may perform such incorporation when it determines that in an existing human-to-computer dialog session, the automated assistant has effectively fulfilled its obligations to the user (e.g., the automated assistant is awaiting further instructions).

This can be as simple as the user uttering “Good morning” and the automated assistant providing a generic response, “Good morning to you.”

In such a scenario, the user may likely still be engaged (at least briefly) with the human-to-computer dialog session (e.g., a chatbot screen showing an ongoing transcript of the human-to-computer dialog may still be open, the user may still be within earshot of an audio input/output device through which the human-to-computer dialog is implemented, etc.).

Accordingly, any unsolicited content that is incorporated into the human-to-computer dialog session is likely to be consumed (e.g., heard, seen, perceived, understood, etc.) by the user.

Advantages of Inserting Unsolicited Content Into a Human to Computer Dialog Session

Incorporating unsolicited content of potential interest into a human-to-computer dialog session may have several technical advantages.

An automated assistant may appear more “lifelike” or “human” to the user, which may lead to increased interaction with the automated assistant
The incorporated content may be interesting to the user because it is selected based on characteristics of the user, such as the user’s current content, such as location, status travel plans, known topics of interest
A user does not have to solicit such content, which may be helpful for users having diminished capabilities to provide inputs, such as while driving, or with physical limitations
A searcher may receive potentially helpful content that it may not otherwise have occurred to the user to solicit
The user may be provided with information that the user otherwise may have sought by submitting additional requests, and may conserve computing resources required to parse and interpret these requests

Ways that A Computer May Provide Unsolicited Content

The automated assistant may initiate a human-to-computer dialog for the incorporation of unsolicited content and incorporate unsolicited content into an existing human-to-computer dialog in response to various events.

One event may comprise when a user is within earshot of the automated assistant.

It is possible that a standalone interactive speaker that operates an automated assistant could use a sensor to detect that a human is nearby using:

An IP webcam
A motion sensor/camera incorporated into an appliance such as a smart thermostat, smoke detector, carbon monoxide detector
Detecting co-presence of another computing device carried by the user

Examples of unsolicited content could include:

Don’t forget your umbrella today, rain is expected
Don’t forgot today is your sister’s birthday
Did you hear about last night’s injury to power forward?
has gone up 8% over the last several hours

An automated assistant on a first computer may receive signals from another computer, via a sensore such as bluetooth. These signals may include:

Computing interactions by the user, such as performing a search, researching a topic, or reading an article
Runing an app, such as playing media, playing games
So forth

I have had my speaker ask me if I wanted to receive news stories on my phone for news that it just sent to me, and that made receiving those stories very convenient.

How does this passage from the patent sound?

For example, suppose a user is listening to a particular musical artist on a standalone interactive speaker (which may or may not operate an instance of the automated assistant). An automated assistant on the user’s smartphone may audibly detect the music and receive signals from the standalone interactive speaker, and in response, incorporate, into a newly-initiated or preexisting human-to-computer dialog, unsolicited content such as additional information about the artist (or the song), recommendations regarding other similar artists/songs, notification(s) of upcoming tour dates or other artist-related events, etc.

This approach to a device providing information could be very helpful. The patent goes on to provide additional examples:

Unsolicited content that is incorporated into a human-to-computer dialog may include information of potential interest to a user, such as weather, scores, traffic information, answers to questions, reminders, and actions of potential interest to a user, such as playing music, create a reminder, add the item to shopping list.

What Types of signals could bring unsolicited content?

This patent tells us that information and actions of potential interest could be selected based on a variety of signals.

Past human-to-computer dialogs between a user and an automated assistant Suppose a user researches flights to a particular destination, but does not purchase any tickets. And that a subsequent human-to-computer dialog between the automated assistant and the user is triggered, and that the automated assistant determines that it has responded to all-natural language inputs from the user. In such a scenario, the user has not yet provided any additional natural language input to the automated assistant. Accordingly, the automated assistant may take the initiative to incorporate unsolicited content including information relating to the user’s previous flight search, such as “Did you ever buy a ticket to your destination,” or “I don’t know if you’re still looking for a flight, but I found a good deal on a website.”
User location which might prompt the automated assistant to proactively suggest particular menu items, specials, calendar entries (e.g., “Don’t forget your anniversary is next Monday”), appointments (e.g., an upcoming flight may prompt an automated assistant to proactively remind the user to perform online check-in and to start packing), reminders, search history, browsing history, topics of interest, such as interest in a particular sports team might cause an automated assistant to proactively ask the user, “Did you see the score last night?”), documents (e.g., an email including an invitation to an upcoming event might prompt the automated assistant to proactively remind the user of the upcoming event), application states (e.g., “I’ve performed updates of these three applications,” “I see that you still have several applications open, which may strain the resources of your device,” “I see you’re currently streaming to your television; did you know that ?”, etc.), newly-available features (e.g., “Welcome back. While you were away I learned to call a taxi. Just tell me whenever you need one”), weather (e.g., “I see that it is nice outside. Would you like me to search for restaurants with outdoor dining?”), and so forth.

This unsolicited content for human to computer dialogs is at:

Proactive incorporation of unsolicited content into human-to-computer dialogs
Inventors: Ibrahim Badr, Zaheed Sabur, Vladimir Vuskovic, Adrian Zumbrunnen, and Lucas Mirelmann
Assignee: Google LLC
US Patent: 11,232,792
Granted: January 25, 2022
Filed: March 25, 2020

Abstract

Methods, apparatus, and computer-readable media are described related to automated assistants that proactively incorporate, into human-to-computer dialog sessions, unsolicited content of potential interest to a user.
In various implementations, in an existing human-to-computer dialog session between a user and an automated assistant, it may be determined that the automated assistant has responded to all-natural language input received from the user.
Based on the characteristics of the user, information of potential interest to the user or actions of potential interest to the user may be identified.
Unsolicited content indicative of the information of potential interest to the user or the actions may be generated and incorporated by the automated assistant into the existing human-to-computer dialog session.
In various implementations, the incorporating may be performed in response to the determining that the automated assistant has responded to all-natural language input received from the user during the human-to-computer dialog session.