Suggesting Activities
A method includes receiving inputs indicative of a user state of a user. The received inputs include one or more of: sensor inputs from one or more sensors in communication with the data processing hardware; application inputs received from one or more software applications executing on the data processing hardware or a remote device in communication with the data processing hardware; or user inputs received from a graphical user interface. The method further includes determining possible information to propose to the user based on the received inputs and executing behavior routines having corresponding objectives. Each behavior routine is configured to evaluate the possible information based on whether the possible information is related to or achieves the corresponding objective. The method includes selecting suggested information from the possible information based on evaluations of one or more behavior routines for presentation to the user.
Latest Blackwerks LLC Patents:
This U.S. patent application is a continuation of, and claims priority under 35 U.S.C. §120 from, U.S. patent application Ser. No. 14/883,991, filed on Oct. 15, 2015, which claims priority under 35 U.S.C. §119(e) to U.S. Provisional Application 62/064,053, filed Oct. 15, 2014. The disclosures of these prior applications are considered part of the disclosure of this application and are hereby incorporated by reference in their entireties
TECHNICAL FIELDThis disclosure relates to suggesting information and/or activities to a user based on an assessment of the user's current state of being.
BACKGROUNDThe use of mobile devices, such as smartphones, tablet PCs, cellular telephones, or portable digital assistants, has become widespread. At their inception, mobile devices were mainly used for voice communication, but recently they have become a reliable source for performing a range of business and personal tasks. Mobile devices are useful to obtain information by using a data connection to access the World Wide Web. The user may input a search query on a search engine website, using the mobile device, to obtain requested information. The information may relate to a location of a restaurant, hotel, shopping center, or other information. Users may use mobile devices for social media, which allows the users to create, share, or exchange information and ideas in virtual communities or networks. Social media depends on mobile and web-based technologies to allow people to share, co-create, collaborate on, discuss, and modify user-generated content.
SUMMARYOne aspect of the disclosure provides a method that includes receiving, at data processing hardware, inputs indicative of a user state of a user. The received inputs include sensor inputs from one or more sensors in communication with the data processing hardware and/or user inputs received from a graphical user interface. The method includes determining, by the data processing hardware, possible activities for the user to perform based on the received inputs, determining, by the data processing hardware, one or more predicted outcomes for each possible activity based on the received inputs, and executing, by the data processing hardware, behaviors having corresponding objectives. Each behavior is configured to evaluate a possible activity based on whether the possible activity and the corresponding one or more predicted outcomes of the possible activity achieves the corresponding objective. The method further includes selecting, by the data processing hardware, one or more possible activities based on evaluations of one or more behaviors, and outputting results including the selected one or more possible activities.
Another aspect includes a method that includes receiving, at data processing hardware, inputs indicative of a user state of a user. The received inputs include one or more of sensor inputs from one or more sensors in communication with the data processing hardware, application inputs received from one or more software applications executing on the data processing hardware or a remote device in communication with the data processing hardware, and/or user inputs received from a graphical user interface. The method includes determining, by the data processing hardware, possible information for the user based on the received inputs and executing, by the data processing hardware, behaviors having corresponding objectives. Each behavior is configured to evaluate the possible information based on whether the possible information is related to the corresponding objective. The method includes selecting, by the data processing hardware, suggested information from the possible information based on evaluations of one or more behaviors for presentation to the user.
Implementations of the disclosure may include one or more of the following optional features. The received inputs may include biometric data of the user, environmental data regarding a surrounding of the user, and/or application inputs received from one or more software applications executing on the data processing hardware or a remote device in communication with the data processing hardware. In some implementations, one or more behaviors elect to participate or not participate in evaluating the possible activities based on the received inputs. The method may include, for each behavior, determining whether any input of the received inputs is of an input type associated with the behavior, and when an input of the received inputs is of an input type associated with the behavior, incrementing an influence value associated with the behavior. When the influence value of the behavior satisfies an influence value criterion, the behavior participates in evaluating the possible activities, and when the influence value of the behavior does not satisfy the influence value criterion, the behavior does not participate in evaluating the possible activities.
The method may include, for each behavior, determining whether a decrement criterion is satisfied for the behavior and decrementing the influence value of the behavior when the decrement criterion is satisfied. In some examples, the decrement criterion is satisfied when a threshold period of time has passed since lasting incrementing the influence value. The evaluation of at least one behavior may be weighted based on the corresponding influence value of the at least one behavior.
In some implementations, the method includes determining, using the data processing hardware, the possible activities based on one or more preferences of the user. At least one behavior may evaluate a possible activity based on at least one of a history of selected activities for the user or one or more preferences of the user. In some examples, a first behavior evaluates a possible activity based on an evaluation by a second behavior of the possible activity.
Another aspect of the disclosure provides a system that includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed by the data processing hardware, cause the data processing hardware to perform operations including receiving inputs indicative of a user state of a user. The received inputs include sensor inputs from one or more sensors in communication with the data processing hardware and/or user inputs received from a graphical user interface. The operations include determining possible activities for the user to perform based on the received inputs, determining one or more predicted outcomes for each possible activity based on the received inputs, and executing behaviors having corresponding objectives. Each behavior is configured to evaluate a possible activity based on whether the possible activity and the corresponding one or more predicted outcomes of the possible activity achieves the corresponding objective. The operations further include selecting one or more possible activities based on evaluations of one or more behaviors, and outputting results including the selected one or more possible activities.
Yet another aspect provides a system that includes data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed by the data processing hardware, cause the data processing hardware to perform operations including receiving inputs indicative of a user state of a user. The inputs include sensor inputs from one or more sensors in communication with the data processing hardware, application inputs received from one or more software applications executing on the data processing hardware or a remote device in communication with the data processing hardware, and/or user inputs received from a graphical user interface. The operations include determining possible information for the user based on the received inputs and executing behaviors having corresponding objectives. Each behavior is configured to evaluate the possible information based on whether the possible information is related to the corresponding objective. The operations further include selecting suggested information from the possible information based on evaluations of one or more behaviors for presentation to the user.
Implementations of these aspects may include one or more of the following optional features. The received inputs may include biometric data of the user and/or environmental data regarding a surrounding of the user. In some implementations, one or more behaviors elect to participate or not participate in evaluating the possible activities based on the received inputs. The operations may include, for each behavior, determining whether any input of the received inputs is of an input type associated with the behavior, and when an input of the received inputs is of an input type associated with the behavior, incrementing an influence value associated with the behavior. When the influence value of the behavior satisfies an influence value criterion, the behavior participates in evaluating the possible activities, and when the influence value of the behavior does not satisfy the influence value criterion, the behavior does not participate in evaluating the possible activities.
The operations may include, for each behavior, determining whether a decrement criterion is satisfied for the behavior and decrementing the influence value of the behavior when the decrement criterion is satisfied. In some examples, the decrement criterion is satisfied when a threshold period of time has passed since lasting incrementing the influence value. The evaluation of at least one behavior may be weighted based on the corresponding influence value of the at least one behavior.
In some implementations, the operations include determining, using the data processing hardware, the possible activities based on one or more preferences of the user. At least one behavior may evaluate a possible activity based on at least one of a history of selected activities for the user or one or more preferences of the user. In some examples, a first behavior evaluates a possible activity based on an evaluation by a second behavior of the possible activity.
Another aspect of the disclosure provides a method that includes receiving, at data processing hardware, inputs indicative of a user state of a user. The inputs include sensor inputs from one or more sensors in communication with the data processing hardware and/or user inputs received from a graphical user interface displayed on a screen in communication with the data processing hardware. The method includes determining, using the data processing hardware, a collective user state based on the received inputs and determining one or more possible activities for the user and one or more predicted outcomes for each activity based on the collective user state. The method includes executing, at the data processing hardware, one or more behaviors that evaluate the one or more possible activities and/or the corresponding one or more predicted outcomes. Each behavior models a human behavior and/or a goal oriented task. The method further includes selecting, using the data processing hardware, one or more activities based on the evaluations of the one or more possible activities and/or the corresponding one or more predicted outcomes and sending results including the selected one or more activities from the data processing hardware to the screen for display on the screen.
Implementations of the disclosure may include one or more of the following optional features. In some implementations, the inputs include biometric data of the user and/or environmental data regarding a surrounding of the user. The one or more sensors may include at least one of a global positioning system, a temperature sensor, a camera, a three dimensional volumetric point cloud imaging sensor, a fingerprint reader, a blood glucose monitor, a skin PH meter, an inertial measurement unit, a microphone, a blood oxygen meter, a humidistat, or a barometer. Other sensors are possible as well.
In some implementations, the method includes querying one or more remote data sources in communication with the data processing hardware to identify possible activities and/or predicted outcomes. The method may include determining, using the data processing hardware, the one or more possible activities and the one or more predicted outcomes for each activity based on one or more preferences of the user. Each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves an objective of the behavior. Moreover, each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves a user preference stored in non-transitory memory in communication with the data processing hardware. In some examples, a first behavior evaluates an activity or a corresponding outcome based on an evaluation by a second behavior of the activity or the corresponding outcome. Each behavior may elect to participate or not participate in evaluating the one or more activities and/or the one or more predicted outcomes for each activity based on the collective user state.
When an input is related to a behavior, the method may include incrementing an influence value associated with the behavior. The input is related to the behavior when the input is of an input type associated with the behavior. In some implementations, the evaluations of each behavior can be weighted based on the influence value of the corresponding behavior. The method may include decrementing the influence value of each behavior after a threshold period of time. When an influence value equals zero, the method may include deactivating the corresponding behavior. Any behaviors having an influence value greater than zero may participate in evaluating the activities or the corresponding outcomes; and any behaviors having an influence value equal to zero may not participate in evaluating the activities or the corresponding outcomes.
In some implementations, the method includes selecting for the results a threshold number of activities having the highest evaluations or a threshold number of activities having corresponding predicted outcomes that have the highest evaluations. The method may include combining selected activities and sending a combined activity in the results.
The data processing hardware may include a user computer processor of a user device including the screen and/or one or more remote computer processors in communication with the user computer processor. For example, the computer device can be the computer processor of a mobile device, a computer processor of an elastically scalable cloud resource, or a combination thereof.
Another aspect of the disclosure provides a system that includes data processing hardware and non-transitory memory in communication with the data processing hardware. The non-transitory memory stores instructions that, when executed by the data processing hardware, cause the data processing hardware to perform operations that include receiving inputs indicative of a user state of a user. The inputs include sensor inputs from one or more sensors in communication with the data processing hardware and/or user inputs received from a graphical user interface displayed on a screen in communication with the data processing hardware. The operations include determining a collective user state based on the received inputs, determining one or more possible activities for the user and one or more predicted outcomes for each activity based on the collective user state, and executing one or more behaviors that evaluate the one or more possible activities and/or the corresponding one or more predicted outcomes. Each behavior models a human behavior and/or a goal oriented task. The operations further include selecting one or more activities based on the evaluations of the one or more possible activities and/or the corresponding one or more predicted outcomes and sending results including the selected one or more activities to the screen for display on the screen.
In some implementations, the inputs include biometric data of the user and/or environmental data regarding a surrounding of the user. The one or more sensors may include at least one of a global positioning system, a temperature sensor, a camera, a three dimensional volumetric point cloud imaging sensor, a fingerprint reader, a blood glucose monitor, a skin PH meter, an inertial measurement unit, a microphone, a blood oxygen meter, a humidistat, or a barometer. Other sensors are possible as well.
In some implementations, the operations include querying one or more remote data sources in communication with the data processing hardware to identify possible activities and/or predicted outcomes. The operations may include determining, using the data processing hardware, the one or more possible activities and the one or more predicted outcomes for each activity based on one or more preferences of the user. Each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves an objective of the behavior. Moreover, each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves a user preference stored in non-transitory memory in communication with the data processing hardware. In some examples, a first behavior evaluates an activity or a corresponding outcome based on an evaluation by a second behavior of the activity or the corresponding outcome. Each behavior may elect to participate or not participate in evaluating the one or more activities and/or the one or more predicted outcomes for each activity based on the collective user state.
When an input is related to a behavior, the operations may include incrementing an influence value associated with the behavior. The input is related to the behavior when the input is of an input type associated with the behavior. In some implementations, the evaluations of each behavior can be weighted based on the influence value of the corresponding behavior. The operations may include decrementing the influence value of each behavior after a threshold period of time. When an influence value equals zero, the operations may include deactivating the corresponding behavior. Any behaviors having an influence value greater than zero may participate in evaluating the activities or the corresponding outcomes; and any behaviors having an influence value equal to zero may not participate in evaluating the activities or the corresponding outcomes.
In some implementations, the operations include selecting for the results a threshold number of activities having the highest evaluations or a threshold number of activities having corresponding predicted outcomes that have the highest evaluations. The operations may include combining selected activities and sending a combined activity in the results.
The data processing hardware may include a user computer processor of a user device including the screen and/or one or more remote computer processors in communication with the user computer processor. For example, the computer device can be the computer processor of a mobile device, a computer processor of an elastically scalable cloud resource, or a combination thereof.
Another aspect of the disclosure provides a method that includes receiving, at data processing hardware, inputs indicative of a user state of each user of a group of users. The inputs include sensor inputs from one or more sensors in communication with the data processing hardware and/or user inputs received from a graphical user interface displayed on one or more screens in communication with the data processing hardware. The method includes determining, using the data processing hardware, a collective user state for each user based on the received inputs (e.g., inputs of that user and/or inputs associated with other users in the group) and determining one or more possible activities for group of users and one or more predicted outcomes for each activity based on the collective user states. The method includes executing, at the data processing hardware, one or more behaviors that evaluate the one or more possible activities and/or the corresponding one or more predicted outcomes. Each behavior models a human behavior and/or a goal oriented task. The method further includes selecting, using the data processing hardware, one or more activities based on the evaluations of the one or more possible activities and/or the corresponding one or more predicted outcomes and sending results including the selected one or more activities from the data processing hardware to the one or more screens for display on the one or more screens.
Implementations of the disclosure may include one or more of the following optional features. In some implementations, the inputs include biometric data of at least one user and environmental data regarding a surrounding of the at least one user. The one or more sensors may include at least one of a global positioning system, a temperature sensor, a camera, a three dimensional volumetric point cloud imaging sensor, a fingerprint reader, a blood glucose monitor, a skin PH meter, an inertial measurement unit, a microphone, a blood oxygen meter, a humidistat, or a barometer. Other sensors are possible as well.
In some implementations, the method includes querying one or more remote data sources in communication with the data processing hardware to identify possible activities and/or predicted outcomes. The method may include determining, using the data processing hardware, the one or more possible activities and the one or more predicted outcomes for each activity based on one or more preferences of the user. Each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves an objective of the behavior. Moreover, each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves a user preference stored in non-transitory memory in communication with the data processing hardware. In some examples, a first behavior evaluates an activity or a corresponding outcome based on an evaluation by a second behavior of the activity or the corresponding outcome. Each behavior may elect to participate or not participate in evaluating the one or more activities and/or the one or more predicted outcomes for each activity based on the collective user state.
When an input is related to a behavior, the method may include incrementing an influence value associated with the behavior. The input is related to the behavior when the input is of an input type associated with the behavior. In some implementations, the evaluations of each behavior can be weighted based on the influence value of the corresponding behavior. The method may include decrementing the influence value of each behavior after a threshold period of time. When an influence value equals zero, the method may include deactivating the corresponding behavior. Any behaviors having an influence value greater than zero may participate in evaluating the activities or the corresponding outcomes; and any behaviors having an influence value equal to zero may not participate in evaluating the activities or the corresponding outcomes.
In some implementations, the method includes selecting for the results a threshold number of activities having the highest evaluations or a threshold number of activities having corresponding predicted outcomes that have the highest evaluations. The method may include combining selected activities and sending a combined activity in the results.
The data processing hardware may include a user computer processor of a user device including the screen and/or one or more remote computer processors in communication with the user computer processor. For example, the computer device can be the computer processor of a mobile device, a computer processor of an elastically scalable cloud resource, or a combination thereof.
Another aspect of the disclosure provides a system that includes data processing hardware and non-transitory memory in communication with the data processing hardware. The non-transitory memory stores instructions that, when executed by the data processing hardware, cause the data processing hardware to perform operations that include receiving inputs indicative of a user state of each user of a group of users. The inputs include sensor inputs from one or more sensors in communication with the data processing hardware and/or user inputs received from a graphical user interface displayed on one or more screens in communication with the data processing hardware. The operations include determining a collective user state for each user based on the received inputs (e.g., inputs of that user and/or inputs associated with other users in the group), determining one or more possible activities for the group of users and one or more predicted outcomes for each activity based on the collective user states, and executing one or more behaviors that evaluate the one or more possible activities and/or the corresponding one or more predicted outcomes. Each behavior models a human behavior and/or a goal oriented task. The operations further include selecting one or more activities based on the evaluations of the one or more possible activities and/or the corresponding one or more predicted outcomes and sending results including the selected one or more activities to the one or more screens for display on the one or more screens.
In some implementations, the inputs include biometric data of at least one user and environmental data regarding a surrounding of at least one user. The one or more sensors may include at least one of a global positioning system, a temperature sensor, a camera, a three dimensional volumetric point cloud imaging sensor, a fingerprint reader, a blood glucose monitor, a skin PH meter, an inertial measurement unit, a microphone, a blood oxygen meter, a humidistat, or a barometer. Other sensors are possible as well.
In some implementations, the operations include querying one or more remote data sources in communication with the data processing hardware to identify possible activities and/or predicted outcomes. The operations may include determining, using the data processing hardware, the one or more possible activities and the one or more predicted outcomes for each activity based on one or more preferences of the user. Each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves an objective of the behavior. Moreover, each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves a user preference stored in non-transitory memory in communication with the data processing hardware. In some examples, a first behavior evaluates an activity or a corresponding outcome based on an evaluation by a second behavior of the activity or the corresponding outcome. Each behavior may elect to participate or not participate in evaluating the one or more activities and/or the one or more predicted outcomes for each activity based on the collective user state.
When an input is related to a behavior, the operations may include incrementing an influence value associated with the behavior. The input is related to the behavior when the input is of an input type associated with the behavior. In some implementations, the evaluations of each behavior can be weighted based on the influence value of the corresponding behavior. The operations may include decrementing the influence value of each behavior after a threshold period of time. When an influence value equals zero, the operations may include deactivating the corresponding behavior. Any behaviors having an influence value greater than zero may participate in evaluating the activities or the corresponding outcomes; and any behaviors having an influence value equal to zero may not participate in evaluating the activities or the corresponding outcomes.
In some implementations, the operations include selecting for the results a threshold number of activities having the highest evaluations or a threshold number of activities having corresponding predicted outcomes that have the highest evaluations. The operations may include combining selected activities and sending a combined activity in the results.
The data processing hardware may include a user computer processor of a user device including the screen and/or one or more remote computer processors in communication with the user computer processor. For example, the computer device can be the computer processor of a mobile device, a computer processor of an elastically scalable cloud resource, or a combination thereof.
Yet another aspect of the disclosure provides a method that includes receiving, at data processing hardware, inputs indicative of a user state of a user. The inputs include sensor inputs from one or more sensors in communication with the data processing hardware and/or user inputs received from a graphical user interface displayed on a screen in communication with the data processing hardware. In response to receiving a trigger sensor input, the method includes determining, using the data processing hardware, a collective user state based on the received inputs and determining one or more possible activities for the user and one or more predicted outcomes for each activity based on the collective user state. The method includes executing, at the data processing hardware, one or more behaviors that evaluate the one or more possible activities and/or the corresponding one or more predicted outcomes. Each behavior models a human behavior and/or a goal oriented task. The method further includes selecting, using the data processing hardware, one or more activities based on the evaluations of the one or more possible activities and/or the corresponding one or more predicted outcomes and sending results including the selected one or more activities from the data processing hardware to the screen for display on the screen.
Implementations of the disclosure may include one or more of the following optional features. In some implementations, the inputs include biometric data of the user and environmental data regarding a surrounding of the user. The one or more sensors may include at least one of a global positioning system, a temperature sensor, a camera, a three dimensional volumetric point cloud imaging sensor, a fingerprint reader, a blood glucose monitor, a skin PH meter, an inertial measurement unit, a microphone, a blood oxygen meter, a humidistat, or a barometer. Other sensors are possible as well. The trigger sensor input may be from the inertial measurement unit, indicating a threshold amount of shaking of the inertial measurement unit (e.g., indicating that a user is shaking a mobile device).
In some implementations, the method includes querying one or more remote data sources in communication with the data processing hardware to identify possible activities and/or predicted outcomes. The method may include determining, using the data processing hardware, the one or more possible activities and the one or more predicted outcomes for each activity based on one or more preferences of the user. Each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves an objective of the behavior. Moreover, each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves a user preference stored in non-transitory memory in communication with the data processing hardware. In some examples, a first behavior evaluates an activity or a corresponding outcome based on an evaluation by a second behavior of the activity or the corresponding outcome. Each behavior may elect to participate or not participate in evaluating the one or more activities and/or the one or more predicted outcomes for each activity based on the collective user state.
When an input is related to a behavior, the method may include incrementing an influence value associated with the behavior. The input is related to the behavior when the input is of an input type associated with the behavior. In some implementations, the evaluations of each behavior can be weighted based on the influence value of the corresponding behavior. The method may include decrementing the influence value of each behavior after a threshold period of time. When an influence value equals zero, the method may include deactivating the corresponding behavior. Any behaviors having an influence value greater than zero may participate in evaluating the activities or the corresponding outcomes; and any behaviors having an influence value equal to zero may not participate in evaluating the activities or the corresponding outcomes.
In some implementations, the method includes selecting for the results a threshold number of activities having the highest evaluations or a threshold number of activities having corresponding predicted outcomes that have the highest evaluations. The method may include combining selected activities and sending a combined activity in the results. The results may include one or more activity records, where each activity record includes an activity description and an activity location. The method may include displaying on the screen a map, and for each activity record, displaying the activity location on the map and the activity description.
The data processing hardware may include a user computer processor of a user device including the screen and/or one or more remote computer processors in communication with the user computer processor. For example, the computer device can be the computer processor of a mobile device, a computer processor of an elastically scalable cloud resource, or a combination thereof.
Another aspect of the disclosure provides a system that includes data processing hardware and non-transitory memory in communication with the data processing hardware. The non-transitory memory stores instructions that, when executed by the data processing hardware, cause the data processing hardware to perform operations that include receiving inputs indicative of a user state of a user. The inputs include sensor inputs from one or more sensors in communication with the data processing hardware and/or user inputs received from a graphical user interface displayed on a screen in communication with the data processing hardware. In response to receiving a trigger sensor input, the operations include determining a collective user state based on the received inputs, determining one or more possible activities for the user and one or more predicted outcomes for each activity based on the collective user state, and executing one or more behaviors that evaluate the one or more possible activities and/or the corresponding one or more predicted outcomes. Each behavior models a human behavior and/or a goal oriented task. The operations further include selecting one or more activities based on the evaluations of the one or more possible activities and/or the corresponding one or more predicted outcomes and sending results including the selected one or more activities to the screen for display on the screen.
In some implementations, the inputs include biometric data of the user and environmental data regarding a surrounding of the user. The one or more sensors may include at least one of a global positioning system, a temperature sensor, a camera, a three dimensional volumetric point cloud imaging sensor, a fingerprint reader, a blood glucose monitor, a skin PH meter, an inertial measurement unit, a microphone, a blood oxygen meter, a humidistat, or a barometer. Other sensors are possible as well. The trigger sensor input may be from the inertial measurement unit, indicating a threshold amount of shaking of the inertial measurement unit (e.g., indicating that a user is shaking a mobile device).
In some implementations, the operations include querying one or more remote data sources in communication with the data processing hardware to identify possible activities and/or predicted outcomes. The operations may include determining, using the data processing hardware, the one or more possible activities and the one or more predicted outcomes for each activity based on one or more preferences of the user. Each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves an objective of the behavior. Moreover, each behavior may evaluate an activity or a corresponding outcome positively when the activity or the corresponding outcome at least partially achieves a user preference stored in non-transitory memory in communication with the data processing hardware. In some examples, a first behavior evaluates an activity or a corresponding outcome based on an evaluation by a second behavior of the activity or the corresponding outcome. Each behavior may elect to participate or not participate in evaluating the one or more activities and/or the one or more predicted outcomes for each activity based on the collective user state.
When an input is related to a behavior, the operations may include incrementing an influence value associated with the behavior. The input is related to the behavior when the input is of an input type associated with the behavior. In some implementations, the evaluations of each behavior can be weighted based on the influence value of the corresponding behavior. The operations may include decrementing the influence value of each behavior after a threshold period of time. When an influence value equals zero, the operations may include deactivating the corresponding behavior. Any behaviors having an influence value greater than zero may participate in evaluating the activities or the corresponding outcomes; and any behaviors having an influence value equal to zero may not participate in evaluating the activities or the corresponding outcomes.
In some implementations, the operations include selecting for the results a threshold number of activities having the highest evaluations or a threshold number of activities having corresponding predicted outcomes that have the highest evaluations. The operations may include combining selected activities and sending a combined activity in the results. The results may include one or more activity records, where each activity record includes an activity description and an activity location. The method may include displaying on the screen a map, and for each activity record, displaying the activity location on the map and the activity description.
The data processing hardware may include a user computer processor of a user device including the screen and/or one or more remote computer processors in communication with the user computer processor. For example, the computer device can be the computer processor of a mobile device, a computer processor of an elastically scalable cloud resource, or a combination thereof.
Another aspect provides a method that includes receiving, at data processing hardware, inputs indicative of a user state of a user. The received inputs include one or more of: 1) sensor inputs from one or more sensors in communication with the data processing hardware; 2) application inputs received from one or more software applications executing on the data processing hardware or a remote device in communication with the data processing hardware; and/or 3) user inputs received from a graphical user interface. The method includes determining, by the data processing hardware, a collective user state of the user based on the received inputs and obtaining, at the data processing hardware, user data of other users. The user data of each other user includes a collective user state of the corresponding other user. The method includes displaying, on a screen in communication with the data processing hardware other user glyphs representing the other users. Each other user glyph: 1) at least partially indicates the collective user state of the corresponding other user; and/or 2) is associated with a link to a displayable view indicating the collective user state of the corresponding other user and/or the inputs used to determine the collective user state of the corresponding other user.
In some implementations, the method includes obtaining the user data of the other users that have corresponding collective user states satisfying a threshold similarity with the collective user state of the user. The method may include arranging each other user glyph on the screen based on a level of similarity between the collective user state of the user and the collective user state of the corresponding other user. In some examples, a size, a shape, a color, a border, and/or a position on the screen of each other user glyph is based on a level of similarity between the collective user state of the corresponding other user and the collective user state of the user.
The method may include displaying a user glyph representing the user in a center portion of the screen and the other user glyphs around the user glyph. The other user glyphs may be displayed in concentric groupings about the user glyph based on a level of similarity between the collective user states of the corresponding other users and the collective user state of the user.
In some implementations, the method includes receiving, at the data processing hardware, an indication of a selection of one or more other user glyphs and executing, by the data processing hardware, messaging (e.g., via a messaging view) between the user and the one or more other users corresponding to the selected one or more other user glyphs. The method may include receiving a gesture across the screen, where the gesture indicates selection of the one or more other user glyphs. In some examples, the method includes receiving, at the data processing hardware, an indication of a selection of a messenger glyph displayed on the screen. The messenger glyph has a reference to an application executable on the data processing hardware and indicates one or more operations that cause the application to enter an operating state that allows messaging between the user and the one or more other users corresponding to the selected one or more other user glyphs.
In some implementations, the method includes displaying a map on the screen and arranging the other user glyphs on the screen based on geolocations of the corresponding other users. The user data of each other user may include the geolocation of the corresponding other user. Moreover, the method may include displaying a user glyph representing the user on the map based on a geolocation of the user.
The method may include receiving, at the data processing hardware, an indication of a selection of one or more other user glyphs and determining, by the data processing hardware, possible activities for the user and the one or more other users corresponding to the selected one or more other user glyphs to perform based on the collective user states of the user and the one or more other users. The method may also include executing, by the data processing hardware, behaviors having corresponding objectives. Each behavior is configured to evaluate a possible activity based on whether the possible activity achieves the corresponding objective. The method includes selecting, by the data processing hardware, one or more possible activities based on evaluations of one or more behaviors and displaying, by the data processing hardware, results on the screen. The results include the selected one or more possible activities. In some examples, the method includes determining, by the data processing hardware, one or more predicted outcomes for each possible activity based on the collective user states of the user and the one or more other users. In such examples, each behavior is configured to evaluate a possible activity based on whether the possible activity and the corresponding one or more predicted outcomes of the possible activity achieves the corresponding objective. In additional examples, the method may include receiving an indication of a gesture across the screen indicating selection of the one or more other user glyphs.
In some implementations, at least one behavior is configured to elect to participate or not participate in evaluating the possible activities based on the received inputs. The method may include, for each behavior determining whether any input of the received inputs is of an input type associated with the behavior, and when an input of the received inputs is of an input type associated with the behavior, incrementing an influence value I associated with the behavior. When the influence value I of the behavior satisfies an influence value criterion, the behavior participates in evaluating the possible activities; and when the influence value I of the behavior does not satisfy the influence value criterion, the behavior does not participate in evaluating the possible activities. In some examples, the method includes, for each behavior, determining whether a decrement criterion is satisfied for the behavior and decrementing the influence value of the behavior when the decrement criterion is satisfied. The decrement criterion may be satisfied when a threshold period of time has passed since last incrementing the influence value. In some examples, the evaluation of at least one behavior is weighted based on the corresponding influence value of the at least one behavior. Moreover, the method may include determining the possible activities based on one or more preferences of the user. At least one behavior may evaluate a possible activity based on at least one of a history of selected activities for the user or one or more preferences of the user. Furthermore, a first behavior may evaluate a possible activity based on an evaluation by a second behavior of the possible activity.
In some implementations, the method includes receiving, at the data processing hardware a selection of a suggestion glyph displayed on the screen and, in response to the selection of the suggestion glyph, displaying, by the data processing hardware, an activity type selector on the screen. The method may further include receiving, at the data processing hardware, a selection of an activity type and filtering, by the data processing hardware, the results based on the selected activity type.
Another aspect provides a method that includes receiving, at data processing hardware, a request of a requesting user to identify other users as likely participants for a possible activity. Each user has an associated collective user state based on corresponding inputs that include one or more of: 1) sensor inputs from one or more sensors; 2) application inputs received from one or more software applications executing on the data processing hardware or a remote device in communication with the data processing hardware; and/or 3) user inputs received from a graphical user interface. The method may include, for each other user: 1) executing, by the data processing hardware, behaviors having corresponding objectives, where each behavior is configured to evaluate the possible activity based on whether the possible activity achieves the corresponding objective; and 2) determining, by the data processing hardware, whether the other user is a likely participant for the possible activity based on evaluations of one or more of the behaviors. The method includes outputting results identifying the other users determined as being likely participants for the possible activity.
In some implementations, each other user is associated with the user based on a geographical proximity to the user, a linked relationship (e.g., family member, friend, co-worker, acquaintance, etc.). Other relationships are possible as well to narrow a pool of other users.
In some implementations, at least one behavior is configured to elect to participate or not participate in evaluating the possible activity based on the corresponding inputs of the other user. The method may include, for each behavior determining whether any input of the other user is of an input type associated with the behavior and, when an input of the other user is of an input type associated with the behavior, incrementing an influence value associated with the behavior. When the influence value of the behavior satisfies an influence value criterion, the behavior participates in evaluating the possible activity; and when the influence value of the behavior does not satisfy the influence value criterion, the behavior does not participate in evaluating the possible activity. The method may include, for each behavior, determining whether a decrement criterion is satisfied for the behavior and decrementing the influence value of the behavior when the decrement criterion is satisfied. The decrement criterion may be satisfied when a threshold period of time has passed since last incrementing the influence value.
In some examples, the evaluation of at least one behavior is weighted based on the corresponding influence value of the at least one behavior. At least one behavior may evaluate the possible activity based on at least one of a history of positively evaluated activities for the other user or one or more preferences of the other user. Moreover, a first behavior may evaluate the possible activity based on an evaluation by a second behavior of the possible activity.
The method may include displaying, on a screen in communication with the data processing hardware, other user glyphs representing the selected other users. Each other user glyph: 1) at least partially indicates the collective user state of the corresponding other user; and/or 2) is associated with a link to a displayable view indicating the collective user state of the corresponding other user and/or inputs used to determine the collective user state of the corresponding other user.
Another aspect provides a method that includes receiving, at data processing hardware, inputs indicative of a user state of a user. The received inputs include one or more of: 1) sensor inputs from one or more sensors in communication with the data processing hardware; 2) application inputs received from one or more software applications executing on the data processing hardware or a remote device in communication with the data processing hardware; and/or 3) user inputs received from a graphical user interface. The method includes determining, by the data processing hardware, a collective user state of the user based on the received inputs and receiving, at the data processing hardware, a request of a requesting user to identify other users as likely participants for a possible activity. The method further includes obtaining, at the data processing hardware, user data of other users having corresponding collective user states satisfying a threshold similarity with the collective user state of the user and outputting results identifying the other users based on the corresponding user data.
The details of one or more implementations of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims.
Like reference symbols in the various drawings indicate like elements.
DETAILED DESCRIPTIONThe present disclosure describes a system that allows a user to learn about a current state of physical and emotional well-being of herself/himself and other users to foster meaningful communications and interactions amongst the user and the other users. The system may gather inputs from a variety of sources that include, but are not limited to, sensors, software applications, and/or the user to determine a collective user state of the user. The system may display representations (e.g., icons or images) of the user and other users in an arrangement that allows the user to identify and connect (e.g., message) with other users most similar/dissimilar to the user at that moment. Moreover, the user may view the collective user states of the user and other users to learn more about each of them. The system may suggest activities or information to the user or a group of users based on the collective user state of each user.
The system may gather data of the user and his/her surrounding environment to know the context of the user's current state of being, and may model the human thought process to suggest activities and/or information to the user. Unlike reactive systems that provide information in response to a user-entered query, the system may proactively suggest activities and information based on the user's current state of being. Moreover, the activities can be tailored for the user to enhance a life objective or certain relationships with other users.
Referring to
The search system 300 may use (e.g., query) the data sources 130 when generating search results 230. Data retrieved from the data sources 130 can include any type of data related to assessing a current state of the user 10. Moreover, the data retrieved from the data sources 130 may be used to create and/or update one or more databases, indices, tables (e.g., an access table), files, or other data structures of the search system 300.
The data sources 130 may include a variety of different data providers. The data sources 130 may include application developers 130a, such as application developers' websites and data feeds provided by developers and operators of digital distribution platforms 130b configured to distribute content to user devices 200. Example digital distribution platforms 130b include, but are not limited to, the GOOGLE PLAY® digital distribution platform by Google, Inc., the APP STORE® digital distribution platform by Apple, Inc., and WINDOWS PHONE® Store developed by Microsoft Corporation.
The data sources 130 may also include websites, such as websites that include web logs 130c (i.e., blogs), review websites 130d, or other websites including data related to assessing a state of the user 10. Additionally, the data sources 130 may include social networking sites 130e, such as “FACEBOOK®” by Facebook, Inc. (e.g., Facebook posts) and “TWITTER®” by Twitter Inc. (e.g., text from tweets). Data sources 130 may also include online databases 130f that include, but are not limited to, data related to movies, television programs, music, and restaurants. Data sources 130 may also include additional types of data sources in addition to the data sources described above. Different data sources 130 may have their own content and update rate.
User devices 200 can be any computing devices capable of communicating with the search system 300. User devices 200 include, but are not limited to, mobile computing devices, such as laptops, tablets, smart phones, and wearable computing devices (e.g., headsets and/or watches). User devices 200 may also include other computing devices having other form factors, such as desktop computers, vehicles, gaming devices, televisions, or other appliances (e.g., networked home automation devices and home appliances).
The user device 200 may use any of a variety of different operating systems 206a. In examples where a user device 200 is a mobile device, the user device 200 may run an operating system including, but not limited to, ANDROID® developed by Google Inc., IOS® developed by Apple Inc., or WINDOWS PHONE® developed by Microsoft Corporation. Accordingly, the operating system 206a running on the user device 200 may include, but is not limited to, one of ANDROID®, IOS®, or WINDOWS PHONE®. In an example where a user device is a laptop or desktop computing device, the user device may run an operating system including, but not limited to, MICROSOFT WINDOWS® by Microsoft Corporation, MAC OS® by Apple, Inc., or Linux. User devices 200 may also access the search system 300 while running operating systems 206a other than those operating systems 206a described above, whether presently available or developed in the future.
In some implementations, the user device 200 includes one or more sensors 208 in communication with the computing device 202 and capable of measuring a quality, such as a biometric quality, of the user 10. The sensor(s) 208 may be part of the user device 200 (e.g., integrally attached) and/or external from (e.g., separate and remote from, but in communication with) the user device 200. Sensors 208 separate and remote from the user device may communicate with the user device 200 through the network 120, wireless communication, such as Bluetooth or Wi-Fi, wired communication, or some other form of communication. The computing device 202 receives biometric data 212 (e.g., sensor signals or bioinformatics) and/or environmental data 214 (e.g., sensor signals, data structures, data objects, etc.) from one or more sensors 208. Examples of biometric data 212 may include, but are not limited to, a temperature of the user 10, an image (e.g., 2D image, 3D image, infrared image, etc.) of the user 10, a fingerprint of the user 10, a sound of the user 10, a blood oxygen concentration of the user 10, a blood glucose level of the user 10, a skin PH of the user 10, a blood alcohol level of the user 10, an activity level of the user 10 (e.g., walking step count or other movement indicator), a wake-up time of the user 10, a sleep time of the user 10, eating times, eating duration, eating type (e.g., meal vs. snack), etc. Examples of environmental data 214 may include, but are not limited to, a geolocation of the user device 200, a temperature, humidity, and/or barometric pressure about the user device 200, a weather forecast for a location of the user device 200, an image (e.g., 2D image, 3D image, infrared image, etc.) of a surrounding of the user device 200, a sound about the user 10, etc.
Example sensors 208 that may be included with the user device 200 include, but are not limited to, a camera 208a (e.g., digital camera, video recorder, infrared imaging sensor, 3D volumetric point cloud imaging sensor, stereo imaging sensor, etc.), a microphone 208b, a geolocation device 208c, an inertial measurement unit (IMU) 208d (e.g., 3-axis accelerometer), a fingerprint reader 208e, a blood oxygen meter 208f, a PH meter 208g, etc. The camera 208a may capture image data indicative of an appearance of the user 10 and/or an environment or scene about the user 10. The microphone 208b may sense audio of the user 10 and/or an environment or scene about the user 10. Example sensors 208 that may be separate from the user device 200 include a camera 208a, a temperature sensor 208h, a humidistat 208i, a barometer 208j, or any sensing device 208n capable of delivering a signal to the user device 200 that is indicative of the user 10, a surrounding of the user 10, or something that can affect the user 10.
If the user device 200 does not include a geolocation device 208c, the user device 200 may provide location data as an input 210 in the form of an internet protocol (IP) address, which the search system 300 may use to determine a location of the user device 200. Any of the sensors 208 described as being included in the user device 200 may be separate from the user device 200, and any of the sensors 208 described as being separate from the user device 200 may be included with the user device 200.
In some examples, the user device 200 runs a native application 206 dedicated to interfacing with the search system 300; while in other examples, the user device 200 communicates with the search system 300 using a more general application 206, such as a web-browser application 206 accessed using a web browser. In some implementations, the search application 206b receives one or more inputs 210, such as biometric data 212 or environmental data 214 from the sensor(s) 208, associated software, and/or the user 10 via a graphical user interface (GUI) 250 and transmits a search request 220 based on the received inputs 210 to the search system 300. The search application 206b may also receive platform data 218 form the user device 200 (e.g., version of the operating system 206a, device type, and web-browser version), an identity of the user 10 of the user device 200 (e.g., a username), partner specific data, and/or other data and include that information in the search request 220 as well. The search application 206b receives a search result set 230 from the search system 300, in response to submitting the search request 220, and optionally displays one or more result records 232 of the search results set 230 on the display 240 of the user device 200. In some implementations, the search request 220 includes a search query 222 containing a user specified selection (e.g., a category, genre, or string). The search application 206b may display a graphical user interface (GUI) 250 on the display 240 that may provide a structured environment to receive inputs 210 and display the search results 230, 232. In some implementations, the search application 206b is a client-side application and the search system 300 executes on the remote system 110 as a server-side system.
Referring to
In some implementations, the state analyzer 400 models the user 10 using the received user state indicator(s) 210 to generate the collective user state 420. Each received user state indicator 210 provides an aspect of the modeled user 10 as the collective user state 420. The state analyzer 400 may store the collective user state 420 in memory, such as the storage resources 114 of the remote system 110. In some examples, the state analyzer 400 generates and/or stores the collective user state 420 as an object, such as a Java script object notation (JSON) object, a metadata data object, structured data, or unstructured data. Other methods of storing the collective user state 420 are possible as well.
The input/user state indicator 210 may include any information indicative of the user's state of being, in terms of a physical state of being and/or an emotional state of being. Optional examples of a physical state may include, but are not limited to, date and/or time stamp 210a (e.g., from the computing device 202 of the user device 200), a location 210b of the user 10, a user-identified state indicator 210c of a physical well-being of the user 10, and a sensed indicator 210d of a physical well-being of the user 10 (e.g., biometrics). The location 210b of the user 10 may be a geolocation (e.g., latitude and longitude coordinates) of the user device 200, a description of the physical location of the user 10 in terms of landmarks and/or environmental descriptions, a description of a dwelling or building, a floor of the dwelling or building, a room of the dwelling or building, an altitude, etc. Examples of emotional states include, but are not limited to, user-identified state indicators 210c of an emotional state of the user 10 and sensed indicators 210d of a physical well-being of the user 10 (e.g., biometrics).
In some examples, the state analyzer 400 receives image inputs 210 of the user 10 from the camera 208a and determines an emotional state of the user 10 based on the images 210 (and optionally other inputs 210). The state analyzer 400 can gauge whether the user 10 is angry, happy, sad, surprised, eating, moving, etc. based on the image inputs 210. The image inputs 210 may be considered as user-identified state indicators 210c and/or sensed indicators 210d.
The user device 200 may receive the user-identified state indicator 210c from the user 10 through the GUI 250. The user-identified state indicator 210c may include one or more selections of images and/or description indicative of different states of physical or emotional well-being or states of the user 10. In some examples, the user 10 can select one or more user-identified state indicators 210c on the GUI 250 that correspond to a combination of different user state indicators 210. The search application 206b may execute logic that disallows inconsistent selections. For example, the search application 206b may not allow the user 10 to select user-identified state indicators 210c of happy and sad at the same time.
The user state indicator 210 may optionally include user state indicators 210 of friends (referred to as friend state indicator 210e) from the remote system 110 or a data source. The friend state indicator 210e may be from any person having an identified association with the user 10. The user 10 may designate the associations of other people with their account and/or the state analyzer 400 may identify and designate other people as having an association with the user 10 based on the user state indicator 210 of other users 10 and/or authorized searching of email account(s), social networking account(s), or other online resources of the user 10.
The user state indicator 210 may optionally include a partner metric 210f (e.g., available funds from a banking institution) received from the user device 200 (e.g., as a user input) and/or from a remote data source 130 (e.g., a linked back account). The partner entities may be data sources 130 that provide information relative the user's state. For example, a mobile payment plan can provide mobile payment information, such as a purchase time, purchase location, store entity, goods purchased, purchase amount, and/or other information, which the search system 300 can use to determine the user collective state 420. Moreover, the activity system 500 may use partner metrics 210f to suggest activities A and predict outcomes O, the behavior system 600 may use partner metrics 210f to evaluate the activities A and predicted outcomes O, and the activity selector 700 may use partner metrics 210f to select one or more activities A. Other examples of partner metrics 210f include, but are not limited to, fitness and/or nutrition information of the user 10 from a fitness application, e.g., a data source 130, dating information from a dating application, work history or work activities from a work related application, such as LinkedIn®, or any other application. The application(s) may be installed on the user device 200 or offered as web-based applications. The search system 300 may access the partner metrics 210f via an application programming interface (API) associated with each application or other data retrieval methods.
Similarly, the user state indicator 210 may optionally include a user schedule 210g received from the user device 200 (e.g., as a user input) and/or from a remote data source 130 (e.g., a linked scheduler or partner system 122). The schedule may be related to eating, exercise, work, to-do list, etc.
Referring to
In the example shown in
In the example shown in
Referring to
Referring to
Referring to
Referring also to
In some implementations, the activity generator 530 seeks activities A relevant to active behaviors 610. The activity generator 530 may identify all or a sub-set of the active behaviors 610 and then seek activities A that each behavior 610 can evaluate positively. For example, if a sports behavior 610e is active, then the activity generator 530 may seek possible activities A related to sports.
After the activity generator 530 generates a collection of possible activities A, the outcome generator 540 generates a collection of one or more predicted outcomes O for each activity A. In some implementations, the outcome generator 540 retrieves possible outcomes O from a data store storing outcomes O for various activities A. For example, the outcome generator 540 may query the data source(s) 130 for possible outcomes O matching criteria indicative of the activity A. The data source(s) 130 may include databases, partner systems 122, and other sources of information.
Referring to
A behavior 610 may model a human behavior and/or a goal oriented task. Each behavior 610 may have a specific objective. Example behaviors 610 include, but are not limited to, an eating behavior 610a, a happiness behavior 610b (e.g., a pursuit of happiness), a retail shopping behavior 610c, a grocery shopping behavior 610d, a sports behavior 610e, a love behavior 610f, a work behavior 610g, a leisure behavior 610h, etc.
Behaviors 610 may model psychological decision making of humans. Moreover, the behaviors 610 may be configurable. In some examples, the user 10 may set a preference P to configure or bias one or more behaviors 610 to evaluate activities A and/or outcomes O toward that bias. In some examples, the user 10 can set a preference P to have the search system 300 aid the user 10 in making better choices (e.g., choices toward a healthier lifestyle). For example, the user 10 may set a preference P to bias one or more behaviors 610 to evaluate activities A and/or outcomes O that help the user 10 live a healthier lifestyle (e.g., in terms of diet, exercise, relationships, work, etc.).
A behavior 610 may have one or more objectives that it uses when evaluating activities A and/or outcomes O. The behavior 610 may evaluate activities A, outcomes O, or activities A and outcomes O. The behavior 610 may execute a scoring algorithm or model that evaluates outcomes O against the one or more objectives. The behavior 610 may score activities A and/or outcomes O fulfilling the objective(s) higher than other activities A and/or outcomes O that do not fulfill the objective(s). Moreover, the evaluations of the activities A and/or outcomes O may be weighted. For example, an eating behavior 610a may evaluate an activity A based on whether the predicted outcome O will make the user 10 less hungry. Moreover, the outcome evaluation may be weighted based on a user state of hunger and on the likelihood of fulfilling the objective of making the user 10 less hurry. For example, the eating behavior 610a may evaluate a first activity A1 of going to a restaurant to eat pizza more favorably than a second activity A2 of going to the cinema, because a predicted first outcome O1 of going to a restaurant to pizza will more likely have an outcome O of satisfying a user state of hunger than going to the cinema, even though a predicted second outcome O2 for the second activity A2 of going to the cinema may include eating popcorn.
A behavior 610 may optionally base its evaluations E on preferences P, P1-Pn in the user profile 14 of the user 10. For example, the eating behavior 610a may evaluate a third activity A3 of going to LOU MALNATIS® (a registered trademark of Lou Malnatis, Inc.) to eat pizza more favorably than the first activity A1 of going to PIZZA HUT® (a registered trademark of Pizza Hut, Inc.) to eat pizza, when a first preference P1 in the user profile 14 indicates that LOU MALNATIS pizza is the user's favorite brand of pizza. Therefore, a behavior 610 may use the one or more objectives of that behavior 610 in combination with one or more preferences P, P1-Pn of the user profile 14 of the user 10 to evaluate activities A and/or outcomes O of those activities A.
The activity-outcome evaluation E of one behavior 610 may be used by another behavior 610 when evaluating the corresponding activity A and/or outcome O. For example, a happiness behavior 610b may evaluate the third activity A3 of going to eat LOU MALNATIS pizza more favorably based the favorable evaluation of the eating behavior 610a and on the corresponding predicted outcome O3 that eating pizza will make the user 10 more happy (e.g., versus sad). Moreover, the collective user state 420 may indicate that the user 10 is cold, based on sensor data 212 of a sensor 208, and the happiness behavior 610b may evaluate the third activity A3 of going to eat LOU MALNATIS pizza even more favorably based on the predicted outcome O3 that eating pizza will make the user 10 warmer and therefore happier. Therefore, the behavior system 600 may execute many combinations of evaluations by behaviors 610 (some in parallel or some in series) based on prior evaluations, preferences P, etc.
Based on internal policy or external input (e.g., the collective user state 420 or other information), each behavior 610 may optionally decide whether or not it wants to participate in evaluating any activities A in the activity-outcome set 520. In some examples, if the collective user state 420 indicates that the user 10 is full (i.e., not hungry), the eating behavior 610a may opt out of evaluating the activities A and outcomes O. In other examples, if the collective user state 420 indicates that the user 10 is full (i.e., not hungry), the eating behavior 610a may evaluate activities A having predicted outcomes O of making the user 10 more full as undesirable (e.g., a poor evaluation or a low score). Each behavior 610 may decide to participate or not participate in evaluating activities A and/or outcomes O based on the inputs 210 (e.g., based on the collective user state 420, a history of received inputs 210, a rate of received inputs 210, input types 216, and/or other factors related to inputs 210).
Different inputs/user state indicators 210 can trigger different behaviors 610, 610a-n. A behavior 610 may persist for a duration of time. In some examples, a behavior 610 has a state 612 and exists in an active state or an inactive state. Certain types of inputs 210 may pertain to certain types of behaviors 610. One or more input types/user state indicator types 216 may be associated with each behavior 610. In other words, each behavior 610 may have an associated collection of input types 216 that the behavior 610 finds pertinent to its operation. For example, an input type 216 of hunger level for a user-defined input 210 of hunger having a scale (e.g. 1-10) indicating a level of hunger can be related to an eating behavior 610a. Another input type 216 that may be associated with the eating behavior 610a is proximity (which may be strongly typed as a distance in miles) for an input 210 of distance to a nearest restaurant. When the search system 300 (e.g., in particular, the behavior system 600) receives an input 210 of a type 216 associated with a behavior 610, the receipt of that input 210 may trigger activation of the behavior 610. The receipt of the input 210 may cause a behavior 610 to change state 612 from an inactive state to an active state.
In addition to becoming active, upon the receipt of one or more inputs 210 having a type 216 associated with the behavior 610, the number of those inputs 210, in some implementations, has a direct correlation to an influence I of the behavior 610. In other words, the greater the number of received inputs 210 having a type 216 associated with the behavior 610, the greater the influence I of that behavior 610. Evaluations of predicted outcomes O of a behavior 610 may be weighted based on the influence I of the behavior 610. For example, the evaluation E can be a number, which is multiplied by the influence I (e.g., a number). As a result, behaviors 610 with greater influence I have a relatively greater influence on the selection of an activity A.
In some implementations, the influence I is a count. Each time the behavior system 600 receives an input 210, the behavior system 600 increments a value of the influence I of each behavior 610 that has associated therewith the input type 216 of the received input 210. The behavior system 600 may include an input type filter 602 that receives the inputs 210 identifies which behaviors 610, if any, are associated with the input type 216 of the input 210 and increment the influence I of the affected behavior(s) 610.
In some implementations, each behavior 610 has an associated duration D. Receipt of an input 210 having a type 216 associated with the behavior 610 commences an input timer 614 set for a duration of time associated with the input 210 or the input type 216. When the input timer 614 expires, the behavior system 600 decrements the influence I of the behavior 610 (which was previously incremented for that input 210). Alternatively or additionally, the behavior system 600 may decrement the influence I of each behavior 610 every threshold period of time or since a last received input 210. When the influence I of a behavior 610 is zero, the behavior 610 changes state 612 from the active state to the inactive state. If the behavior system 600 receives an input 210 having an input type 216 associated with an inactive behavior 610, the behavior system 600 increments the influence I of that behavior 610, causing the behavior 610 to have an influence I greater than zero, which causes the behavior 610 to change state 612 from the inactive state to the active state. Once in the active state, the behavior 610 can participate in evaluating predicted outcomes O of activities A and/or the activities A themselves.
Behaviors 610 may evaluate activities A and/or predicted outcomes O of activities A. By evaluating both an activity A and the predicted outcomes O of the activity A, the behavior 610 offers a multi-pronged evaluation E. For example, while the behavior 610 may positively evaluate an activity A, it may negatively evaluate one or more of the predicted outcomes O of that activity A. As an illustrative example, if the behavior system 600 receives inputs 210 indicating that the user 10 is outdoors and on a street, then a sports behavior 610e may positively evaluate an activity A to ride a bicycle. If additional inputs 210 indicate that the user 10 is on a very busy street, then the sports behavior 610e may negatively evaluate a predicted outcome O of getting hit by a car.
In some implementations, a behavior 610 evaluates activities A and/or predicted outcomes O of activities A positively when the activity A has a type associated with the behavior 610, and negatively when the activity A has a type not associated with the behavior 610. The behavior system 600 may reference behavior-activity associations stored in non-transitory memory 130. The behavior-activity associations may have several nested layers (e.g., associations in a nested arrangement).
In some examples, an assistive behavior 610 is linked to an external resource and can manipulate, control, or at least bias the external resource based on the objective of the assistive behavior 610, a preference P set by the user 10, or one or more inputs 210. In some examples, the assistive behavior 610 becomes active after receipt of one or more inputs 210 having an input type 216 associated with the assistive behavior 610. While active, the assistive behavior 610 may cause, instruct, or influence an action of an external resource (e.g., other software or hardware directly or indirectly in communication with user device 200). For example, an assistive behavior 610 having an objective of accommodating the environmental comfort of the user 10 may become active after receiving a temperature input 210 from a temperature sensor 208h, a humidity input 210 from a humidity sensor 208i, or some other input related to the environmental comfort of the user 10. While active, the assistive behavior 610 may cause a thermostat near the user 10 to change temperature (e.g., to a preferred temperature, as set by the user 10 in a corresponding preferences P). Moreover, the assistive behavior 610 can be influenced by other behaviors 610 and/or a previously selected activity A. If a previously selected activity A entailed running, the assistive behavior 610 may adjust the thermostat to a post-running temperature cooler than a standard temperature, and then re-adjust the thermostat to the standard temperature after receiving a body temperature input indicating that the user 10 has cooled down to a normal body temperature. Assistive behaviors 610 may communicate with home automation systems, security systems, vehicle systems, networked devices, and other systems to adjust those systems to accommodate one or more preferences P of the user 10 and/or to facilitate participation in a suggested activity A. For example, if the search system 300 suggests a romantic evening with the spouse of the user 10, one or more assistive behaviors 610 (which may have scored the selected activity A favorably) may communicate with a home automation system of the user 10 to cause that system to dim the home lights, play romantic music (e.g., music have a category of romance), and set the indoor temperature to a temperature preferred by the spouse of the user 10.
Referring to
In some implementations, the activity selector 700 assesses the evaluations E, E1-En of the possible outcomes O, O1-On of the activities A, A1-Aj and determines a combination of activities A, A1-Aj that provides a combined outcome O. The combined outcome O may achieve higher user stratification than any single individual outcome O. The activity selector 700 may select the combination of activities A, A1-Aj having the determined combined outcome O as the selected activity set 720. For example, if the inputs 210 indicate that the user 10 is hungry and likely seeking entertainment, a combined outcome O of both eating and watching a show may be very favorable. Therefore, a combined action may be going to a dinner-theater event that includes eating and watching a show.
Referring again also to
In some implementations, the search application 206b, executing on the user device 200, generates a result view 800 based on the received search results 230 and displays the result view 800 in the GUI 250. The result view 800 includes one or more activity messages 810 corresponding to each result record 232 in the search results 230.
In additional examples, the search application 206b groups the search results 230 by activity type. When the GUI 250 allows the user 10 to select an activity type, the GUI 250 limits/filters the search results 230 to activities A having the selected activity type. For example, when the user 10 wishes to receive a suggestion for eating dinner, the user 10 may select an activity type of eating and the search application 206b (via the search system 300) suggests an activity A of eating at a nearby restaurant.
The search system 300 may autonomously generate and provide search results 230 to the user 10 based on one or more inputs 210. In such examples, the search system 300 may suggest information (activity A) relevant to the current state and context of the user 10. The suggested information may help the user 10 improve his/her current state. For example, if the search system 300 identifies that the user 10 is far from a scheduled appointment and traffic is heavy (e.g., based on inputs 210), the search system 300 may suggest that the user leave for the appointment early. Moreover, the search system 300 may suggest on-device features (software and/or hardware features) of the user device 200 or for application 206 executable on the user device 200 (or a web-based application accessible by the user device 200) that may be helpful to the user 10 at that moment. For example, the search system 300 may recommend an application 206 executable on the user device 200 relevant to the user 10 at that moment, based on one or more inputs 210 or the collective user state 420. Moreover, the recommended feature may be one of the inputs 210 or related to functionality of one of the inputs 210. For example, when the search system 300 recommends an outdoor activity A, the search system 300 may also provide information about a weather application 206 or an outdoor related application 206 installed on or executable by the user device 200.
In some implementations, the search system 300 provides a suggestion on demand. When the user 10 is seeking a particular type of suggestion, the user 10 may select a suggestion type to guide the selection of the suggestion by the search system 300. The suggestion type provides the search system 300 with a user intent.
Referring to
In some implementations, the result view 800, 800a includes an activity message 810 that includes the activity name 234a, the activity description 234b, the link 234c, the evaluation score 234e, and/or the popularity score 234f from the corresponding result record 232. The result view 800 may also include a result view selector 820 having icons 822a-n corresponding to alternative result views 800, 800a-n. When the user selects one of the icons 822a-n, the search application 206b displays the corresponding result view 800a-n.
In response to selection of a link 234c, the user device 200 may launch a corresponding software application 206 (e.g., a native application or a web-browser application) referenced by the link 234c and perform one or more operations indicated in the link 234c and/or the display data 234d. For example, the link 234c may include a URL having query string containing data to be passed to the software application 206 or software running on a remote server 112 (e.g., the query string may contain name/value pairs separated by delimiters, such as ampersands). If the link 234c is configured to access a native application 206, the link 234c may include a string (e.g., a query string) that includes a reference to the native application 206 and indicates one or more operations for the user device 200 to perform. When the user 10 selects the link 234c for the native application 206, the user device 200 launches the native application 206 referenced in the link 234c and performs the one or more operations indicated in the link 234c. If the references application 206 is not installed on the user device 200, the link 234c may direct the user 10 to a location (e.g., a digital distribution platform 130b) where a native application 206 can be downloaded. If the link 234c is configured to access a web-based application 206, the link 234c may include a string (e.g., a query string) that includes a reference to a web resource (e.g., a page of a web application/website). For example, the link 234c may include a URL (i.e., a web address) used with hypertext transfer protocol (HTTP). When the user 10 selects the link 234c, the user device 200 launches a web browser application 206 and retrieves the web resource indicated in the resource identifier.
The search application 206b may display the search results 230 to the user 10 in a variety of different ways, depending on what information is transmitted to the user device 200. Moreover, the search application 206b may display the search results 230 in the GUI 250 based on the display data 234d. The display data 234d may include text, images, layout information, a display template, a style guide (e.g., style sheet), etc.
In the example shown in
Referring to
In some implementations, the user 10 can enter feedback in the GUI 250 of the search application 206b, so that the search system 300 can learn whether the suggested activities A were well-received by the user 10. The search system 300 can use the user feedback for future activity selections.
In the example shown, the user 10a has the largest icon 910, 910a in the center portion of the screen 240, surrounded by other user icons 910, 910b-m. Each other user icon 910, 910b-m has a size similar to or smaller than the user icon and corresponds to another user 10b-m having a collective user state 420 having a degree of similarity to the collective user state 420 of the user 10a and/or located within some geographical distance of the user 10a. The other user icon 910, 910b-m may be arranged in groups 920 about the user icon 910a. Other users 10b-m having collective user states 420 satisfying a first threshold similarity to the collective user state 420 of the user 10a and/or located within a first threshold geographical distance of the user 10a have other user icons 910, 910b-i arranged in a first icon group 920a around the user icon 910a. While the first icon group 920a is shown as a circular arrangement around the user icon 910a, other arrangements are possible as well. Other users 10b-m having collective user states 420 satisfying a second threshold similarity less than the first threshold similarity to the collective user state 420 of the user 10a and/or located within a second threshold geographical distance of the user 10a further away than the first threshold geographical distance have corresponding other user icons 910, 910j-m arranged in a second icon group 920b around the user icon 910a and the first icon group 920a. In the example shown, the second icon group 920b has corresponding other user icons 910, 910j-m smaller than the other user icons 910, 910b-i of the first icon group 920a. The user 10a may scroll or otherwise navigate (e.g., in any direction on the screen) to view other user icons 910, 910b-m and their visual representation/arrangement indicating the relative similarity of the collective user state 420 of the other users 10b-m and/or the geographical proximity of the other users 10b-m.
In the example shown in
The home view 900a may visually distinguish between other users 10b-m having collective user states 420 that satisfy a threshold similarity to the collective user state 420 of the user 10a and other users 10b-m located within a threshold geographical distance of the user 10a. For example, other user icons 910b-m of other users 10b-m having collective user states 420 satisfying the threshold similarity to the collective user state 420 of the user 10a may have a first outline color (e.g., border color), whereas other user icons 910b-m of other users 10b-m located within a threshold geographical distance of the user 10a may have a second outline color different from the first outline color. Moreover, other users 10b-m satisfying the threshold similarity to the collective user state 420 of the user 10a and being located within the threshold geographical distance of the user 10a may have a third outline color different from the first and second outline colors.
Referring to
In some implementations, the user 10a may select another user icon 910b-m for enlargement to see additional information. For pressure sensitive screens 240, the user 10a may execute a long press, for example, causing the GUI 250 to display an enlarged other user icon 910b-m and/or other information about the corresponding other user 10b-m. In additional examples, selection of the other user icon 910b-m may open a separate window/view providing additional information about the corresponding other user 10b-m.
Referring to
Referring to
Referring to
The suggestion view 900f may include textual and/or graphical representation of the suggested activity A, an “accept” graphical input 944 allowing the user 10, 10a to accept the suggested activity A, a decline graphical input 946 allowing the user 10, 10a to decline the suggested activity A, a re-try graphical input 948 allowing the user 10, 10a to request another suggested activity A for the group of users 10, an information graphical input 950 allowing the user 10, 10a to view an activity information view 900g, as shown in
Referring to
Referring to
In the example shown in
In some implementations, the user 10 may view a real-time image/video (e.g., as a user icon 910) of another user 10 on the screen 240 of the user device 200 using the camera 208a. The search application 206b may augment the real-time image by overlaying graphics depicting the collective user state 420 and/or inputs 210 of the other user 10. In some examples, the overlain graphics include the collective user state icon 970 and/or the input icons 980, 980a-n. As such, the user 10, 10a may view another user 10b (e.g., image or video) augmented with overlain graphics (e.g., the collective user state icon 970 and/or the input icons 980, 980a-n) depicting the collective user state 420 of the other user 10b, allowing the user 10, 10a to know and understand the current state of being of the other user 10b without having to actually ask the other user 10b. By knowing more about the other user 10b, the user 10a can initiate a meaningful conversation with the other user 10b.
In some implementations, the method 1000 includes querying one or more remote data sources 130 in communication with the computing device 112, 202 to identify possible activities A, A1-Aj and/or predicted outcomes O, O1-On. The method 1000 may include determining, using the computing device 112, 202, the one or more possible activities A, A1-Aj and the one or more predicted outcomes O, O1-On for each activity A based on one or more preferences P1-Pn of the user 10. Each behavior 610 may evaluate an activity A or a corresponding outcome O positively when the activity A or the corresponding outcome O at least partially achieves an objective of the behavior 610. For example, the eating behavior 610a may positively evaluate an eating activity; whereas the sports behavior 610e may negatively evaluate the eating activity. Moreover, each behavior 610 may evaluate an activity A or a corresponding outcome O positively when the activity A or the corresponding outcome O at least partially achieves a user preference P1-Pn. In some examples, a first behavior 610 evaluates an activity A or a corresponding outcome O based on an evaluation E by a second behavior 610 of the activity A or the corresponding outcome O. This allows evaluations E of one behavior 610 to be based on evaluations E of another behavior 610. Each behavior 610 may elect to participate or not participate in evaluating the one or more activities A, A1-Aj and/or the one or more predicted outcomes O, O1-On for each activity A based on the collective user state 420.
When an input 210 is related to a behavior 610, the method 1000 may include incrementing an influence value I associated with the behavior 610. The input 210 may be related to the behavior 610 when the input 210 is of an input type associated with the behavior 610. In some implementations, the evaluations E of each behavior 610 can be weighted based on the influence value I of the corresponding behavior 610. The method 1000 may include decrementing the influence value I of each behavior 610 after a threshold period of time. When an influence value I equals zero, the method 1000 may include deactivating the corresponding behavior 610. Any behaviors 610 having an influence value I greater than zero may participate in evaluating the activities A or the corresponding outcomes O; and any behaviors 610 having an influence value I equal to zero may not participate in evaluating the activities A or the corresponding outcomes O.
In some implementations, the method 1000 includes selecting for the results 230 a threshold number of activities A having the highest evaluations E or a threshold number of activities A having corresponding predicted outcomes O that have the highest evaluations E. The method 1000 may include combining selected activities A and sending a combined activity A in the results 230.
The computing device 112, 202, may include a user computer processor 202 of a user device 200 including the screen 240 and/or one or more remote computer processors 112 in communication with the user computer processor 202. For example, the computer device can be the computer processor of a mobile device, a computer processor of an elastically scalable cloud resource, or a combination thereof.
Referring to
In some implementations, the method 1100 includes obtaining the user data 15 of the other users 10, 10b-m that have corresponding collective user states 420 satisfying a threshold similarity with the collective user state 420 of the user 10, 10a. The method 1100 may include arranging each other user glyph 910, 910b-m on the screen 240 based on a level of similarity between the collective user state 420 of the user 10, 10a and the collective user state 420 of the corresponding other user 10, 10b-m. In some examples, a size, a shape, a color, a border, and/or a position on the screen 240 of each other user glyph 910, 910b-m is based on a level of similarity between the collective user state 420 of the corresponding other user 10, 10b-m and the collective user state 420 of the user 10, 10a.
The method 1100 may include displaying a user glyph 910, 910a representing the user 10, 10a in a center portion of the screen 240 and the other user glyphs 910, 910b-m around the user glyph 910, 910a. The other user glyphs 910, 910b-m may be displayed in concentric groupings 920, 920a, 920b about the user glyph 910, 910a based on a level of similarity between the collective user states 420 of the corresponding other users 10, 10b-m and the collective user state 420 of the user 10, 10a.
In some implementations, the method 1100 includes receiving, at the data processing hardware 112, 202, an indication of a selection of one or more other user glyphs 910, 910b-m and executing, by the data processing hardware 112, 202, messaging (e.g., via the messaging view 900d) between the user 10, 10a and the one or more other users 10, 10b-m corresponding to the selected one or more other user glyphs 910, 910b-m. The method 1100 may include receiving a gesture across the screen 240, where the gesture indicates selection of the one or more other user glyphs 910, 910b-m. In some examples, the method 1100 includes receiving, at the data processing hardware 112, 202, an indication of a selection of a messenger glyph 920 displayed on the screen 240. The messenger glyph 920 has a reference to an application 206 executable on the data processing hardware 112, 202 and indicates one or more operations that cause the application 206 to enter an operating state that allows messaging between the user 10, 10a and the one or more other users 10, 10b-m corresponding to the selected one or more other user glyphs 910, 910b-m.
In some implementations, the method 1100 includes displaying a map 930 on the screen 240 and arranging the other user glyphs 910, 910b-m on the screen 240 based on geolocations of the corresponding other users 10, 10b-m. The user data 15 of each other user 10, 10b-m may include the geolocation of the corresponding other user 10, 10b-m. Moreover, the method 1100 may include displaying a user glyph 910, 910a representing the user 10, 10a on the map 930 based on a geolocation of the user 10, 10a.
The method 1100 may include receiving, at the data processing hardware 112, 202, an indication of a selection of one or more other user glyphs 910, 910b-m and determining, by the data processing hardware 112, 202, possible activities A for the user 10, 10a and the one or more other users 10, 10b-m corresponding to the selected one or more other user glyphs 910, 910b-m to perform based on the collective user states 420 of the user 10, 10a and the one or more other users 10, 10b-m. The method 1100 may also include executing, by the data processing hardware 112, 202, behaviors 610 having corresponding objectives. Each behavior 610 is configured to evaluate a possible activity A based on whether the possible activity A achieves the corresponding objective. The method 1100 includes selecting, by the data processing hardware 112, 202, one or more possible activities A based on evaluations E of one or more behaviors 610 and displaying, by the data processing hardware 112, 202, results 230 on the screen 240. The results 230 include the selected one or more possible activities A. In some examples, the method 1100 includes determining, by the data processing hardware 112, 202, one or more predicted outcomes O for each possible activity A based on the collective user states 420 of the user 10, 10a and the one or more other users 10, 10b-m. In such examples, each behavior 610 is configured to evaluate a possible activity A based on whether the possible activity A and the corresponding one or more predicted outcomes O of the possible activity A achieves the corresponding objective. In additional examples, the method 1100 may include receiving an indication of a gesture across the screen 240 indicating selection of the one or more other user glyphs 910, 910b-m.
In some implementations, at least one behavior 610 is configured to elect to participate or not participate in evaluating the possible activities A based on the received inputs 210. The method 1100 may include, for each behavior 610 determining whether any input 210 of the received inputs 210 is of an input type 216 associated with the behavior 610, and when an input 210 of the received inputs 210 is of an input type 216 associated with the behavior 610, incrementing an influence value I associated with the behavior 610. When the influence value I of the behavior 610 satisfies an influence value criterion, the behavior 610 participates in evaluating the possible activities A; and when the influence value I of the behavior 610 does not satisfy the influence value criterion, the behavior 610 does not participate in evaluating the possible activities A. In some examples, the method 1100 includes, for each behavior 610, determining whether a decrement criterion is satisfied for the behavior 610 and decrementing the influence value I of the behavior 610 when the decrement criterion is satisfied. The decrement criterion may be satisfied when a threshold period of time has passed since last incrementing the influence value I. In some examples, the evaluation E of at least one behavior 610 is weighted based on the corresponding influence value I of the at least one behavior 610. Moreover, the method 1100 may include determining the possible activities A based on one or more preferences P of the user 10. At least one behavior 610 may evaluate a possible activity A based on at least one of a history of selected activities A, 720 for the user 10 or one or more preferences P of the user 10. Furthermore, a first behavior 610, 610a may evaluate a possible activity A based on an evaluation E by a second behavior 610, 610b of the possible activity A.
In some implementations, the method 1100 includes receiving, at the data processing hardware 112, 202, a selection of a suggestion glyph 940 displayed on the screen 240 and, in response to the selection of the suggestion glyph 940, displaying, by the data processing hardware 112, 202, an activity type selector 942 on the screen 240. The method 1100 may further include receiving, at the data processing hardware 112, 202, a selection of an activity type and filtering, by the data processing hardware 112, 202, the results 230 based on the selected activity type.
Referring to
In some implementations, each other user 10, 10b-m is associated with the user 10, 10a based on a geographical proximity to the user 10, 10a, a linked relationship (e.g., family member, friend, co-worker, acquaintance, etc.). Other relationships are possible as well to narrow a pool of other users 10, 10b-m.
In some implementations, at least one behavior 610 is configured to elect to participate or not participate in evaluating the possible activity A based on the corresponding inputs 210 of the other user 10, 10b-m. The method 1200 may include, for each behavior 610 determining whether any input 210 of the other user 10, 10b-m is of an input type 216 associated with the behavior 610 and, when an input 210 of the other user is of an input type 216 associated with the behavior 610, incrementing an influence value I associated with the behavior 610. When the influence value I of the behavior 610 satisfies an influence value criterion, the behavior 610 participates in evaluating the possible activity A; and when the influence value I of the behavior 610 does not satisfy the influence value criterion, the behavior 610 does not participate in evaluating the possible activity A. The method 1200 may include, for each behavior 610, determining whether a decrement criterion is satisfied for the behavior 610 and decrementing the influence value I of the behavior 610 when the decrement criterion is satisfied. The decrement criterion may be satisfied when a threshold period of time has passed since last incrementing the influence value I.
In some examples, the evaluation E of at least one behavior 610 is weighted based on the corresponding influence value I of the at least one behavior 610. At least one behavior 610 may evaluate the possible activity A based on at least one of a history of positively evaluated activities A, 720 for the other user 10 or one or more preferences P of the other user 10. Moreover, a first behavior 610, 610a may evaluate the possible activity A based on an evaluation E by a second behavior 610, 610b of the possible activity A.
The method 1200 may include displaying, on a screen 240 in communication with the data processing hardware 112, 202, other user glyphs 910, 910b-m representing the selected other users 10, 10b-m. Each other user glyph 910, 910b-m: 1) at least partially indicates the collective user state 420 of the corresponding other user 10, 10b-m; and/or 2) is associated with a link to a displayable view 900j, 900k indicating the collective user state 420 of the corresponding other user 10, 10b-m and/or inputs 210 used to determine the collective user state 420 of the corresponding other user 10, 10b-m.
Referring to
The computing device 1400 includes a processor 1410, memory 1420, a storage device 1430, a high-speed interface/controller 1440 connecting to the memory 1420 and high-speed expansion ports 1450, and a low speed interface/controller 1460 connecting to low speed bus 1470 and storage device 1430. Each of the components 1410, 1420, 1430, 1440, 1450, and 1460, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor 1410 can process instructions for execution within the computing device 1400, including instructions stored in the memory 1420 or on the storage device 1430 to display graphical information for a graphical user interface (GUI) on an external input/output device, such as display 1480 coupled to high speed interface 1440. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices 1400 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).
The memory 1420 stores information non-transitorily within the computing device 1400. The memory 1420 may be a computer-readable medium, a volatile memory unit(s), or non-volatile memory unit(s). The non-transitory memory 1420 may be physical devices used to store programs (e.g., sequences of instructions) or data (e.g., program state information) on a temporary or permanent basis for use by the computing device 1400. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM)/programmable read-only memory (PROM)/erasable programmable read-only memory (EPROM)/electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware, such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM) as well as disks or tapes.
The storage device 1430 is capable of providing mass storage for the computing device 1400. In some implementations, the storage device 1430 is a computer-readable medium. In various different implementations, the storage device 1430 may be a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. In additional implementations, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory 1420, the storage device 1430, or memory on processor 1410.
The high speed controller 1440 manages bandwidth-intensive operations for the computing device 1400, while the low speed controller 1460 manages lower bandwidth-intensive operations. Such allocation of duties is exemplary only. In some implementations, the high-speed controller 1440 is coupled to the memory 1420, the display 1480 (e.g., through a graphics processor or accelerator), and to the high-speed expansion ports 1450, which may accept various expansion cards (not shown). In some implementations, the low-speed controller 1460 is coupled to the storage device 1430 and low-speed expansion port 1470. The low-speed expansion port 1470, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be coupled to one or more input/output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.
The computing device 1400 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server 1400a or multiple times in a group of such servers 1400a, as a laptop computer 1400b, or as part of a rack server system 1400c.
Various implementations of the systems and techniques described here can be realized in digital electronic and/or optical circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, non-transitory computer readable medium, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor.
Implementations of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Moreover, subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The terms “data processing apparatus”, “computing device” and “computing processor” encompass all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.
A computer program (also known as an application, program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio player, a Global Positioning System (GPS) receiver, to name just a few. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
To provide for interactivity with a user, one or more aspects of the disclosure can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen for displaying information to the user and optionally a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide interactivity with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.
One or more aspects of the disclosure can be implemented in a computing system that includes a backend component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a frontend component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), an inter-network (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).
The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some implementations, a server transmits data (e.g., an HTML page) to a client device (e.g., for purposes of displaying data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., a result of the user interactivity) can be received from the client device at the server.
While this specification contains many specifics, these should not be construed as limitations on the scope of the disclosure or of what may be claimed, but rather as descriptions of features specific to particular implementations of the disclosure. Certain features that are described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable sub-combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a sub-combination or variation of a sub-combination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multi-tasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, other implementations are within the scope of the following claims. For example, the activities recited in the claims can be performed in a different order and still achieve desirable results.
Claims
1. A method comprising:
- receiving, at data processing hardware, inputs indicative of a user state of a user, the received inputs comprising one or more of: sensor inputs from one or more sensors in communication with the data processing hardware; application inputs received from one or more software applications executing on the data processing hardware or a remote device in communication with the data processing hardware; or user inputs received from a graphical user interface;
- determining, by the data processing hardware, possible information to propose to the user based on the received inputs;
- executing, by the data processing hardware, behavior routines having corresponding objectives, each behavior routine configured to evaluate the possible information based on whether the possible information is related to or achieves the corresponding objective; and
- selecting, by the data processing hardware, suggested information from the possible information based on evaluations of one or more behavior routines for presentation to the user.
2. The method of claim 1, wherein each behavior routine elects to participate or not participate in evaluating the possible information based on the received inputs.
3. The method of claim 1, further comprising, for each behavior routine:
- determining whether any input of the received inputs is of an input type associated with the behavior routine; and
- when an input of the received inputs is of an input type associated with the behavior routine, incrementing an influence value associated with the behavior routine,
- wherein when the influence value of the behavior routine satisfies an influence value criteria, the behavior routine participates in evaluating the possible information, and when the influence value of the behavior routine does not satisfy the influence value criteria, the behavior routine does not participate in evaluating the possible information.
4. The method of claim 3, further comprising, for each behavior routine:
- determining whether a decrement criteria is satisfied for the behavior routine; and
- decrementing the influence value of the behavior routine when the decrement criteria is satisfied.
5. The method of claim 4, wherein the decrement criteria is satisfied when a threshold period of time has passed since lasting incrementing the influence value.
6. The method of claim 3, wherein the evaluation of at least one behavior routine is weighted based on the corresponding influence value of the at least one behavior routine.
7. The method of claim 1, further comprising displaying in the graphical user interface a representation of the user state based on the received inputs.
8. The method of claim 1, wherein at least one behavior routine evaluates the possible information based on at least one of a history of previously selected information for the user or one or more preferences of the user.
9. The method of claim 1, wherein a first behavior routine evaluates the possible information based on an evaluation by a second behavior routine of the possible information.
10. The method of claim 1, wherein the received inputs comprise biometric data of the user and environmental data regarding a surrounding of the user.
11. A method comprising:
- receiving, at data processing hardware, inputs indicative of a user state of a user, the received inputs comprising one or more of: sensor inputs from one or more sensors in communication with the data processing hardware; application inputs received from one or more software applications executing on the data processing hardware or a remote device in communication with the data processing hardware; or user inputs received from a graphical user interface;
- executing, by the data processing hardware, behavior routines having corresponding objectives, at least one behavior routine configured to control at least one system in communication with the data processing hardware, each behavior routine configured to evaluate possible instructions for the user or the at least one system based on whether the possible instructions are related to or achieves the corresponding objective;
- determining, by the data processing hardware, the possible instructions based on the received inputs;
- selecting, by the data processing hardware, suggested instructions from the possible instructions based on evaluations of one or more behavior routines; and
- issuing, by the data processing hardware, the suggested instructions.
12. The method of claim 11, wherein each behavior routine elects to participate or not participate in evaluating the possible instructions based on the received inputs.
13. The method of claim 11, further comprising, for each behavior routine:
- determining whether any input of the received inputs is of an input type associated with the behavior routine; and
- when an input of the received inputs is of an input type associated with the behavior routine, incrementing an influence value associated with the behavior routine,
- wherein when the influence value of the behavior routine satisfies an influence value criteria, the behavior routine participates in evaluating the possible instructions, and when the influence value of the behavior routine does not satisfy the influence value criteria, the behavior routine does not participate in evaluating the possible instructions.
14. The method of claim 13, further comprising, for each behavior routine:
- determining whether a decrement criteria is satisfied for the behavior routine; and
- decrementing the influence value of the behavior routine when the decrement criteria is satisfied.
15. The method of claim 14, wherein the decrement criteria is satisfied when a threshold period of time has passed since lasting incrementing the influence value.
16. The method of claim 13, wherein the evaluation of at least one behavior routine is weighted based on the corresponding influence value of the at least one behavior routine.
17. The method of claim 11, wherein the at least one system comprises at least one of a networked system, a home automation system, a security system, or a vehicle system.
18. The method of claim 11, wherein at least one behavior routine evaluates the possible instructions based on at least one of a history of previously selected instructions for the user or one or more preferences of the user.
19. The method of claim 11, wherein a first behavior routine evaluates the possible instructions based on an evaluation by a second behavior routine of the possible instructions.
20. The method of claim 11, wherein the received inputs comprise biometric data of the user and environmental data regarding a surrounding of the user.
Type: Application
Filed: Sep 29, 2016
Publication Date: Jan 19, 2017
Applicant: Blackwerks LLC (Glen Ellyn, IL)
Inventors: Brett A. Krueger (Rochester Hills, MI), Kraig T. Krueger (Glen Ellyn, IL)
Application Number: 15/280,960
