Abstract: One disclosed example provides a computing system configured to receive input defining a machine conversation dialog flow, display in an editing user interface a first representation of the machine conversation dialog flow in the form of a symbolic representation, receive input requesting display of a second representation of the machine conversation dialog flow, and in response to the request display in the editing user interface the machine conversation dialog flow in the character-based representation. The computing system is further configured to, based upon the machine conversation dialog flow, update a machine conversation schema template to form an updated machine conversation schema, and form an agent definition file based upon the updated machine conversation schema for use in executing the machine conversation dialog flow.

Abstract: A computing device is provided, which may include a display, an input device and a processor configured to execute an application program including an application user interface presented via the display, the application user interface including a session state of a current session of a user, and execute a bot client program configured to execute a dialog with a user, the bot client program including a conversation canvas presented via the display, wherein the bot client program is configured to receive a query in the dialog from the user via the input device and conversation canvas, determine that the query is directed to content related to the state of the application program, send a context request to the application program, receive context data from the application program, the context data being derived from the state of the application program, and determine a response to the query.

Abstract: A server is provided, including a processor configured to execute a bot server program. The bot server program may receive from a computing device an input with an input type that includes one or more of speech and text. The bot server program may programmatically generate an output, wherein the output is generated based on the input. The bot server program may detect one or more output types capable of being output by the computing device and select an output type from a plurality of output types that may include speech and text. The selected output type may be an output type capable of being output by the computing device. The bot server program may modify the programmatically generated output to produce a modified output with the selected output type, and may convey the modified output to the computing device for output on a display and/or speaker.

Abstract: Examples are disclosed that relate to a conversation runtime for automating transitions of conversational user interfaces. One example provides a computing system comprising a logic subsystem and a data-holding subsystem. The data-holding subsystem comprises instructions executable by the logic subsystem to execute a conversation runtime configured to receive one or more agent definitions for a conversation robot program, each agent definition defining a state machine including a plurality of states, detect a conversation trigger condition, select an agent definition for a conversation based on the conversation trigger condition, and execute a conversation dialog with a client computing system using the agent definition selected for the conversation and automatically transition the state machine between different states of the plurality of states during execution of the conversation dialog.

Abstract: Techniques are described for headlessly completing a task of an application in the background of a digital personal assistant. For example, a method can include receiving a voice input via a microphone. Natural language processing can be performed using the voice input to determine a user voice command. The user voice command can include a request to perform a task of the application. The application can be caused to execute the task as a background process without a user interface of the application appearing. A user interface of the digital personal assistant can provide a response to the user, based on a received state associated with the task, so that the response comes from within a context of the user interface of the digital personal assistant without surfacing the user interface of the application.

Abstract: Computer systems, methods, and storage media for changing the state of an application by detecting neurological user intent data associated with a particular operation of a particular application state, and changing the application state so as to enable execution of the particular operation as intended by the user. The application state is automatically changed to align with the intended operation, as determined by received neurological user intent data, so that the intended operation is performed. Some embodiments relate to a computer system creating or updating a state machine, through a training process, to change the state of an application according to detected neurological data.

Abstract: Techniques and systems for communicating information via a computer-implemented agent are described. A computing device may obtain sensor data of an individual, such as visual data, audible data, physiological data, or combinations thereof. An emotional state of the individual may be determined based on the sensor data. A communications framework may be identified based on the emotional state of the individual. The communications framework may indicate a manner in which the computer-implemented agent communicates information to the individual. For example, the communications framework may specify voice features, facial features, body language, positioning in the environment, or combinations thereof, that may be utilized to produce a representation of a computer-implemented agent that communicates information to the individual. In some cases, the individual may provide feedback indicating a preference to have the computer-implemented agent communicate information in a different manner.

Abstract: Computer systems, methods, and storage media for tailoring a user interface to a user according to a determined user state and a determined interface context corresponding to the determined user state. The user interface is tailored by modifying the format of at least a portion of the interface, including modifying the content, layout of the content, presentation sequence, or visual display of the interface. A user interface includes a selectable formatting object for controlling the formatting of the user interface and for generating feedback data for training an ensemble learning component to enable more effective predictive formatting changes.

Abstract: Computer systems, methods, and storage media for generating a continuous motion control using neurological data and for associating the continuous motion control with a continuous user interface control to enable analog control of the user interface control. The user interface control is modulated through a user's physical movements within a continuous range of motion associated with the continuous motion control. The continuous motion control enables fine-tuned and continuous control of the corresponding user interface control as opposed to control limited to a small number of discrete settings.

Abstract: Computer systems, methods, and storage media for changing the state of an application by detecting neurological user intent data associated with a particular operation of a particular application state, and changing the application state so as to enable execution of the particular operation as intended by the user. The application state is automatically changed to align with the intended operation, as determined by received neurological user intent data, so that the intended operation is performed. Some embodiments relate to a computer system creating or updating a state machine, through a training process, to change the state of an application according to detected neurological data.

Abstract: A computing platform for presenting contextual content based on detected user confusion is described. In at least one example, sensor data can be received from at least one sensor. The sensor data can be associated with measurements of at least one physiological attribute of a user. Based at least in part on the sensor data, an occurrence of an event corresponding to a confused mental state of the user can be determined. In at least one example, contextual data associated with the event can be determined. The contextual data can identify at least an application being executed at a time corresponding to the occurrence of the event. The contextual data can be leveraged to access content data for mitigating the confused mental state of the user and the content data can be presented via an output interface associated with a device corresponding to the user.

Abstract: Techniques are described for headlessly completing a task of an application in the background of a digital personal assistant. For example, a method can include receiving a voice input via a microphone. Natural language processing can be performed using the voice input to determine a user voice command. The user voice command can include a request to perform a task of the application. The application can be caused to execute the task as a background process without a user interface of the application appearing. A user interface of the digital personal assistant can provide a response to the user, based on a received state associated with the task, so that the response comes from within a context of the user interface of the digital personal assistant without surfacing the user interface of the application.