Abstract: Technology is described herein for facilitating a user's interaction with a digital ink document. The technology internally represents the ink document using a data structure having a hierarchy of nodes. The nodes describe respective elements in the ink document. The technology leverages the data structure to identify a set of nodes that grows upon the user's repeated selection of a particular part of the ink document. At each stage of the selection, the technology highlights a set of elements in the ink document that correspond to the current set of identified nodes. According to another illustrative aspect, the technology produces the data structure by modifying an original data structure provided by a text analysis engine. The technology performs this task with the objective of accommodating structured interaction by the user with the ink document.

Abstract: Technology is described herein for facilitating a user's interaction with a digital ink document. The technology internally represents the ink document using a data structure having a hierarchy of nodes. The nodes describe respective elements in the ink document. The technology leverages the data structure to identify a set of nodes that grows upon the user's repeated selection of a particular part of the ink document. At each stage of the selection, the technology highlights a set of elements in the ink document that correspond to the current set of identified nodes. According to another illustrative aspect, the technology produces the data structure by modifying an original data structure provided by a text analysis engine. The technology performs this task with the objective of accommodating structured interaction by the user with the ink document.

Abstract: Technology is described herein for facilitating a user's interaction with a digital ink document. The technology internally represents the ink document using a data structure having a hierarchy of nodes. The nodes describe respective elements in the ink document. The technology leverages the data structure to identify a set of nodes that grows upon the user's repeated selection of a particular part of the ink document. At each stage of the selection, the technology highlights a set of elements in the ink document that correspond to the current set of identified nodes. According to another illustrative aspect, the technology produces the data structure by modifying an original data structure provided by a text analysis engine. The technology performs this task with the objective of accommodating structured interaction by the user with the ink document.

Abstract: Intelligent assistant devices and methods for interacting with a user are disclosed. In some examples, a method for interacting with a user comprises predicting suggested action(s) for the user and displaying the action(s) via a display of the device. While the suggested action(s) are displayed, audio input comprising a command followed by a keyword is received from the user. The audio input is processed locally on the intelligent assistance device to (1) determine that the keyword follows the command and (2) recognize that the command applies to the suggested action(s). Based on determining that the keyword follows the command and recognizing that the command applies to the suggested action(s), a user selection of the suggested action(s) is established. Based on establishing the user selection, the one or more suggested actions are executed.

Abstract: Intelligent assistant systems, methods and computing devices are disclosed for resolving alias identifiers. A method comprises receiving and parsing data comprising a current user input that includes an alias identifier. The data and/or other sensor data are analyzed to identify the user. Based at least on identifying the user and recognizing the alias identifier, usage pattern data comprising at least one previous user input that includes the alias identifier and corresponding context information is accessed. The usage pattern data is used to resolve the alias identifier to mean the alias identifier in an alias record of a known entity. Based at least on resolving the alias identifier, an output device is controlled to one or more of generate a message and perform an action with respect to the known entity.

Abstract: To address the issues of handling conversations with multiple users, an intelligent digital assistant system is provided. The system may include at least one microphone configured to receive an audio input, a speaker configured to emit an audio output, and a processor. The processor may be configured engage in a conversation with a first user, and, concurrent with the first user being engaged in the conversation with the system, recognize speech of one or more additional users in the audio input. The processor may process the recognized speech of the one or more additional users to determine a context for each additional user, and execute a conversation disentanglement module to select and perform one or more predetermined conversation disentanglement actions according to the context of the recognized speech of each additional user.

Abstract: Intelligent assistant systems, methods and computing devices are disclosed for training a machine learning-based parser to derive user intents. A method comprises analyzing with a feeder parser a surface form of a user input. A user intent underlying the surface form is derived by the feeder parser. The surface form and the user intent are provided to a machine learning-based parser and used to enhance a training set of the machine learning-based parser.

Abstract: Sensor data is provided to a deep neural network previously trained to detect a feature within the physical environment. Result signals are received from the neural network, and the computing system determines if the feature is present within the physical environment based on the result signals. Responsive to determining that the feature is present, the computing system implements a function of a rule assigned to the feature. Responsive to determining that the feature is not present, the computing system determines whether one or more activation parameters of the neural network have been met indicative of an alternative feature being present within the physical environment. An indication that the activation parameters have been met is output by the computing system, enabling the rule to be extended to the alternative feature.

Abstract: Sensor data is provided to a deep neural network previously trained to detect a feature within the physical environment. Result signals are received from the neural network, and the computing system determines if the feature is present within the physical environment based on the result signals. Responsive to determining that the feature is present, the computing system implements a function of a rule assigned to the feature. Responsive to determining that the feature is not present, the computing system determines whether one or more activation parameters of the neural network have been met indicative of an alternative feature being present within the physical environment. An indication that the activation parameters have been met is output by the computing system, enabling the rule to be extended to the alternative feature.

Abstract: A method to execute computer-actionable directives conveyed in human speech comprises: receiving audio data recording speech from one or more speakers; converting the audio data into a linguistic representation of the recorded speech; detecting a target corresponding to the linguistic representation; committing to the data structure language data associated with the detected target and based on the linguistic representation; parsing the data structure to identify one or more of the computer-actionable directives; and submitting the one or more of the computer-actionable directives to the computer for processing.

Abstract: Intelligent assistant devices and methods for interacting with a user are disclosed. In some examples, a method for interacting with a user comprises predicting suggested action(s) for the user and displaying the action(s) via a display of the device. While the suggested action(s) are displayed, audio input comprising a command followed by a keyword is received from the user. The audio input is processed locally on the intelligent assistance device to (1) determine that the keyword follows the command and (2) recognize that the command applies to the suggested action(s). Based on determining that the keyword follows the command and recognizing that the command applies to the suggested action(s), a user selection of the suggested action(s) is established. Based on establishing the user selection, the one or more suggested actions are executed.

Abstract: To address the issues of handling conversations with multiple users, an intelligent digital assistant system is provided. The system may include at least one microphone configured to receive an audio input, a speaker configured to emit an audio output, and a processor. The processor may be configured engage in a conversation with a first user, and, concurrent with the first user being engaged in the conversation with the system, recognize speech of one or more additional users in the audio input. The processor may process the recognized speech of the one or more additional users to determine a context for each additional user, and execute a conversation disentanglement module to select and perform one or more predetermined conversation disentanglement actions according to the context of the recognized speech of each additional user.

Abstract: Intelligent assistant systems, methods and computing devices are disclosed for resolving alias identifiers. A method comprises receiving and parsing data comprising a current user input that includes an alias identifier. The data and/or other sensor data are analyzed to identify the user. Based at least on identifying the user and recognizing the alias identifier, usage pattern data comprising at least one previous user input that includes the alias identifier and corresponding context information is accessed. The usage pattern data is used to resolve the alias identifier to mean the alias identifier in an alias record of a known entity. Based at least on resolving the alias identifier, an output device is controlled to one or more of generate a message and perform an action with respect to the known entity.

Abstract: Intelligent assistant systems, methods and computing devices are disclosed for training a machine learning-based parser to derive user intents. A method comprises analyzing with a feeder parser a surface form of a user input. A user intent underlying the surface form is derived by the feeder parser. The surface form and the user intent are provided to a machine learning-based parser and used to enhance a training set of the machine learning-based parser.

Abstract: A method for managing simultaneous display of multiple windows in a graphical user interface (GUI). A computer system with a display and user interface input device displays a primary or container window on the display, comprising a predetermined region for displaying information and/or receiving user commands associated with a first computer process. The system displays a target window comprising a predetermined region for displaying information and/or receiving user commands associated with a different computer process. The target window is typically displayed in a higher z-order relative to the primary window to maintain it on top. In response to a predetermined triggering condition, the target window is manipulated by movement, temporary disappearance, and/or size reduction, while maintaining the relative z-order of the target window relative to the primary window and away from a predetermined avoidance region associated with the first computer process.