Abstract: A method comprises determining a threshold based on a shape or size of a graphical user interface (GUI); selecting one or more sequences of digital actions (actions) as one or more compound actions from a list of sequences of actions, the one sequence of actions being performed by multiple computer devices of a plurality of computer devices, including at least one input device and at least one output device, the one or more sequences of actions having a total number of sequences that is no more than the threshold and satisfying a set of criteria related to a number of actions in each sequence of actions; creating a deep link as a shortcut representing a specific compound action, the shortcut when invoked causing performing the specific compound action and producing a specific result; causing presenting the shortcut as an UI element in the GUI.

Abstract: A data management server computer (“server”) is disclosed. The server receives action data associated with one or more users identifying a plurality of actions performed through one or more computer devices in a multi-modal networked environment, identifies useful sequences of actions as compound actions, and generates deep links representing the compound actions as UI elements which when invoked cause execution of the represented compound actions.

Abstract: A component management server computer (“server”) and processing methods are disclosed. In some embodiments, the server is programmed to continuously receive input data regarding what is happening in the physical room from one or more input devices. The server is programmed to then detect an utterance of a spoken word from the input data and generate one or more sound metrics based on the input data. Based on the sound metrics as applied to certain criteria, the server is programmed to activate a component, such as an input device, variable, software system, or output device, and cause one or more output devices to execute an action that alerts a user of the activated component. The server can also be programmed to turn on, off, up, or down any of the components based on the activated component.

Abstract: A data management server computer (“server”) is disclosed. The server receives action data associated with one or more users identifying a plurality of actions performed through one or more computer devices in a multi-modal networked environment, identifies useful sequences of actions as compound actions, and generates deep links representing the compound actions as UI elements which when invoked cause execution of the represented compound actions.

Abstract: A device management server computer (“server”) and processing methods are disclosed. In some embodiments, the server is programmed to manage a plurality of input devices and output devices in a physical room. The server is programmed to analyze media data capturing actions performed by a user in real time as a participant in the physical room, determine how the user would like to connect at least one of the input devices and one of the output devices from the analysis, and enable the connection. The sever is programmed to interpret the actions and derive commands for connecting two or more devices based on predetermined data regarding the input devices and output devices and rules for referring to and connecting these devices.

Abstract: Systems, methods, and computer program products to perform an operation comprising receiving text input via a chat interface of a tutor application, identifying, by at least one classifier applied to the text input, a concept in the text input, mapping the concept in the text input to at least one of a visual action and a first visual object, generating, based on a first machine learning (ML) model, a first program code statement corresponding to the at least one of the visual action and the first visual object, and executing the first program code statement to modify a visualization interface of the tutor application based on the text input received via the chat interface.

Abstract: A method and system of providing education content to a student is provided. A topic to be taught to a student is received. A user model of the student is identified. An electronic content model for the topic is determined. One or more presentation templates are identified based on the electronic content model. A concept map is determined based on the one or more presentation templates. A presentation template is selected from the one or more presentation templates. A segment of an educational content of the topic is provided to a user device of the student, based on the selected presentation template.

Abstract: Method and apparatus for generating natural language observations using sequence-to-sequence models are provided. The method includes receiving a first electronic document and dynamically generating, without requiring user intervention, a first natural language observation corresponding to a first portion of the first electronic document by processing the first portion of the first electronic document using a first sequence-to-sequence model. A second natural language observation is received for the first portion of the first electronic document, and the generated first natural language observation and the authoritative natural language observation are provided. The method further includes receiving an evaluation of the first natural language observation and the second natural language observation, and refining the first sequence-to-sequence model based on the evaluation of the first natural language observation and second natural language observation.

Abstract: A device management server computer (“server”) and processing methods are disclosed. In some embodiments, the server is programmed to manage a plurality of input devices and output devices in a physical room. The server is programmed to analyze media data capturing actions performed by a user in real time as a participant in the physical room, determine how the user would like to connect at least one of the input devices and one of the output devices from the analysis, and enable the connection. The sever is programmed to interpret the actions and derive commands for connecting two or more devices based on predetermined data regarding the input devices and output devices and rules for referring to and connecting these devices.

Abstract: A device management server computer (“server”) is programmed to manage a plurality of input devices and output devices in a physical room. The server is programmed to analyze media data capturing actions performed by a user in real time as a participant in the physical room, determine how the user would like to connect at least one of the input devices and one of the output devices from the analysis, and enable the connection. The sever is programmed to interpret the actions and derive commands for connecting two or more devices based on predetermined data regarding the input devices and output devices and rules for referring to and connecting these devices.

Abstract: A device management server computer (“server”) is programmed to manage a plurality of input devices and output devices in a physical room. The server is programmed to analyze media data capturing actions performed by a user in real time as a participant in the physical room, determine how the user would like to connect at least one of the input devices and one of the output devices from the analysis, and enable the connection. The sever is programmed to interpret the actions and derive commands for connecting two or more devices based on predetermined data regarding the input devices and output devices and rules for referring to and connecting these devices.

Abstract: Techniques for conditional presentation of auxiliary information are provided. In one example, a computer-implemented method comprises determining, by a device operatively coupled to a processor, feedback information for a user in response to a media presentation, wherein the feedback information is indicative of a mental state of the user during presentation of the media presentation. The computer-implemented method can further comprise determining content of the media presentation associated with the mental state of the user, and determining auxiliary information about the content based on the mental state of the user. The computer-implemented method can further comprise providing, by the device, the content to a user device during the presentation of the media presentation and for rendering at the user device in association with presentation of the content.

Abstract: A room and activity management server computer (“server”) and processing methods are disclosed. In some embodiments, the server is programmed to manage multi-role activities collaboratively performed by multiple participants in a physical room with multiple media communications. For each activity, the server is configured to assign roles to participants and enforce rules that govern how the participants in given roles interact with one another or engage with the room at given times. In enforcing the rules, the server is programmed to improve such interaction and engagement through multimedia communications.

Abstract: A method and system of providing education content to a student is provided. A topic to be taught to a student is received. A user model of the student is identified. An electronic content model for the topic is determined. One or more presentation templates are identified based on the electronic content model. A concept map is determined based on the one or more presentation templates. A presentation template is selected from the one or more presentation templates. A segment of an educational content of the topic is provided to a user device of the student, based on the selected presentation template.

Abstract: A method and system of providing education content to a student is provided. A topic to be taught to a student is received. A user model of the student is identified. An electronic content model for the topic is determined. One or more presentation templates are identified based on the electronic content model. A concept map is determined based on the one or more presentation templates. A presentation template is selected from the one or more presentation templates. A segment of an educational content of the topic is provided to a user device of the student, based on the selected presentation template.

Abstract: Techniques for conditional presentation of auxiliary information are provided. In one example, a computer-implemented method comprises determining, by a device operatively coupled to a processor, feedback information for a user in response to a media presentation, wherein the feedback information is indicative of a mental state of the user during presentation of the media presentation. The computer-implemented method can further comprise determining content of the media presentation associated with the mental state of the user, and determining auxiliary information about the content based on the mental state of the user. The computer-implemented method can further comprise providing, by the device, the content to a user device during the presentation of the media presentation and for rendering at the user device in association with presentation of the content.

Abstract: Techniques for generating auxiliary information for a media presentation are provided. In one example, a computer-implemented method comprises receiving, by a device operatively coupled to a processor, first feedback information for a group of users regarding mental states of the users during a media presentation. The computer-implemented method can further comprise determining, by the device, first parts of the media presentation considered confusing to at least some of the users and second parts of the media presentation considered interesting to at least some of the users based on the first feedback information, and generating, by the device, an index data structure comprising first information identifying the first parts and classifying the first parts as confusing and second information identifying the second parts and classifying the second parts as interesting.

Abstract: Techniques for generating auxiliary information for a media presentation are provided. In one example, a computer-implemented method comprises receiving, by a device operatively coupled to a processor, first feedback information for a group of users regarding mental states of the users during a media presentation. The computer-implemented method can further comprise determining, by the device, first parts of the media presentation considered confusing to at least some of the users and second parts of the media presentation considered interesting to at least some of the users based on the first feedback information, and generating, by the device, an index data structure comprising first information identifying the first parts and classifying the first parts as confusing and second information identifying the second parts and classifying the second parts as interesting.

Abstract: Techniques for generating auxiliary information for a media presentation are provided. In one example, a computer-implemented method comprises receiving, by a device operatively coupled to a processor, first feedback information for a group of users regarding mental states of the users during a media presentation. The computer-implemented method can further comprise determining, by the device, first parts of the media presentation considered confusing to at least some of the users and second parts of the media presentation considered interesting to at least some of the users based on the first feedback information, and generating, by the device, an index data structure comprising first information identifying the first parts and classifying the first parts as confusing and second information identifying the second parts and classifying the second parts as interesting.

Abstract: Techniques for conditional presentation of auxiliary information are provided. In one example, a computer-implemented method comprises determining, by a device operatively coupled to a processor, feedback information for a user in response to a media presentation, wherein the feedback information is indicative of a mental state of the user during presentation of the media presentation. The computer-implemented method can further comprise determining content of the media presentation associated with the mental state of the user, and determining auxiliary information about the content based on the mental state of the user. The computer-implemented method can further comprise providing, by the device, the content to a user device during the presentation of the media presentation and for rendering at the user device in association with presentation of the content.