Abstract: Systems and methods are described in which the control of virtual activities is cooperatively shared among two or more interactive users. Within selected virtual actions, an initiation component is identified that, when enacted, automatically informs one or more shared-experience users of the intent by the initiator to complete the virtual action and triggers monitoring of the shared-experience users for indications signaling agreement to proceed. When the degree of agreement meets a predetermined threshold or is judged by the initiator to be adequate, the completion component of the virtual actions is enacted on all devices. Automatic incorporation of initiation and completion components during shared activities may allow for cooperative control of virtual activities without repeated and potentially monotonous questions and responses regarding when to proceed.

Abstract: Systems and methods are described in which the location of a tap on the body of a handheld device is determined in real time using data streams from an embedded inertial measurement unit (IMU). Taps may be generated by striking the handheld device with an object (e.g., a finger), or by moving the handheld device in a manner that causes it to strike another object. IMU accelerometer, gyroscopic and/or orientation (relative to the magnetic and/or gravitational pull of the earth) measurements are examined for signatures that distinguish a tap at a location on the body of the device compared with signal characteristics produced by taps at other locations. Neural network and/or numerical methods may be used to perform such classifications.

Abstract: Systems and methods are described to enhance interactive engagement during simultaneous delivery of serial or digital content (e.g., audio, video) to a plurality of users. A machine-based awareness of the context of the content and/or one or more user reactions to the presentation of the content may be used as a basis to interrupt content delivery in order to intersperse a snippet that includes a virtual agent with an awareness of the context(s) of the content and/or the one or more user reactions. This “contextual virtual agent” (CVA) enacts actions and/or dialog based on the one or more machine-classified contexts coupled with identified interests and/or aspirations of individuals within the group of users. The CVA may also base its activities on a machine-based awareness of “future” content that has not yet been delivered to the group, but classified by natural language and/or computer vision processing.

Abstract: Systems and methods are described to enact machine-based, collective control by two individuals of one or more displayed virtual objects. Collective interactions may be implemented by combining an ability to specify one or more locations on a touch-sensitive display using one or more digits of a first user with an ability to monitor a portable, handheld controller manipulated by the other user. Alternatively or in addition, pointing by the first hand to the one or more locations on a display may be enhanced by a stylus or other pointing device. The handheld controller may be tracked within camera-acquired images by following camera-trackable controller components and/or by acquiring measurements from one or more embedded internal measurement units (IMUs). Optionally, one or more switches or sensors may be included within the handheld controller, operable by one or more digits of the second hand to enable alternative virtual object display and/or menu selections during collective interactions.

Abstract: Systems and methods are described to enact machine-based, simultaneous classification of emotional and cognitive states of an individual in substantially real time. A bootstrap approach is used to identify indicators of emotional state, including boredom, engagement and feeling overwhelmed, by finding additional interactions temporally adjacent to known indicators. A similar bootstrap approach is used to identify new indicators of knowledge acquisition (or confusion), by locating frequent temporally adjacent interactions. Systems and methods may enhance teaching by identifying knowledge presentations that result in successful knowledge acquisition, and learning by allowing instructors associated with an individual to track materials that have been understood (or misunderstood). Machine-based tracking of learning may circumvent boredom by avoiding needless repetition, and facilitate teaching of synergistic topics related to materials recently understood.

Abstract: Systems and methods are described to enhance interactive engagement during simultaneous delivery of serial or digital content (e.g., audio, video) to a plurality of users. A machine-based awareness of the context of the content and/or one or more user reactions to the presentation of the content may be used as a basis to interrupt content delivery in order to intersperse a snippet that includes a virtual agent with an awareness of the context(s) of the content and/or the one or more user reactions. This “contextual virtual agent” (CVA) enacts actions and/or dialog based on the one or more machine-classified contexts coupled with identified interests and/or aspirations of individuals within the group of users. The CVA may also base its activities on a machine-based awareness of “future” content that has not yet been delivered to the group, but classified by natural language and/or computer vision processing.

Abstract: Systems and methods to enact machine-based, substantially simultaneous, two-handed interactions with one or more displayed virtual objects. Bimanual interactions may be implemented by combining an ability to specify one or more locations on a touch-sensitive display using one or more digits of a first hand with an ability to monitor a portable, handheld controller manipulated by the other hand. Alternatively or in addition, pointing by the first hand to the one or more locations on a display may be enhanced by a stylus or other pointing device. The handheld controller may be tracked within camera-acquired images by following camera-trackable controller components and/or by acquiring measurements from one or more embedded internal measurement units (IMUs). Optionally, one or more switches or sensors may be included within the handheld controller, operable by one or more digits of the second hand to enable alternative virtual object display and/or menu selections during bimanual interactions.

Abstract: Systems and methods are described for time-shifting interactions by sharing an artificial intelligence personality (AIP). An AIP is an understanding construct that may control a variety of communication experiences to support a sense of ongoing social connectedness. An AIP may be instantiated within one or more HIEs that interact with humans in a human, cartoon or pet-like manner. HIEs may include robots, robotic pets, toys, simple-to-use devices, and graphical user interfaces. The AIP may be periodically updated based on human interactions sensed by the HIEs as well as knowledge of historical and ongoing events. The systems may provide users with intuitive machine companions that exhibit an expert knowledge base and a familiar, cumulative personality. HIEs may continue to operate without interruption in the presence of telecommunications delays or interruptions, and/or the absence of one or more human participants; allowing participants to “time-shift” their sense of connectedness.

Abstract: Systems and methods are described for time-sharing interactions using a shared artificial intelligence personality (AIP) incorporated within multiple human interaction entities (HIEs). An AIP is an understanding construct that may control a variety of communication experiences to support a sense of ongoing social connectedness. An AIP may be instantiated within two or more HIEs that interact with humans in a human, cartoon or pet-like manner. HIEs may include robots, robotic pets, toys, simple-to-use devices, and graphical user interfaces. The AIP may be periodically updated based on human interactions sensed by the HIEs as well as knowledge of historical and ongoing events. The systems may provide two or more users with intuitive machine companions that exhibit an expert knowledge base and a familiar, cumulative personality.

Abstract: Systems and methods are described for sharing an artificial intelligence personality (AIP) among multiple human interaction entities (HIEs). An AIP is an understanding construct that interacts with one or more humans in a human- or pet-like manner, implementing a variety of communication experiences to support a sense of ongoing social connectedness. HIEs may include robots, robotic pets, toys, and avatars. The system may be implemented using two or more HIEs and, optionally, one or more remote and/or distributed processors to compute AIPs and sequence telecommunications. AIPs are periodically updated primarily based on human interactions sensed by the two or more HIEs. HIEs may continue to operate without interruption in the presence of significant telecommunications delays. The systems may provide two or more users with intuitive machine companions that exhibit an integrated knowledge base and personality cumulatively acquired from all users.