Abstract: A foot operated controller apparatus is disclosed. The apparatus includes a ball large enough for a human to manipulate with two feet, and a support apparatus configured to support the ball from below, restrict translation of the ball and permit rotation of the ball about its center with respect to at least two axes. One or more position encoders are disposed proximate a surface of the ball. The encoder(s) are configured to determine a rotational displacement of the ball with respect to two or more axes.

Abstract: A method, system and computer product for training a control input system involve taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value and generating a subsequent output value using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold. Surface damping interactions with at least a simulated environment, a rigid body position and a position of the one or more movable joints based on an integral of the subsequent output value are simulated. The Motion Decision Neural Network is trained with the machine learning algorithm based upon at least a result of the simulation of the simulated environment and position of the one or more movable joints.

Abstract: Respective pose and motion sensors are engaged with the left and right feet of a sitting person and signals from the sensors are used to control the motion of a VR object presented on, e.g., a head-mounted display worn by the person. The sensor signals can be mapped to translational, rotational, and elevational motion of the VR object, and/or can be used to select controls to send controller-like signals to the game engine to control the VR object.

Abstract: An event driven sensor (EDS) is used for simultaneous localization and mapping (SLAM) and in particular is used in conjunction with a constellation of light emitting diodes (LED) to simultaneously localize all LEDs and track EDS pose in space. The EDS may be stationary or moveable and can track moveable LED constellations as rigid bodies. Each individual LED is distinguished at a high rate using minimal computational resources (no image processing). Thus, instead of a camera and image processing, rapidly pulsing LEDs detected by the EDS are used for feature points such that EDS events are related to only one LED at a time.

Abstract: An autonomous personal companion utilizing a method of object identification that relies on a hierarchy of object classifiers for categorizing one or more objects in a scene. The classifier hierarchy is composed of a set of root classifiers trained to recognize objects based on separate generic classes. Each root acts as the parent of a tree of child nodes, where each child node contains a more specific variant of its parent object classifier. The method covers walking the tree in order to classify an object based on more and more specific object features. The system is further comprised of an algorithm designed to minimize the number of object comparisons while allowing the system to concurrently categorize multiple objects in a scene.

Abstract: Machine learning is used to transform screen renders from the viewpoint of the player's character to be from the viewpoint of other non-player characters (NPC) in the room. One or more neural networks are trained using game images captured during a human play session, and then subsequently the neural networks are used to create realistic video from the NPC viewpoints. To avoid rendering multiple viewpoints simultaneously, a single viewpoint is rendered, and neural networks are used to transform it for the viewpoints of other NPCs in the area. A group of NPCs may be treated as a batch and a single viewpoint transformed to multiple viewpoints in a single inference pass. For cloud gaming the game video can be rendered/sent immediately for the player but the actions of the NPCs are delayed for a frame while the neural network generates the behavior.

Abstract: Computer animation involving monocular pose prediction is disclosed. A plurality of candidate pose sequences of a three-dimensional model of an animation character is generated such that each candidate pose of each sequence has a segmentation map that matches a segmentation map of a corresponding character derived from a corresponding frame of a video. A distance between candidate poses at each time step is maximized. An optimum pose sequence is determined and used to generate a corresponding sequence of frames of animation.

Abstract: A foot operated controller apparatus is disclosed. The apparatus includes a ball large enough for a human to manipulate with two feet, and a support apparatus configured to support the ball from below, restrict translation of the ball and permit rotation of the ball about its center with respect to at least two axes. One or more position encoders are disposed proximate a surface of the ball. The encoder(s) are configured to determine a rotational displacement of the ball with respect to two or more axes.

Abstract: A controller apparatus includes two or more balls having a size sufficient for a human to manipulate with a single foot. Three or more bearings support at least one of the balls from below and permit rotation with respect to at least two axes. One or more position encoders are disposed proximate a surface of each of the two or more balls. The encoders are configured to determine a rotational displacement of each of the two or more balls with respect to two or more axes.

Abstract: An event driven sensor (EDS) is used for simultaneous localization and mapping (SLAM) and in particular is used in conjunction with a constellation of light emitting diodes (LED) to simultaneously localize all LEDs and track EDS pose in space. The EDS may be stationary or moveable and can track moveable LED constellations as rigid bodies. Each individual LED is distinguished at a high rate using minimal computational resources (no image processing). Thus, instead of a camera and image processing, rapidly pulsing LEDs detected by the EDS are used for feature points such that EDS events are related to only one LED at a time.

Abstract: Computer animation involving pose disambiguation is disclosed. Two or more source segmentation masks are generated from corresponding contemporaneous video images of a character from different points of view. A three-dimensional model of an animation character corresponding to the character in the two or more contemporaneous video images is generated. Two or more different target segmentation masks corresponding to different views of the animation character corresponding to the character in the two or more video images. Each target segmentation mask is compared to a corresponding source segmentation mask from the comparison it is determined whether a pose of the three-dimensional model corresponds to a pose of the character in the video images. The model is used to generate a frame of animation of the animated character when the pose of model corresponds to the pose of the character in the video images.

Abstract: A controller apparatus includes a capacitive or resistive touch pad having a touch sensitive surface sized sufficient for operation by two human feet. One or more markers are coupled to the touch pad. The markers are configured for use by a computer system for room mapping.

Abstract: An autonomous personal companion executing a method including capturing data related to user behavior. Patterns of user behavior are identified in the data and classified using predefined patterns associated with corresponding predefined tags to generate a collected set of one or more tags. The collected set is compared to sets of predefined tags of a plurality of scenarios, each to one or more predefined patterns of user behavior and a corresponding set of predefined tags. A weight is assigned to each of the sets of predefined tags, wherein each weight defines a corresponding match quality between the collected set of tags and a corresponding set of predefined tags. The sets of predefined tags are sorted by weight in descending order. A matched scenario is selected for the collected set of tags that is associated with a matched set of predefined tags having a corresponding weight having the highest match quality.

Abstract: Computer animation involving monocular pose prediction is disclosed. A plurality of candidate pose sequences of a three-dimensional model of an animation character is generated such that each candidate pose of each sequence has a segmentation map that matches a segmentation map of a corresponding character derived from a corresponding frame of a video. A distance between candidate poses at each time step is maximized. An optimum pose sequence is determined and used to generate a corresponding sequence of frames of animation.

Abstract: An event driven sensor (EDS) is used for simultaneous localization and mapping (SLAM) and in particular is used in conjunction with a constellation of light emitting diodes (LED) to simultaneously localize all LEDs and track EDS pose in space. The EDS may be stationary or moveable and can track moveable LED constellations as rigid bodies. Each individual LED is distinguished at a high rate using minimal computational resources (no image processing). Thus, instead of a camera and image processing, rapidly pulsing LEDs detected by the EDS are used for feature points such that EDS events are related to only one LED at a time.

Abstract: A method, system and computer product for training a control input system involve taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value and generating a subsequent output value using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold. Surface damping interactions with at least a simulated environment, a rigid body position and a position of the one or more movable joints based on an integral of the subsequent output value are simulated. The Motion Decision Neural Network is trained with the machine learning algorithm based upon at least a result of the simulation of the simulated environment and position of the one or more movable joints.

Abstract: A method, system and computer product for training a control input system involve taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value and generating a subsequent output value using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold. Surface stiffness interactions with at least a simulated environment, a rigid body position and a position of the one or more movable joints based on an integral of the subsequent output value are simulated. The Motion Decision Neural Network is trained with the machine learning algorithm based upon at least a result of the simulation of the simulated environment and position of the one or more movable joints.

Abstract: Computer animation involving pose disambiguation is disclosed. Two or more source segmentation masks are generated from corresponding contemporaneous video images of a character from different points of view. A three-dimensional model of an animation character corresponding to the character in the two or more contemporaneous video images is generated. Two or more different target segmentation masks corresponding to different views of the animation character corresponding to the character in the two or more video images. Each target segmentation mask is compared to a corresponding source segmentation mask from the comparison it is determined whether a pose of the three-dimensional model corresponds to a pose of the character in the video images. The model is used to generate a frame of animation of the animated character when the pose of model corresponds to the pose of the character in the video images.

Abstract: A method, system and computer program product for training a control input system involve taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value and comparing the integrated output value to a backlash threshold. A subsequent output value is generated using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold. A position of the one or more movable joints is simulated based on an integral of the subsequent output value; and the Motion Decision Neural Network is trained with the machine learning algorithm based upon at least a result of the simulation of the position of the one or more movable joints.

Abstract: Computer animation involving monocular pose prediction is disclosed. A plurality of candidate pose sequences of a three-dimensional model of an animation character is generated such that each candidate pose of each sequence has a segmentation map that matches a segmentation map of a corresponding character derived from a corresponding frame of a video. A distance between candidate poses at each time step is maximized. An optimum pose sequence is determined and used to generate a corresponding sequence of frames of animation.