Abstract: Non-player characters (NPC) in a computer game may be controlled using context in the form of player statements from the game engine to an LLM to drive NPC dialog and control NPC actions. Imperative statements are executed by game engine, and the result of this execution plus the original player request are converted to plain text description and injected back into the NPC dialog to ensure the NPC remains within defined bounds.

Abstract: Instead of feeding data pairs “input”-> “output” into a LLM to train it, an artificial “output”-> “input” dataset is generated to fine tune the LLM on it. As an example, it may be difficult to parse object/action words from a random sentence. But generating a random sentence given an object, action pair can be solved easily, and these reversed pairs are used to fine tune the LLM.

Abstract: Non-AV output of a game and specifically tactile/haptics output on a controller is automatically generated by AI based on player motion, game content, and a virtual/real environment. This helps developers and gives players real time feedback on how well they are doing.

Abstract: System, process and device configurations are provided for detecting user socialization preferences and controlling multiplayer selection. A method can include receiving user voice output during gameplay for a first user, and determining a socialization preference for the first user based on the user voice output. The method may also include updating a user profile for the first user based on the socialization preference, selecting at least one second user, and controlling a gameplay session for the first user and the at least one second user, wherein audio output for the first user is controlled using a the user profile. Socialization preferences detected during gameplay may be used to select other users for multiplayer game sessions. The socialization preferences may also be used to control audio output by way of selection/terminating players from a session, filtering audio output to a user, filtering intensity and/or modifying audio output to a user.

Abstract: System, process and device configurations are provided for analyzing video data, such as electronic game data, and generating scoring data for video data. Processes and device configurations include training a video data analysis model to generate scoring parameters, and for an evaluation tool for video data. Highly viewed and popular videos may be used as training input to the video analysis model. At least one score parameter may be determined for scoring videos. For electronic games, at least one score may be associated with a game scene, gameplay storylines, and for scoring video data in general. Analysis of game titles and game design may be performed during design to ensure that game scenes will have a high likelihood of user interest. Embodiments may also include modifying gameplay and gameplay storylines, and evaluating game promotional videos.

Abstract: An event driven sensor (EDS) is used for simultaneous localization and mapping (SLAM) and in particular is used in conjunction with a constellation of light emitting diodes (LED) to simultaneously localize all LEDs and track EDS pose in space. The EDS may be stationary or moveable and can track moveable LED constellations as rigid bodies. Each individual LED is distinguished at a high rate using minimal computational resources (no image processing). Thus, instead of a camera and image processing, rapidly pulsing LEDs detected by the EDS are used for feature points such that EDS events are related to only one LED at a time.

Abstract: Machine learning is used to transform screen renders from the viewpoint of the player's character to be from the viewpoint of other non-player characters (NPC) in the room. One or more neural networks are trained using game images captured during a human play session, and then subsequently the neural networks are used to create realistic video from the NPC viewpoints. To avoid rendering multiple viewpoints simultaneously, a single viewpoint is rendered, and neural networks are used to transform it for the viewpoints of other NPCs in the area. A group of NPCs may be treated as a batch and a single viewpoint transformed to multiple viewpoints in a single inference pass. For cloud gaming the game video can be rendered/sent immediately for the player but the actions of the NPCs are delayed for a frame while the neural network generates the behavior.

Abstract: A method, system and computer program product for training a control input system involve taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value and comparing the integrated output value to a backlash threshold. A subsequent output value is generated using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold. A position of the one or more movable joints is simulated based on an integral of the subsequent output value; and the Motion Decision Neural Network is trained with the machine learning algorithm based upon at least a result of the simulation of the position of the one or more movable joints.

Abstract: A method, system and non-transitory instructions for control input, comprising, taking an integral of an output value from a Motion Decision Neural Network for a movable joint to generate an integrated output value. Generating a subsequent output value using a machine learning algorithm that includes a sensor value and the integrated output value as inputs to the Motion Decision Neural Network and imparting movement with the moveable joint according to an integral of the subsequent output value.

Abstract: System, process and device configurations are provided for analyzing video data, such as electronic game data, and generating scoring data for video data. Processes and device configurations include training a video data analysis model to generate scoring parameters, and for an evaluation tool for video data. Highly viewed and popular videos may be used as training input to the video analysis model. At least one score parameter may be determined for scoring videos. For electronic games, at least one score may be associated with a game scene, gameplay storylines, and for scoring video data in general. Analysis of game titles and game design may be performed during design to ensure that game scenes will have a high likelihood of user interest. Embodiments may also include modifying gameplay and gameplay storylines, and evaluating game promotional videos.

Abstract: System, process and device configurations are provided for detecting user socialization preferences and controlling multiplayer selection. A method can include receiving user voice output during gameplay for a first user, and determining a socialization preference for the first user based on the user voice output. The method may also include updating a user profile for the first user based on the socialization preference, selecting at least one second user, and controlling a gameplay session for the first user and the at least one second user, wherein audio output for the first user is controlled using a the user profile. Socialization preferences detected during gameplay may be used to select other users for multiplayer game sessions. The socialization preferences may also be used to control audio output by way of selection/terminating players from a session, filtering audio output to a user, filtering intensity and/or modifying audio output to a user.

Abstract: A controller apparatus includes two or more balls having a size sufficient for a human to manipulate with a single foot. Three or more bearings support at least one of the balls from below and permit rotation with respect to at least two axes. One or more position encoders are disposed proximate a surface of each of the two or more balls. The encoders are configured to determine a rotational displacement of each of the two or more balls with respect to two or more axes.

Abstract: A foot operated controller apparatus is disclosed. The apparatus includes a ball large enough for a human to manipulate with two feet, and a support apparatus configured to support the ball from below, restrict translation of the ball and permit rotation of the ball about its center with respect to at least two axes. One or more position encoders are disposed proximate a surface of the ball. The encoder(s) are configured to determine a rotational displacement of the ball with respect to two or more axes.

Abstract: A method, system and computer product for training a control input system involve taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value and generating a subsequent output value using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold. Surface damping interactions with at least a simulated environment, a rigid body position and a position of the one or more movable joints based on an integral of the subsequent output value are simulated. The Motion Decision Neural Network is trained with the machine learning algorithm based upon at least a result of the simulation of the simulated environment and position of the one or more movable joints.

Abstract: Respective pose and motion sensors are engaged with the left and right feet of a sitting person and signals from the sensors are used to control the motion of a VR object presented on, e.g., a head-mounted display worn by the person. The sensor signals can be mapped to translational, rotational, and elevational motion of the VR object, and/or can be used to select controls to send controller-like signals to the game engine to control the VR object.

Abstract: An event driven sensor (EDS) is used for simultaneous localization and mapping (SLAM) and in particular is used in conjunction with a constellation of light emitting diodes (LED) to simultaneously localize all LEDs and track EDS pose in space. The EDS may be stationary or moveable and can track moveable LED constellations as rigid bodies. Each individual LED is distinguished at a high rate using minimal computational resources (no image processing). Thus, instead of a camera and image processing, rapidly pulsing LEDs detected by the EDS are used for feature points such that EDS events are related to only one LED at a time.

Abstract: An autonomous personal companion utilizing a method of object identification that relies on a hierarchy of object classifiers for categorizing one or more objects in a scene. The classifier hierarchy is composed of a set of root classifiers trained to recognize objects based on separate generic classes. Each root acts as the parent of a tree of child nodes, where each child node contains a more specific variant of its parent object classifier. The method covers walking the tree in order to classify an object based on more and more specific object features. The system is further comprised of an algorithm designed to minimize the number of object comparisons while allowing the system to concurrently categorize multiple objects in a scene.

Abstract: Machine learning is used to transform screen renders from the viewpoint of the player's character to be from the viewpoint of other non-player characters (NPC) in the room. One or more neural networks are trained using game images captured during a human play session, and then subsequently the neural networks are used to create realistic video from the NPC viewpoints. To avoid rendering multiple viewpoints simultaneously, a single viewpoint is rendered, and neural networks are used to transform it for the viewpoints of other NPCs in the area. A group of NPCs may be treated as a batch and a single viewpoint transformed to multiple viewpoints in a single inference pass. For cloud gaming the game video can be rendered/sent immediately for the player but the actions of the NPCs are delayed for a frame while the neural network generates the behavior.

Abstract: Computer animation involving monocular pose prediction is disclosed. A plurality of candidate pose sequences of a three-dimensional model of an animation character is generated such that each candidate pose of each sequence has a segmentation map that matches a segmentation map of a corresponding character derived from a corresponding frame of a video. A distance between candidate poses at each time step is maximized. An optimum pose sequence is determined and used to generate a corresponding sequence of frames of animation.

Abstract: A foot operated controller apparatus is disclosed. The apparatus includes a ball large enough for a human to manipulate with two feet, and a support apparatus configured to support the ball from below, restrict translation of the ball and permit rotation of the ball about its center with respect to at least two axes. One or more position encoders are disposed proximate a surface of the ball. The encoder(s) are configured to determine a rotational displacement of the ball with respect to two or more axes.