Abstract: Herein is presented a light-weight, high-quality texture synthesis algorithm that generalizes to other applications. We utilize an optimal transport optimization process within a bottleneck layer of an auto-encoder, achieving quality and flexibility on par with expensive back-propagation based neural texture synthesis methods, but at interactive rates. In addition to superior synthesis quality, our statistically motivated approach generalizes better to other special case texture synthesis problems such as Style Transfer, Inverse-Texture Synthesis, Texture Mixing, Multi-Scale Texture Synthesis, Structured Image Hybrids and Texture Painting. We treat the texture synthesis problem as the optimal transport between Probably Density Function of the deep neural activation vectors of the image being synthesized and the exemplar texture.

Abstract: A method of aligning a digital model of a structure with a displayed portion of the structure within a video stream is disclosed. An approximate position of the camera device in the digital model is determined. A position and an orientation are determined for a plurality of digital surfaces within the digital model visible from the approximate position of the camera device. A position and an orientation of a plurality of object surfaces visible in a video stream are determined. A 3D translation, a 3D scale, and a 3D rotation that maximize an alignment of the position and orientation of the plurality of digital surfaces with the position and orientation of the plurality of object surfaces are determined. The 3D translation, the 3D scale, and the 3D rotation are applied to the digital model and the digital model is displayed contemporaneously with a display of the video stream.

Abstract: Systems and methods for generating a dynamic note and associating the dynamic note to a game object are disclosed. The game object is selected in a digital environment. The dynamic note is associated to the game object. Data is added to the dynamic note. The added data can include data describing a state of the environment and a state of the game object and include a link to the game object. Data associated with the dynamic note is sent over a network to a data storage device on a server. An aspect of the dynamic note is displayed within the digital environment.

Abstract: In example embodiments, a method of interactive imitation learning method is disclosed. An input is received from an input device. The input includes data describing a first set of example actions defining a behavior for a virtual character. Inverse reinforcement learning is used to estimate a reward function for the set of example actions. The reward function and the set of example actions is used as input to a reinforcement learning model to train a machine learning agent to mimic the behavior in a training environment. Based on a measure of failure of the training of the machine learning agent reaching a threshold, the training of the machine learning agent is paused to request a second set of example actions from the input device. The second set of example actions is used in addition to the first set of example actions to estimate a new reward function.

Abstract: A method for optimizing a computer-implemented game for a target metric is disclosed. Based on a detecting that an optimization point has been reached during a runtime of the computer game, user data, game state data, and personalized gaming experience (PGE) question data is communicated to a PGE server. The PGE question data is linked to the optimization point. An answer corresponding to the PGE question data is received from the PGE server. The answer is selected from a plurality of answers linked to the optimization point based on an application of a machine-learned model to the user data, game state data, and PGE question data. The received answer is implemented within the computer-implemented game.

Abstract: A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

Abstract: A method is disclosed for allowing a custom version of video session to be created for presentation on at least one viewer device. Game metadata generated from a live or recorded video feed is received. The game metadata includes three-dimensional modeling data associated with the live or recorded video feed. Viewer metadata collected from a plurality of viewer devices is received. The viewer metadata includes information pertaining to a plurality of responses of a plurality of viewers to a presentation of the video session on a plurality of viewer devices. Additional game metadata is created based on the game metadata and the viewer metadata. The additional game metadata includes camera data based on the three-dimensional modeling data. The additional game metadata is integrated into the game metadata for at least near-real-time presentation of the custom version of the video session.

Abstract: Systems and methods for spawning a digital object in an environment are disclosed. Data describing the environment is received. The data includes data describing properties of the environment, a state of the environment, and properties of a plurality of objects within the environment. The data is analyzed to detect and categorize one or more of the plurality of objects, and to detect one or more surfaces related to the plurality of objects. Data is received that describes a placement of the digital object on one of the detected surfaces or detected objects and determines properties of the placement. Conditions are associated with the placed digital object, the conditions including data describing properties of the placement, data describing properties of the detected object, and data describing a state of the detected object. The spawning of the digital object is performed in the environment based on the conditions.

Abstract: A method of generating a high-resolution image frame for a state of a video game within a 2D or 3D environment is disclosed. A low-resolution data map of a virtual camera frustum view of the 2D or 3D environment for the state is determined. The data map is of a data type. A high-resolution output data map of the data type is generated from the low-resolution data map. The generating of the high-resolution output data map includes training a neural network. The training includes associating a low-resolution data map of the data type with a high-resolution data map of the data type within the 2D or 3D environment. A high-resolution image of the frustum view is generated from the high-resolution output data map. The generated high-resolution image is displayed on a display device.

Abstract: Methods and apparatuses to improve the performance of a video game engine using an Entity Component System (ECS) are described herein. In accordance with an embodiment, the ECS creates and uses entities, to represent game objects, which are constructed entirely using value data types. The ECS constructs the entities within a memory in a densely packed linear way, and whereby the ECS constantly monitors (e.g., during game play) objects within a game and adjusts the entity distribution within the memory so that a maximum density of memory usage is maintained in real time as the game is being played.

Abstract: A method of synchronizing digital content between a first mobile device and a second mobile device is disclosed. Device position and orientation data is received at a first application executing in an operating system of the first mobile device. Sensor data is used to determine at least two 3D points associated with a physical location of the second mobile device. At least two 3D points are used to determine an offset for the position, orientation and scale of the first mobile device relative to the second mobile device. The offset and digital content data captured by the first mobile device are shared over a network with a second application executing in an operating system of the second mobile device. The second mobile device uses the offset to display the digital content captured by the first mobile device together with digital content data captured by the second mobile device.

Abstract: A method for improving a display of a user interface element in a mixed reality environment is disclosed. A request to display the user interface element is received. The request includes display instructions, angle threshold data, distance threshold data, and velocity threshold data. Display operations are continuously performed while sensor data is continuously received from a mixed reality user interface device. The display operations include displaying the user interface element according to the display instructions, and, based on the sensor data indicating a distance between the user interface element and the mixed reality user interface device in the mixed reality environment has exceeded a distance threshold or based on the sensor data indicating an angle of view of the mixed reality user interface device has exceeded an angle threshold with respect to the user interface element in the mixed reality environment, hiding the user interface element.

Abstract: A method of generating a digital island is disclosed. A base shape of a digital island virtual object is created with respect to a local coordinate system of a digital island. Conditions are associated with the digital island. The conditions include criteria for properties of objects within an environment. Additional virtual objects are associated with the digital island. Data describing the environment is received. The data is analyzed to determine one or more parts of the environment that match the conditions. Based on the matching of conditions, a display of the digital island is positioned, scaled, and oriented in a mixed reality display device contemporaneously with a view of the environment. The digital island with is aligned with at least one part of the one or more parts based on the conditions and a movement of one or more sensors and the mixed reality display device.

Abstract: A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.

Abstract: A method of generating an image frame for depicting a posing of a character in a game is disclosed. A dominant atom from a set of active atoms associated with the character is determined. A motion controlling ability from a plurality of abilities of the character is determined. A motion fragment for the dominant atom is determined based on pose data and future trajectory data in a motion library associated with the dominant atom; A predicted future trajectory of the character is calculated. Based on a detection of a future collision or problem between the predicted future trajectory and an environment within the game, a combined future trajectory is calculated based on the predicted future trajectory and an additional future trajectory. The character is posed in the image frame based on a weighted combination of poses associated with the set of active atoms.

Abstract: A method for tracking game asset locations is disclosed. Content is created for an application using a first asset. The first asset includes asset data that describes at least a part of the content associated with the asset. The asset data is placed within one or more files within a location within one or more memories. A catalog associated with the application is created on the server. A catalog entry is created for the first asset. The catalog entry includes an address that uniquely identifies the first asset. A request is received from the application for asset data associated with the first asset, the request including the address. The address and the catalog are used to determine location data associated with the address. The determined location data is used to retrieve the asset data at the determined location. The retrieved asset data is returned to the application.

Abstract: A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.

Abstract: A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.

Abstract: A method of generating an image frame for depicting a posing of a character in a game is disclosed. A dominant atom from a set of active atoms associated with the character is determined. A motion controlling ability from a plurality of abilities of the character is determined. A motion fragment for the dominant atom is determined based on pose data and future trajectory data in a motion library associated with the dominant atom; A predicted future trajectory of the character is calculated. Based on a detection of a future collision or problem between the predicted future trajectory and an environment within the game, a combined future trajectory is calculated based on the predicted future trajectory and an additional future trajectory. The character is posed in the image frame based on a weighted combination of poses associated with the set of active atoms.

Abstract: A method is disclosed for allowing a custom version of video session to be created for presentation on at least one viewer device. Game metadata generated from a live or recorded video feed is received. The game metadata includes three-dimensional modeling data associated with the live or recorded video feed. Viewer metadata collected from a plurality of viewer devices is received. The viewer metadata includes information pertaining to a plurality of responses of a plurality of viewers to a presentation of the video session on a plurality of viewer devices. Additional game metadata is created based on the game metadata and the viewer metadata. The additional game metadata includes camera data based on the three-dimensional modeling data. The additional game metadata is integrated into the game metadata for at least near-real-time presentation of the custom version of the video session.