Abstract: According to one aspect, a toy system is adapted for use in a multiplayer game, the toy system comprising a primary user device, a secondary user device, and one or more toys; wherein the primary and secondary user devices are operable in a common multiplayer game session; wherein the primary user device is configured to capture video images of a real-world scene including the one or more toys from a primary view point and to display the captured video images as a primary view of the real-world scene; wherein the secondary user device is configured to display a secondary view of a scene corresponding to the real-world scene as seen from a secondary view point; wherein the one or more toys comprise at least one user-manipulable object adapted to selectively be brought into a user-selected one of a plurality of predetermined states, at least a first one of the predetermined states having a first visual appearance; wherein the primary user device is configured to detect said first predetermined state in a primar

Abstract: Methods and systems are provided for dynamic interaction with an augmented reality environment. In some embodiments, the systems and methods are directed at dynamically interacting with machinery within the augmented reality environment via an augmented reality device.

Abstract: A method is disclosed, the method comprising the steps of identifying a first real object in a mixed reality environment, the mixed reality environment having a user; identifying a second real object in the mixed reality environment; generating, in the mixed reality environment, a first virtual object corresponding to the second real object; identifying, in the mixed reality environment, a collision between the first real object and the first virtual object; determining a first attribute associated with the collision; determining, based on the first attribute, a first audio signal corresponding to the collision; and presenting to the user, via one or more speakers, the first audio signal.

Abstract: Methods and systems are provided for dynamic interaction with an augmented reality environment. In some embodiments, the systems and methods are directed at dynamically interacting with machinery within the augmented reality environment via an augmented reality device.

Abstract: Systems, methods, devices, and other techniques for placing and rendering virtual objects in three-dimensional environments. The techniques include providing, by a device, a view of an environment of a first user. A first computing system associated with the first user receives an instruction to display, within the view of the environment of the first user, a virtual marker at a specified position of the environment of the first user, the specified position derived from a second user's interaction with a three-dimensional (3D) model of at least a portion of the environment of the first user. The device displays, within the view of the environment of the first user, the virtual marker at the specified position of the environment of the first user.

Abstract: Described herein are techniques for learning neural reflectance shaders from images. A set of one or more machine learning models can be trained to optimize an illumination latent code and a set of reflectance latent codes for an object within a set of input images. A shader can then be generated based on a machine learning model of the one or more machine learning models. The shader is configured to sample the illumination latent code and the set of reflectance latent codes for the object. A 3D representation of the object can be rendered using the generated shader.

Abstract: A computer-implemented method includes capturing visual data of an environment using an image sensor of an electronic device and non-visual data of the environment using one or more non-image sensors of the electronic device. Feature descriptors of one or more objects in the environment are generated using the visual data of the environment and the non-visual data of the environment. A map of the environment is generated using the feature descriptors of the one or more objects. One or more virtual objects are anchored to at least one of the objects using the map. The visual data, the non-visual data, and the map are combined in a digital multimedia container file. The digital multimedia container file is stored on the electronic device or on another electronic device connected to the electronic device.

Abstract: An information processing apparatus (2000) detects an abnormal region (30) from a moving image frame (14). The abnormal region (30) is a region that is estimated to represent an abnormal part inside a body of a subject. The information processing apparatus (2000) generates and outputs output information based on the number of detected abnormal regions (30).

Abstract: A method for graphics processing. The method including rendering graphics for an application using a plurality of graphics processing units (GPUs). The method including dividing responsibility for the rendering geometry of the graphics between the plurality of GPUs based on a plurality of screen regions, each GPU having a corresponding division of the responsibility which is known to the plurality of GPUs. The method including generating information regarding a piece of geometry with respect to a first screen region for which a first GPU has a first division of responsibility, while rendering the piece of geometry at a second GPU for an image. The method including rendering the piece of geometry at the first GPU using the information.

Abstract: Example methods, apparatuses, and/or articles of manufacture are disclosed that may be implemented, in whole or in part, techniques to select between and/or among multiple available alternative approaches to perform a temporal anti-aliasing operation in processing an image.

Abstract: A method for capturing extended reality environments includes receiving image data from at least one image sensor associated with a wearable extended reality appliance, the image data representing at least a 140 degrees field of view of a physical environment. At least two extended reality objects are virtually associated with a composite perspective of the physical environment, wherein the at least two extended reality objects are spaced apart by at least 140 degrees from a point of view of the wearable extended reality appliance. Displaying, via the wearable extended reality appliance and during a particular time period, changes in one of the at least two extended reality objects while refraining from displaying changes in another of the at least two extended reality objects. Enabling non-synchronous display of concurrent changes in the at least two extended reality objects that took place during the particular time period.

Abstract: The present invention provides a method of generating a robust global map using a plurality of limited field-of-view cameras to capture an environment. Provided is a method for generating a three-dimensional map comprising: receiving a plurality of sequential image data wherein each of the plurality of sequential image data comprises a plurality of sequential images, further wherein the plurality of sequential images is obtained by a plurality of limited field-of-view image sensors; determining a pose of each of the plurality of sequential images of each of the plurality of sequential image data; determining one or more overlapping poses using the determined poses of the sequential image data; selecting at least one set of images from the plurality of sequential images wherein each set of images are determined to have overlapping poses; and constructing one or more map portions derived from each of the at least one set of images.

Abstract: Methods and systems are provided for dynamic interaction with an augmented reality environment. In some embodiments, the systems and methods are directed at dynamically interacting with machinery within the augmented reality environment via an augmented reality device.

Abstract: A mobile apparatus includes a display device, a location receiving device configured to receive current location information for obtaining a current location, an image obtaining device configured to obtain an image of a surrounding environment, and a controller configured to control display of image information on the image obtained by the image obtaining device during execution of an augmented reality mode, obtain information on a building in the image information based on at least one of the current location information or map information obtained by the location receiving device, and control the display device to overlap and display representative information on the obtained building of the image information.

Abstract: Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an Interaction Engine that generates within a unified three-dimensional (3D) coordinate space: (i) a 3D virtual medical model positioned according to a model pose and (ii) at least one virtual object associated with a physical instrument, the physical instrument having a current instrument pose based at least on current coordinates of one or more fiducial markers disposed on the physical instrument, in the unified 3D coordinate space. The Interaction Engine renders an Augmented Reality (AR) display that includes concurrent display of the 3D virtual medical model and the virtual object. The Interaction Engine modifies the AR display according to a virtual interaction related to the virtual object that incorporates the change of the physical instrument pose.

Abstract: An augmented reality (AR) display device can display a virtual assistant character that interacts with the user of the AR device. The virtual assistant may be represented by a robot (or other) avatar that assists the user with contextual objects and suggestions depending on what virtual content the user is interacting with. Animated images may be displayed above the robot's head to display its intents to the user. For example, the robot can run up to a menu and suggest an action and show the animated images. The robot can materialize virtual objects that appear on its hands. The user can remove such an object from the robot's hands and place it in the environment. If the user does not interact with the object, the robot can dematerialize it. The robot can rotate its head to keep looking at the user and/or an object that the user has picked up.

Abstract: A mobile device comprises one or more processors, a display, and a camera configured to capture an image of a live scene. The one or more processors are configured to determine a location of the mobile device and display an augmented image based on the captured image. The augmented image includes at least a portion of the image of the live scene and a map including an indication of the determined location of the mobile device. The one or more processors are also configured to display the at least a portion of the image of the live scene in a first portion of the display and displaying the map in a second portion of the display. The augmented image is updated as the mobile device is moved, and the map is docked to the second portion of the display as the augmented image is updated.

Abstract: Disclosed are methods for augmenting progressive meter information presented in association with a gaming device. The methods include controlling a camera on a mobile device using an augmented reality gaming assistance component and enabling a user to employ the camera to capture an image of a game screen display, the image including one or more progressive meters associated with a respective progressive award. The captured image is sent via a network to a server that, for each of one or more progressive meters, determines a location of the progressive meters within the captured image, determines a value for each associated progressive award based on the captured image at each respective location, and determines a progressive rating for the associated progressive award based on the value. Content based on the one or more progressive ratings is received at the mobile device and displayed to the user via a display.

Abstract: Methods and systems employing augmented reality techniques via real-world objects for various purposes. Computer implemented methods are provided for animated augmentation of real-time video of static real-world objects. A computing device receives first data defining features and movements of a human subject, captures a video image of an object having features corresponding to human features, maps features defined by first data to corresponding features of the object, and uses the first data to animate the corresponding features of the object to correspond with the movements of the human subject defined by the first data. The object may be a physical 3D object. Embodiments may include positioning and orienting a 3D mesh representation of the object in registration with the object video image, prior to mapping the features defined by the first data to corresponding features on the 3D mesh. Recorded audio messages may be played in synchronism with the object.

Abstract: In a method, a visual hull is generated based on intersections of first 3D primitives of a plurality of first silhouettes with a bounding box of a 3D model. The first silhouettes are generated by projecting the 3D model onto planes perpendicular to a number of selected view directions of the 3D model. Each of the first 3D primitives is obtained by extruding a connected loop of a respective first silhouette along a view direction of the number of selected view directions that is associated with the respective first silhouette. A carved mesh is be generated based on subtractions of second 3D primitives derived from positive parts of the 3D model to carve out redundant structures from the visual hull. The positive parts are obtained based on fitting planes that slices the 3D model. A low-poly mesh sequence is generated based on progressive simplifications of the carved mesh.