Abstract: Aspects of the subject disclosure may include, for example, a camera positioned to capture image information of an immersive experience presented to one or more users engaged in the immersive experience and located in an immersive experience space, a processing system and a memory that stores executable instructions to facilitate performance of operations including receiving the image information from the camera, detecting objects located in the immersive experience space with the one or more users, the objects including at least one virtual object created by the immersive experience, determining the at least one virtual object is a projected virtual object of the immersive experience, generating a signal indicating the at least one virtual object is a projected virtual object, and a projector, responsive to the signal, to provide a visual indication in the immersive experience space to identify the projected virtual object as a virtual object to the one or more users engaged in the immersive experience.

Abstract: The invention relates to a method for ergonomically representing virtual information in a real environment, including the following steps: providing at least one view of a real environment and of a system setup for blending in virtual information for superimposing with the real environment in at least part of the view, the system setup comprising at least one display device, ascertaining a position and orientation of at least one part of the system setup relative to at least one component of the real environment, subdividing at least part of the view of the real environment into a plurality of regions comprising a first region and a second region, with objects of the real environment within the first region being placed closer to the system setup than objects of the real environment within the second region, and blending in at least one item of virtual information on the display device in at least part of the view of the real environment, considering the position and orientation of said at least one part of t

Abstract: A virtual reference plane and a virtual camera are updated based on detection of a characteristic portion in a captured image. A virtual object and the virtual camera are updated based on a shooting state. An overlay image in which an image of the virtual object is overlaid on the captured image is generated. The virtual camera and the virtual object are controlled such that the virtual object is in a field-of-view range, before the detection of the characteristic portion. The virtual object, the virtual reference plane, and the virtual camera are updated such that the virtual object is along the virtual reference plane, based on the shooting state, after the detection of the characteristic portion, and such that an appearance of the virtual object is in association with the shooting state, no matter whether or not the characteristic portion has been detected, when a position fixation condition is satisfied.

Abstract: A system for display for an augmented/virtual reality-based decision-making simulation includes a controller configured to receive input from a user, at least one of an augmented reality or a virtual reality device configured to display a simulated environment, a processor, and a memory coupled to the processor. The memory stores one or more computer-readable instructions, which, when executed by the processor, cause the system to: receive input by the user from the controller, indicating a selected scenario; display, on the at least one of the augmented reality or the virtual reality device, the selected scenario; receive input from the user, from the controller, to interact with the selected scenario; monitor one or more parameters associated with the execution of tasks in the selected scenario using the controller; and evaluate the user based on the monitored one or more parameters.

Abstract: Provided are an HMD device and operating method thereof. The operating method of an HMD device includes displaying at least one object in a display area of a transparent display, obtaining an image of a hand of a user interacting with the displayed object; determining a direction in which the hand is facing based on the obtained image, and performing a function for the object corresponding the direction in which the hand is facing.

Abstract: An augmented reality (AR) display device includes a camera that captures a background image; a distance measuring sensor that measures a distance to a real object in the background image; a position and orientation sensor that detects the position of the camera and the shooting direction of the camera; a controller that recognizes the real object from the background image captured by the camera and associates the predetermined AR object with the recognized real object; a display displaying an image of the associated AR object; and a memory that stores data of the real object and the AR object associated with each other. The controller determines whether or not the real object is movable from the measurement result of the distance measuring sensor, and arranges the AR object according to the current position of the real object when the position of the real object associated with the AR object moves.

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that utilize glyph sets from predefined instances of variable fonts to customize font bounding boxes for custom instances of the variable fonts. The disclosed system customizes digital text including a variable font via one or more adjustable design axes. In response to a request to set a custom value of a design axis, the disclosed system determines a first and second predefined instances of the digital text. For example, the disclosed system determines a Euclidean distance between the custom value and corresponding values for the predefined instances. The disclosed systems determine sets of glyphs that contribute to the font bounding boxes of the first predefined instance and the second predefined instance. The disclosed systems generate a custom font bounding box for the digital text at the custom value of the design axis based on the glyph sets.

Abstract: The present disclosure relates to methods, devices, and systems for presentation of multimedia content. In an aspect of the present disclosure, a method includes receiving, at a processor of a mobile device, a plurality of images from a camera, at least one image of the plurality of images including an object. The method also includes displaying a camera video stream based on the plurality of images, displaying three dimensional (3D) augmented reality content in the camera video content, and displaying video content based on the 3D augmented reality content. The video content is associated with the object.

Abstract: Structural modifications to a person's face in a reference image are captured and automatically applied to the person's face in another image. The reference image is processed to compute landmark information on the person's face and apply a mesh to the reference image. When structural modifications are made to the person's face in the reference image, the mesh is modified, and the modified mesh is stored in association with the landmark information. Another image is analyzed to compute landmark information on the person's face in that image and apply a mesh to the image. A transformation matrix is computed using the landmark information from the reference image and current image, and the modified mesh from the reference image is transformed using the transformation matrix. The mesh in the current image is modified using the transformed mesh, thereby applying the structural modification to the person's face in the current image.

Abstract: A system to facilitate AR processing includes receiving captured media from a user device and context information relating to media that is being delivered to a receiving device. The system may use the media being delivered with the captured media to generate one or more virtual objects. The user device may augment a user's view of reality that is reflected in the captured media by overlaying or otherwise incorporating the virtual objects in the user's view of reality.

Abstract: An augmented reality system having a light source and a camera. The light source projects a pattern of light onto a scene, the pattern being periodic. The camera captures an image of the scene including the projected pattern. A projector pixel of the projected pattern corresponding to an image pixel of the captured image is determined. A disparity of each correspondence is determined, the disparity being an amount that corresponding pixels are displaced between the projected pattern and the captured image. A three-dimensional computer model of the scene is generated based on the disparity. A virtual object in the scene is rendered based on the three-dimensional computer model.

Abstract: An exemplary extended reality presentation system determines that a depiction of an augmentable object within a field of view takes up a first portion of the field of view, and, in response to this determination, presents a first form of an overlay object within the field of view. The first form of the overlay object is graphically associated with the augmentable object. During the presenting of the first form of the overlay object within the field of view, the system determines that the depiction of the augmentable object has come to take up a second portion of the field of view, and, in response to this determination, replaces the first form of the overlay object with a second form of the overlay object within the field of view. The second form of the overlay object is distinct from the first form. Corresponding methods and systems are also disclosed.

Abstract: In some implementations, a method is performed at a device including non-transitory memory and one or more processors. The method includes: presenting, by the device, a computer-generated reality (CGR) environment; while presenting the CGR environment, detecting access to a display device separate from the device; in response to determining that there is access to the display device, triggering the display of video content, associated with the CGR environment, on the display device in order to supplement the CGR environment; and modifying at least a portion of the CGR environment based on the video content displayed by the display device. In some implementations, the display device is present within a physical environment, and the device enables video pass-through or optical see-through of at least a portion of the physical environment including the display device.

Abstract: A device including a storage medium storing instructions, and at least one processor configured to execute the stored instructions to perform operations is provided. The operations include displaying on a display screen an interactive scene containing an augmented reality element, receiving input to change display of the interactive scene, and receiving a request to simulate the capturing of image data representing the interactive scene. The operations further include determining a position of the augmented reality element relative to a region defined on the display screen, simulating the capturing of the image data representing the interactive scene, and providing, based on the determined position, an indication of whether the simulation has succeeded.

Abstract: A scene data obtaining method as well as a model training method and a computer readable storage medium using the same are provided. The method includes: building a virtual simulation scene corresponding to an actual scene, where the virtual simulation scene is three-dimensional; determining a view frustum corresponding to preset view angles in the virtual simulation scene; collecting one or more two-dimensional images in the virtual simulation scene and ground truth object data associated with the one or more two-dimensional images using the view frustum corresponding to the preset view angles; and using all the two-dimensional images and the ground truth object data associated with the one or more two-dimensional images as scene data corresponding to the actual scene. In this manner, the data collection does not require manual annotation, and the obtained data can be used for training deep learning-based perceptual models.

Abstract: An image readjustment system featuring a communication interface, one or more processors, and a persistent storage. The persistent storage is communicatively coupled to the one or more processors. The persistent storage comprises at least image location logic and image cropping logic. The image location logic is configured to receive and analyze an incoming image, using a machine learning analysis, to determine a location of a particular component within the incoming image and to calculate coordinates of a boundary area surrounding the particular component. The image cropping logic is configured to remove portions of the incoming image outside of the boundary area to generate a modified image to be rendered for display.

Abstract: Various implementations disclosed herein include devices, systems, and methods that enable more intuitive and efficient selection and positioning of visual effects. In some implementations, a visual effect is selected and a surface of a physical element is identified for application of the visual effect based on tracking one or more positions or gestures of a user's hand.

Abstract: In one embodiment, a method includes identifying a first user in an input image, accessing social data of the first user in the input image, where social data comprises information from a social graph of an online social network, selecting, based on social data of the first user in the input image, a mask from a set of masks, where the mask specifies one or more mask effects, and for each of the input images, applying the mask to the input image. The set of masks may comprise masks previously selected by friends of the first user within the online social network. The selected mask may be selected from a lookup table that maps the social data to the selected mask.

Abstract: An information processing system includes an attribute-information acquisition unit and an identification unit. The attribute-information acquisition unit acquires an attribute of an object that is present in a real space in accordance with information regarding the object an object image of which is captured. The identification unit identifies an image that is a virtual image associated with the attribute acquired by the attribute-information acquisition unit, the image being combined with the object image.

Abstract: An augmented reality (AR) device can be configured to generate a virtual representation of a user's physical environment. The AR device can capture images of the user's physical environment to generate or identify a user's location. The AR device can project graphics at designated locations within the user's environment to guide the user to capture images of the user's physical environment. The AR device can provide visual, audible, or haptic guidance to direct the user of the AR device to explore the user's environment.