Abstract: Provided herein are method, apparatus, and computer program products for generating a first and second three dimensional interactive environment. The first three dimensional interactive environment may contain one or more engageable virtual interfaces that correspond to one or more items. Upon engagement with a virtual interface the second three dimensional interactive environment is produced to virtual simulation related to the one or more items.

Abstract: Particular embodiments are directed to a passthrough feature. A computing system may display a virtual-reality scene on a device worn by a user. The system may receive a request to display a visual representation of at least a portion of a physical environment surrounding the user. The system may access data associated with the physical environment captured by camera(s) of the device. The system may generate, based the data, depth measurements of one or more objects in the physical environment. The system may generate, based on the depth measurements, one or more models of the one or more objects in the physical environment. The system may render an image based on a viewpoint of the user and the one or more models and, based on the image, generate the visual representation requested by the user. The visual representation may then be displayed with the virtual-reality scene to the user.

Abstract: An adjustable frame assembly for augmented reality eyewear. The frame assembly includes a face portion for supporting at least one waveguide that creates an eye box, a support rest for supporting the face portion on a user, and a coupling for adjusting the position of the face portion relative to the support rest. This enables movement of the waveguide eye box relative to the support rest to position the eye box in front of the wearer's eyes.

Abstract: A non-transitory computer readable medium embodies instructions that cause one or more processors to perform a method. The method includes selecting a 3D model corresponding to an object. The method further includes generating domain-adapted images of the 3D model, the domain-adapted images representing the 3D model at corresponding poses. The method further includes acquiring 2D projections of 3D points on a 3D bounding box defined around the 3D model at the corresponding poses. The method further includes training an algorithm model to learn correspondences between the generated images and the corresponding 2D projections. The method further includes storing, in a memory, parameters representing the algorithm model.

Abstract: An input system and an electronic apparatus with novel structures are provided. An electronic apparatus whose setting can be changed even when both hands are full is provided. An input system and an electronic apparatus including an imaging device having a function of taking an image of eyes and a detection device having a function of detecting a change of the shape of the eyes from data obtained by the imaging device are provided. Settings are changed in accordance with the change of the shape of the eyes. The settings are changed by a different method from a manual method and the change of the settings can be reflected in display.

Abstract: An information processing device has a computation unit for determining an optimal solution of an external force that minimizes a difference between a surface image of an elastic body captured from at least one direction, and a rendering image as observed from the direction which is generated from a three-dimensional model of the elastic body elastically deformed by an unobserved external force.

Abstract: A behavior history data acquisition unit 304 acquires behavior history data on a plurality of users wearing head mounted displays on heads thereof and experiencing a same three-dimensional content space. A line-of-sight direction acquisition unit 312 acquires line-of-sight directions of the plurality of users in the three-dimensional content space, from the behavior history data on the plurality of users. A screen generation unit 316 generates, on a basis of the acquired line-of-sight directions of the plurality of users, an image concerning line of sights of the plurality of users in a three-dimensional virtual space.

Abstract: Systems and methods for rendering three-dimensional (3D) scenes having improved visual characteristics from a pair of 2D images having different viewpoints. The 3D scene is created by obtaining a first two-dimensional (2D) image of a scene object from a first viewpoint, obtaining a second 2D image of the scene object from a second viewpoint that is different than the first viewpoint, creating a depth map from the first and second 2D images, creating a 3D scene from the depth map and the first and second 2D images, detecting regions of the initial 3D scene with incomplete image information, reconstructing the detected regions of the 3D scene, determining replacement information and modify the reconstructed regions, and rendering the 3D scene with the modified reconstructed regions from a plurality of viewpoints.

Abstract: A method (400) of rendering virtual objects is disclosed. The method (400) comprises: detecting (402) light (122) emitted by a lighting device (120), which light (122) comprises an embedded code representative of a property of the lighting device (120), obtaining (404) a virtual object, determining (406) an appearance and/or a behavior of the virtual object based on the property of the lighting device (120), and rendering (408) the virtual object on an image rendering device (104) according to the determined appearance and/or behavior.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure can determine a state for each graphics state group of a plurality of graphics state groups. Further, aspects of the present disclosure can determine whether at least one graphics state group of the plurality of graphics state groups includes a changed state. Additionally, aspects of the present disclosure can communicate state information for the at least one graphics state group when the at least one graphics state group includes a changed state. In some aspects, the state information includes information regarding the state of the at least one graphics state group. Aspects of the present disclosure can also configure a draw state for the plurality of graphics state groups, where the draw state includes state information for each of the graphics state groups.

Abstract: Provided is a face pose correction apparatus for correcting a face pose, using a person-specific frontal model specialized for input three-dimensional (3D) facial data. The face pose correction apparatus may comprise: a generation unit for generating corrected first facial data by performing a 3D rigid transformation of input 3D facial data on the basis of a pre-stored 3D standard facial model; and a correction unit for generating a first subject-specialized frontal model which is a bilateral symmetry model of the first facial data, and correcting the first facial data by performing the 3D rigid transformation of the first facial data on the basis of the generated first subject-specialized frontal model.

Abstract: A human-computer interface system having an exoskeleton including a plurality of structural members coupled to one another by at least one articulation configured to apply a force to a body segment of a user, the exoskeleton comprising a body-borne portion and a point-of-use portion; the body-borne portion configured to be operatively coupled to the point-of-use portion; and at least one locomotor module including at least one actuator configured to actuate the at least one articulation, the at least one actuator being in operative communication with the exoskeleton.

Abstract: Methods are provided for the highlighting of features in composite images through the alternating of images having complementary colors. An image having a feature of interest is used to generate one or more pseudo color images. A series of a pseudo color images and one or more additional pseudo color or original color images are then alternately displayed so that the differently colored regions among the series of images are easily recognizable to an operator. The differently colored regions differ in having hues that are complementary to one another. The methods are particularly useful for the display of information using two or more imaging modalities and channels, such as is the case for some medical applications in which a natural-light image of pink or light-red tissue with deeper red or blue vasculature is overlaid with another functional image.

Abstract: Aspects of this disclosure include technologies for appearance-flow-based image generation. In applications for pose-guided person image generation or virtual try-on, the disclosed system can model the appearance flow between source and target clothing regions. Further, a cascaded appearance flow estimation network is used to progressively refine the appearance flow. The resulting appearance flow can properly encode the geometric changes between the source and the target for image generation.

Abstract: Embodiments of the present application disclose a method and an apparatus for controlling placement of a virtual character and a storage medium. The method includes: receiving a first instruction in a virtual reality (VR) scene, the first instruction indicating moving a virtual character from a first position by a designated movement; determining a second position by the designated movement from the first position in response to the first instruction; removing the virtual character from the first position; and placing the virtual character at the second position, the virtual character not appearing between the first position and the second position after being removed from the first position.

Abstract: At a first computing device, a target image is identified. The target image is provided in a display on the first computing device in a communication area displaying a conversation between a first user using the first computing device and a second user using a second computing device. An indication is received that the first user has initiated editing of the target image. An edit control listing editing options is provided for display in the communication area. At the first computing device, a user selection is received of a particular editing option selected from the editing options. An edited target image is created by editing the target image using an edit operation associated with the particular editing option. The edited target image is provided to the second computing device and the communication area is updated to display the edited target image.

Abstract: A client apparatus is configured to establish remote connection with a server apparatus, display, on the display device, information on a basis of a series of operational procedures remotely input in the operation device by the server apparatus, if it is determined that the series of operational procedures input in the operation device is a first series of operational procedures, un-display, on the display device, information on a basis of a series of operational procedures remotely input in the operation device by the server apparatus, subsequently to the first series of operational procedures, if it is determined that the series of operational procedures input in the operation device subsequently to the first series of operational procedures is a second series of operational procedures, start a remote adjustment mode, the remote adjustment mode being a mode for remotely adjusting the client apparatus by the server apparatus.

Abstract: Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for detecting a pose of a user. The program and method include operations comprising receiving a monocular image that includes a depiction of a body of a user; detecting a plurality of skeletal joints of the body based on the monocular image; accessing a video feed comprising a plurality of monocular images received prior to the monocular image; filtering, using the video feed, the plurality of skeletal joints of the body detected based on the monocular image; and determining a pose represented by the body depicted in the monocular image based on the filtered plurality of skeletal joints of the body.

Abstract: The present invention relates to a method for providing a mixed reality service of a server and a system for performing the same, the method comprising the steps of: loading three-dimensional modeling data including design information for arbitrary building; generating a virtual image for the three-dimensional modeling data and at least one user interface on the basis of tracking information transmitted from a terminal; and transmitting the virtual image to the terminal, wherein the tracking information includes at least one of geographical position information of the terminal, identification information of an actual space where the terminal is located, three-dimensional rotation information, and speed information, and wherein the virtual image is generated such that the actual size of the building may be displayed according to the design information, and is synthesized with an actual image in the at least one terminal and displayed as a mixed reality image.

Abstract: In various example embodiments, a system and method for simulating touch in a virtual environment is disclosed. In one example embodiment, a system includes one or more circuits configured to receive an indicator of a sensed touch in a virtual environment and to determine, based on the indicator, an area of the sensed touch. The one or more circuits are further configured to generate a simulated touch by applying a field to one or more touch simulators, the field actuating the one or more touch simulators by linearly displacing an element of the one or more touch simulators.