Abstract: An exercise instruction apparatus for guiding a user to perform a fitness exercise includes a control unit and a display device. The control unit is configured to control a video imagery displayed by the display device. The video imagery shows an instructor image and a user image simultaneously. The instructor image demonstrates movements of the fitness exercise and the user image presented in a mirror image is a real-time image of the user standing in front of the display device. The control unit is configured to adjust at least one image parameters such as brightness, transparency or contrast of at least one of the instructor image and the user image. The image parameter that is adjusted for the instructor image may be a different parameter than the image parameter that is adjusted for the user image. The instructor image may be overlapped on the user image, and the user can adjust the transparency of at least one of the first image and the second image.

Abstract: Provided is a method for 3D modeling based on an irregular-shaped sketch, in which the method is executed by one or more processors, and includes receiving 2D sketch data of a target object, inputting the 2D sketch data into a 3D model generation model to generate a 3D model of the target object, and displaying the generated 3D model on a display.

Abstract: Methods and systems are disclosed for performing operations comprising: receiving a video that includes a depiction of a real-world object; generating a three-dimensional (3D) body mesh associated with the real-world object that tracks movement of the real-world object across frames of the video; obtaining an external mesh associated with an augmented reality element; automatically establishing a correspondence between the 3D body mesh associated with the real-world object and the external mesh; deforming the external mesh based on movement of the real-world object and the established correspondence with the 3D body mesh; and modifying the video to include a display of the augmented reality element based on the deformed external mesh.

Abstract: An image processing method and a computer system. The method may be applied to a cloud-side server in a cloud mobile phone. The server may be a virtualization server, a host operating system and a guest operating system are deployed on the server, a user mode graphics driver is deployed in the guest operating system, and a kernel mode graphics driver is deployed in the host operating system. The user mode graphics driver and the kernel mode graphics driver collaborate with each other to implement image rendering of the server. Then, the server may send a rendered image to the cloud mobile phone. Accordingly, an instruction translation process is reduced, to reduce overheads of a processor and improve image processing efficiency.

Abstract: A technology that streams graphical components and rendering instructions to a client device, for the client device to perform the final rendering and overlaying of that content onto the client's video stream based on the client's most recent tracking of the device's position and orientation. A client device sends a request for augmented reality drawing data to a network device. In response, the network device generates augmented reality drawing data, which can be augmented reality change data based on the augmented reality information and previous client render state information, and sends the augmented reality drawing data to the client device. The client device receives the augmented reality drawing data and renders a visible representation of an augmented reality scene comprising overlaying augmented reality graphics over a current video scene obtained from a camera of the client device.

Abstract: A system includes an augmented virtual reality (AVR) object creation engine, an AVR object enhancement engine, an AVR object positioning engine, and an AVR media authoring engine. The AVR object creation engine is configured to convert real world data into one or more AVR objects. The AVR object enhancement engine is configured to enhance the one or more AVR objects to include at least one of processed data visualization and multiuser controls. The AVR object positioning engine is configured to position the enhanced one or more AVR objects in a virtual space-time. The AVR media authoring engine is configured to make available, as AVR media, a scene tree including the virtual space-time in which the enhanced one or more AVR objects are positioned.

Abstract: Provided is a virtual object processing method. The virtual object processing method includes: detecting a spatial plane in a scene where a first device is located; detecting real objects in the scene to determine a plurality of real object position boxes; determining, based on a matching relation between the plurality of real object position boxes and the spatial plane in the scene, a candidate position box set from the plurality of real object position boxes; determining, in response to a virtual object configuration operation for a target position box in the candidate position box set, position information of the virtual object in the target position box; transmitting position on the virtual object and the position information of the virtual object in the target position box to a second device for displaying the virtual object on the second device.

Abstract: Aspects of the present disclosure are directed to creating interactive avatars that can be pinned as world-locked artificial reality content. Once pinned, an avatar can interact with the environment according to contextual queues and rules, without active control by the avatar owner. An interactive avatar system can configure the avatar with action rules, visual elements, and settings based on user selections. Once an avatar is configured and pinned to a location by an avatar owner, when other XR devices are at that location, a central system can provide the avatar (with its configurations) to the other XR device. This allows a user of that other XR device to discover and interact with the avatar according to the configurations established by the avatar owner.

Abstract: A method for adjusting an over-rendered area of a display in an AR device is described. The method includes identifying an angular velocity of a display device, a most recent pose of the display device, previous warp poses, and previous over-rendered areas, and adjusting a size of a dynamic over-rendered area based on a combination of the angular velocity, the most recent pose, the previous warp poses, and the previous over-rendered areas.

Abstract: The present disclosure provides an unmanned aerial vehicle platform based vision measurement method for a static rigid object. Aiming at the problem of high professionality but poor versatility of existing vision measurement methods, the present disclosure uses a method combining object detection and three-dimensional reconstruction to mark an object to be measured, and uses a three-dimensional point cloud processing method to further mark a size to be measured and calculate its length, which takes full advantage of the convenience of data collection by an unmanned aerial vehicle platform (UAV), and its global navigation satellite system (GNSS), an inertial measurement unit (IMU) and the like to assist measurement. There is no need to use common auxiliary devices such as a light pen and a marker, which can improve the versatility of vision measurement.

Abstract: A cross reality system enables any of multiple devices to efficiently render shared location-based content. The cross reality system may include a cloud-based service that responds to requests from devices to localize with respect to a stored map. The service may return to the device information that localizes the device with respect to the stored map. In conjunction with localization information, the service may provide information about locations in the physical world proximate the device for which virtual content has been provided. Based on information received from the service, the device may render, or stop rendering, virtual content to each of multiple users based on the user's location and specified locations for the virtual content.

Abstract: Various methods are provided for the generation of motion vectors in the context of 3D computer-generated images. In one example, a method includes generating, for each pixel of one or more objects to be rendered in a current frame, a 1-phase motion vector (MV1) and a 0-phase motion vector (MV0), each MV1 and MV0 having an associated depth value, to thereby form an MV1 texture and an MV0 texture, each MV0 determined based on a camera MV0 and an object MV0, converting the MV1 texture to a set of MV1 pixel blocks and converting the MV0 texture to a set of MV0 pixel blocks and outputting the set of MV1 pixel blocks and the set of MV0 pixel blocks for image processing.

Abstract: Disclosed is a three-dimensional (“3D”) scanning system that synchronizes the scanning of a scene with the viewing of the scan results relative to a live view of the scene. The system includes a first device that scans a first set of surfaces that are exposed to the first device from a first position. The system further includes a second device that receives the scan data as it is generated for each scanned surface of the first set of surfaces. The second device augments a visualization of a second set of surfaces, within a field-of-view of the second device from a second position, with the scan data that is generated for a subset of scanned surfaces from the first position corresponding to one or more surfaces of the second set of surfaces visualized from the second position.

Abstract: This information processing device includes: a control device which determines whether or not a first terminal is in a first position in a real space, controls, on the basis of the determination that the first terminal is in the first position, the first terminal to start superimposing a virtual object according to the first position on the real space, determines whether or not the second terminal different from the first terminal is in the first position, and controls the second terminal to start superimposing the virtual object on the real space, on the basis of the determination that the second terminal is in the first position.

Abstract: A system and method for texture decompression is described. The method comprises receiving a first compressed texture block including two or more disjoint subsets of data and decompressing the first compressed texture block. The decompressing includes decompressing the two or more disjoint subsets in the first compressed texture block to form texels. The two or more disjoint subsets include a first disjoint subset comprising a first set of color endpoints and a second disjoint subset comprising a second set of color endpoints.

Abstract: In one implementation, a method of performing late-stage shift is performed at a device including a display, one or more processors, and non-transitory memory. The method includes generating, based on a first predicted pose of the device for a display time period, a first image. The method includes generating a mask indicating a first region of the first image and a second region of the first image. The method includes generating a second image by shifting, based on a second predicted pose of the device for the display time period, the first region of the first image without shifting the second region of the first image. The method includes displaying, on the display at the display time period, the second image.

Abstract: A method and computer processing system for performing texture compression comprising receiving, from a memory storing a compressed texture block and by a graphics processing unit including at least one rendering pipeline, the compressed texture block including two or more disjoint subsets, and decompressing, by the at least one rendering pipeline, the compressed texture block, wherein decompressing the compressed texture block comprises: decompressing data in the two or more disjoint subsets in the compressed texture block to form texels, wherein the two or more disjoint subsets include a first disjoint subset including a first set of color endpoints and a second disjoint subset including a second set of color endpoints.

Abstract: A method is provided for displaying an augmented reality (AR) representation of physical effects and property damage resulting from a parametric earthquake event. The method includes scanning, using one or more sensors of a user device, a scene in proximity to a user. The method also includes identifying a background and objects in the scene. The method further includes creating an AR background for the background of the scene and AR objects for the objects in the scene. In addition, the method includes receiving at least one seismic characteristic from the user through the display of the user device. The method additionally includes displaying at least one seismic effect on the AR objects and the AR background in the scene based on the at least one received seismic characteristic.

Abstract: An image synthesis system comprises receivers (201, 203, 205) receiving scene data describing at least part of a scene; object data describing a 3D object from a viewing zone having a relative pose with respect to the object, and a view pose in the scene. A pose determiner circuit (207) determines an object pose for the object in the scene in response to the scene data and the view pose; and a view synthesis circuit (209) generates a view image of the object from the object data, the object pose, and the view pose. A circuit (211) determines a viewing region in the scene which corresponds to the viewing zone for the object being at the object pose. The pose determiner circuit (207) determines a distance measure for the view pose relative to the viewing region and changes the object pose if the distance measure meets a criterion including a requirement that a distance between the view pose and a pose of the viewing region exceeds a threshold.

Abstract: Various implementations disclosed herein include devices, systems, and methods that provide XR in which virtual objects are positioned based on the accuracy of localizing an electronic device in a physical environment. In some implementations, the technique assesses the accuracy of localization (e.g., centimeter-level accuracy, room-level accuracy, and building-level accuracy) and dynamically adjusts a display strategy. In some implementations, the technique determines a condition causing inaccuracy (e.g., a semantic condition such as “too fast”, “too far”, “too dark”), and provides a notification (e.g., “too fast-slow down”, “too far-move closer”, “too dark-turn on a light”) at the electronic device based on the condition causing the inaccuracy in the localization.