Abstract: The method of performing an operation on a body part of a patient uses a mixed reality (MR) device worn by the user. A visual representation of an operating scene includes a virtual anatomical feature representing an actual anatomical feature of the patient. A position and/or orientation of the visual representation relative to the anatomical feature of the patient is first adjusted to be superposed with the anatomical feature of the patient. The visual representation is displayed by the MR device superposed with the anatomical feature of the patient in a field of vision of the user. The position and/or orientation of the visual representation is repeatedly or continuously adjusted, so that, when the MR device moves relative to the anatomical feature of the patient, the visual representation remains superposed with the anatomical feature of the patient.

Abstract: This application discloses a secondary light occlusion rendering method performed by a computer device. The method includes: obtaining direction information of a secondary light in an environment including a virtual object and data of a plurality of vertexes on a model of the virtual object; decoding the encoding results based on the direction information to obtain decoded vertex occlusion information; and obtaining secondary light occlusion rendering data of the virtual object based on the decoded vertex occlusion information respectively corresponding to the plurality of vertexes and light intensity information of the secondary light. In this way, textures do not need to be read frame by frame, thereby reducing consumption of performance of a computing device. The rendering solution is based on vertexes on a model and is not affected by a change in a movement direction of a virtual object.

Abstract: An augmented reality. AR, computing server (200) includes a network interface (202), a processor (204), and a memory (206) storing instructions executable by the processor to perform operations. The network interface is configured to receive through a network a three-dimensional (3D) video stream from a user device during a conference session. The operations identify a video object captured in the 3D video stream, and determine a pose of the video object captured in the 3D video stream. The operations obtain AR context information from an AR display device indicating how the video object is to be posed relative to a physical object viewable through a see-through display of the AR display device, and adjust pose of the video object captured in the 3D video stream based on the AR context information. The operations output the video object to the see-through display for display. Related methods and computer program products are disclosed.

Abstract: According to an embodiment of the disclosure, the method may include detecting the at least one real-world object, detecting an occurrence of an event associated with the at least one real-world object in proximity of an user in a real world, predicting at least one user action in the real world subsequent to the occurrence of the event based on a movement of an user in the real world for interacting with the at least one real world object and generating an overlay of the at least one real-world object within the ongoing virtual session based on a position of at least one real world object and a position of the user.

Abstract: Described herein is a process and system for constructing three-dimensional (3D) representations of roof structures. The system can create representations of roof structures of arbitrary complexity and can create representations of dependent roof structures such as dormers. The system can create representations of roof structures that conventional methods cannot create, such as roofs with edges that are not coplanar, roofs with faces that do not connect to exterior edges, roofs composed of sub-structures, or roofs with dependent structures such as dormers.

Abstract: The present disclosure generally relates to methods and interfaces for capturing media with a camera application. In some examples, a computer system captures media based on gaze, modifies video playback to improve viewing comfort, and/or surfaces a view setting for media playback based on media stability characteristics. In some examples, a computer system displays one or more of: a camera preview with a level indicator, a camera preview for spatial media capture with prompts to improve capture quality, a camera preview for spatial media capture with a camera movement indicator, and/or a camera preview for media capture with viewpoint stability guidance.

Abstract: A motion diffusion model may be pre-trained on motion data, and a scene-aware component (e.g., one or more layers of a neural network) may be connected and used to extract and inject a representation of scene information into the pre-trained motion diffusion model. For example, to predict orientations of joint waypoints along a path through a particular 3D scene, a scene-aware input channel that accepts a representation of the 3D structure of the scene may be added to a pre-trained motion diffusion model. To predict orientations of joint waypoints along a path that interacts with a 3D object in the 3D scene, a scene-aware input channel that accepts a representation of the 3D object and/or a surface thereof may be added to a pre-trained motion diffusion model. As such, the resulting scene-aware motion diffusion model(s) may be tuned on motion-scene data and used to generate human motion.

Abstract: Disclosed are systems, computer-program products, and computer-implemented methods for the automatic estimation of geographic boundaries. Implementations of the foregoing may be useful for automatedly determining the geographic boundary of a community. In some embodiments, a computing device may receive birth location data related to the community. Enriched birth locations for the community may be determined based on likelihood metrics. An estimated geographic boundary of the community may be determined. The estimated geographic boundary may correspond to a probability density of the enriched birth locations represented using the geographical coordinates. A graphical user interface may present a map and the estimated geographic boundary of the community overlaying the map.

Abstract: Techniques are disclosed for facilitating electronic commerce in an augmented reality environment. In some embodiments, a method comprises detecting, by a mobile device, presence of the physical product or the real life service; and presenting, on the mobile device, information to conduct the transaction of a physical product or a real life service via the augmented reality environment. In some embodiments, a method comprises detecting one or more targets in the augmented reality platform using a select area in a perspective of a user, the perspective being captured by a mobile device; and prompting the user to choose an object of interest from the one or more detected targets.

Abstract: Disclosed in the embodiments of the present disclosure are a three dimension (3D) target point rendering method, apparatus, a device and a storage medium. The method comprises: selecting and determining a preset number of mesh vertexes in a target model as initial 3D target points; acquiring first depth information of the initial 3D target points, and determining a non-hidden 3D target point based on the first depth information; determining a display size and a rotation angle of the non-hidden 3D target point; and rendering the non-hidden 3D target points based on the display size and the rotation angle.

Abstract: The present invention includes vibrating a second presentation visual object with respect to a first presentation visual object so as to include a vibration component of a specific control frequency, the second presentation visual object being in contact with or in proximity to the first presentation visual object, and making an apparent force appear to be applied from the second presentation visual object to the first presentation visual object.

Abstract: The present invention embodies a novel navigational system that enhances the experience of pilots and passengers of physical aircraft (and other physical vehicles) traveling through actual three-dimensional space. In addition to facilitating the creation of a virtual course through which physical aircraft will travel while flying through actual three-dimensional space, the present invention also facilitates the creation of a virtual environment (modeling remote, inaccessible and imaginary locations) into which the virtual course is integrated, and manages the synchronization of the physical aircraft to the virtual course within the virtual environment during actual physical flight. In this manner, the present invention enhances the experience of pilots and passengers by illustrating dynamic views of the physical aircraft traveling along the virtual course within the virtual environment. The present invention enables competitive, sightseeing, educational, entertaining and various other types of experiences.

Abstract: In some embodiments, a computer system facilitates depth conflict mitigation for a virtual object that is in contact with one or more physical objects in a three-dimensional environment by changing visual properties of one or more portions of the virtual object. In some embodiments, a computer system facilitates depth conflict mitigation for a virtual object that is in contact with one or more physical objects in a three-dimensional environment by displaying the virtual object in a virtual environment within the three-dimensional environment. In some embodiments, a computer system facilitates depth conflict mitigation for a virtual object that is in contact with one or more physical objects in a three-dimensional environment by variably changing visual properties of one or more portions of the virtual object and/or by variably displaying the virtual object in a virtual environment based on one or more characteristics of the depth conflict.

Abstract: A method and apparatus for enhancing user interaction within a virtual environment by linking physical objects to corresponding virtual manifestations. The method comprises: receiving, from a scanning device, an image of a pattern on a surface of a physical object; analyzing the image to recognize the pattern and associate the pattern with corresponding virtual content stored in data storage; rendering, by the computer system, a virtual manifestation within the virtual environment; receiving, from the scanning device, an indication of an occlusion of the pattern caused by user interaction with the physical object; interpreting the occlusion based on a user input to the virtual environment; modifying the virtual manifestation in response to the user input; updating the virtual manifestation in real-time to reflect movements of the physical object.

Abstract: Techniques, which may be embodied herein as systems, computing devices, methods, algorithms, software, code, computer readable media, or the like, are described herein for generating heatmap visualizations. A heatmap visualization can be a graphical representation of data where the coordinates of a cell in the visualization can be a key that describes a group of data represented by the cell. The colors of a heatmap may indicate information about the data represented by the cell (e.g., the amount of data in the cell). These cells can be rendered by calculating a per-pixel metric rate for each dataset in the heatmap visualization and mapping a dataset's rate onto pixels within the visualization that correspond to the dataset. After this mapping, the maximum rate within the visualization can be used to determine the visualization's colors and the cells can be rendered by coloring each pixel (e.g. pixel row) in its corresponding color.

Abstract: A system for digital twinning a lift device includes a lift device, and processing circuitry. The lift device includes at least one of a sensor, a device, or an electronic control system. The processing circuitry is configured to obtain machine-specific data from the at least one of the sensor, device, or the electronic control system of the lift device. The processing circuitry is also configured to obtain a user input regarding a requested configuration of the lift device. The processing circuitry is further configured to generate a virtual lift device using the requested configuration of the lift device and the machine-specific data obtained from the lift device. The processing circuitry is further configured to operate a user device to present the virtual lift device according to the requested configuration in a virtual environment of a webpage or in an augmented reality environment.

Abstract: Provided is an information processing apparatus including a display control unit and a fitting unit. The display control unit displays a distribution image indicating distribution of a plurality of marker points extracted from a face model of a performer in a superimposed manner on a face image of the performer. The fitting unit fits the face model to the face image by using position information of a placed marker each time the marker is placed on a face of the performer after the distribution image is superimposed, and adjusts a superimposed position of the distribution image based on a fitting result.

Abstract: A wearable terminal apparatus to be worn by a user for use includes at least one processor, a camera, and a communication unit. The camera images a space as a visible region of the user. The communication unit performs data communication with an external device to be used by a remote instructor. The at least one processor transmits an image obtained by the camera to the external device via the communication unit. The at least one processor receives instruction information related to an instruction to generate an instruction image via the communication unit, the instruction being made based on the image in the external device. The at least one processor causes a display unit to display the instruction image, the instruction image being visually recognized at a position based on the received instruction information in the space. The at least one processor accepts a user operation to change the displayed instruction image and updates display of the instruction image in accordance with the user operation.

Abstract: The application discloses a method, electronic device, and storage medium for image processing, which includes: in response to determining that the trigger of an effect display function is detected, adding a virtual model to a collected image to be processed to obtain a presentation video frame; magnifying-displaying an image area of the virtual model in the presentation video frame, and processing the virtual model into a target three-dimensional effect model in response to determining that a magnification stop condition is reached; fusing the target three-dimensional effect model and a target object in the image to be processed to present a target video frame.

Abstract: Systems and methods for occluding virtual objects in an augmented reality environment are described. A method includes generating a distance map associated with a physical object, determining a location of a virtual object in an artificial-reality environment, determining, based at least in part upon the distance map and the location of the virtual object, a first portion of the physical object that is closer than a first portion of the virtual object from the viewpoint of the user, determining a second portion of the virtual object that is not obscured by the physical object from the viewpoint of the user, and rendering a display image. The display image comprises a first set of pixels corresponding to the first portion of the physical object in a first state, and the display image comprises a second set of pixels corresponding to the second portion of the virtual object in a second state.