Abstract: An example of an apparatus to infer joint rotations and positions is provided. The apparatus includes a communications interface to receive raw data from an external source. The raw data includes a first joint position and a second joint position of an input skeleton. In addition, the apparatus includes a memory storage unit to store the raw data. Furthermore, the apparatus includes a pre-processing engine to generate normalized data from the raw data. The normalized data is to be stored in the memory storage unit. Also, the apparatus includes an inverse kinematics engine to apply a neural network to infer a joint rotation from the normalized data. The neural network is to use historical data. Training data used to train the neural network includes positional noise.

Abstract: Described are improved systems and methods for navigation and manipulation of interactable objects in a 3D mixed reality environment. Improved systems and methods are provided to implement physical manipulation for creation and placement of interactable objects, such as browser windows and wall hangings. A method includes receiving data indicating a selection of an interactable object contained within a first prism at the start of a user interaction. The method also includes receiving data indicating an end of the user interaction with the interactable object. The method further includes receiving data indicating a physical movement of the user corresponding to removing the interactable object from the first prism between the start and the end of the user interaction. Moreover, the method includes creating a second prism to contain the data associated with the interactable object at the end of the user interaction with the interactable object.

Abstract: An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

Abstract: Augmented reality systems provide graphics over views from a mobile device for both in-venue and remote viewing of a sporting or other event. A server system can provide a transformation between the coordinate system of a mobile device (smart phone, tablet computer, head mounted display) and a real world coordinate system. Requested graphics for the event are displayed over a view of an event.

Abstract: The invention relates to a system and computer-implemented method for enabling correction of a segmentation of an anatomical structure in 3D image data. The segmentation may be provided by a mesh which is applied to the 3D image data to segment the anatomical structure. The correction may for example involve a user directly or indirectly selecting a mesh part, such as a mesh point, that needs to be corrected. The behaviour of the correction, e.g., in terms of direction, radius/neighbourhood or strength, may then be dependent on the mesh normal direction, and in some embodiments, on a difference between the mesh normal direction and the orientation of the viewing plane.

Abstract: Generative image synthesis conditioned on label prompts includes receiving a label prompt comprising a mapping of labels to portions of an image to be synthesized. At least some of the labels in the label prompt correspond to an object to be included in the image to be synthesized. It further includes, based on the label prompt, generating, using a generative model, a synthesized image corresponding to the label prompt.

Abstract: A high-precision map construction method, apparatus, and electronic device are provided. The method can include: displaying a first color image corresponding to a first track point; according to a first color sub-image and a depth image corresponding to the first track point, obtaining point cloud data corresponding to the first sub-color image, wherein the first sub-color image is a sub-image corresponding to an element to be added in the first color image, and the element to be added is an element to be added in a high-precision map for display; extracting a bounding box corresponding to the point cloud data; and generating a newly-added three-dimensional element according to the bounding box in the high-precision map.

Abstract: A modeling system stores a three-dimensional (3D) virtual object that corresponds to a real-world object, the 3D virtual object including a superimposition of an image area showing a portion of the real-world object on a face of a 3D shape. The modeling system stores a virtual reset that includes information about the 3D virtual object and a position of the 3D virtual object in the virtual reset. The modeling system presents the virtual reset, the presentation showing the 3D virtual object at the position.

Abstract: A method for three-dimensional (3D) imaging of an underground goaf includes: obtaining a video image and point cloud data of an underground goaf; determining a posture transformation matrix, and aligning the video image and the point cloud data; identifying a material texture based on the video image to obtain material texture information; dividing the point cloud data into triangular meshes to form an initial 3D model of the underground goaf; mapping the material texture information onto the 3D model to form a target 3D model with a material texture attribute; calculating a bidirectional reflectance distribution function (BRDF) of the target 3D model based on the material texture attribute; simulating global illumination, and constructing a rendering equation; and making a setting to emit ray from a viewpoint, and performing ray tracing based on the rendering equation to obtain a high-brightness real scene image of the underground goaf.

Abstract: An augmented reality-based content authoring tool is presented. A content author arranges machine-recognizable markers in a physical environment. A computing device operating as the authoring tool recognizes the markers and their arrangement based on a captured digital representation of the physical environment. Once recognized, augmented reality primitives corresponding to the markers can be bound together via their primitive interfaces to give rise to a content set. The individual primitives and content set are instantiated based on the nature of the marker's arrangement.

Abstract: Disclosed is a mobile terminal that provides an augmented reality navigation screen in a state of being hold in a vehicle, the mobile terminal including: at least one camera configured to obtain a front image; a display; and at least one processor configured to calibrate the front image, and to drive an augmented reality navigation application so that the augmented reality navigation screen including at least one augmented reality (AR) graphic object and the calibrated front image is displayed on the display.

Abstract: A system and method utilizing a canonical model for monitoring fixed entities, transit entities, and/or moveable objects include a circuit having one or more data collectors and a configuration database in communication with the circuit. The configuration database has a plurality of location asset designators (“LADs”) and modal asset designators (“MADs”), and/or sensor asset designators (“SADs”). The circuit is configured to periodically receive information from the transit entities and update the MADs, SADs, and/or LADs of the canonical model. Location information regarding the LADs, MADs, and/or SADs may be provided in a word-based format, such as what3words.

Abstract: Embodiments provide a method for implementing augmented reality interaction and presentation, a smart eyewear apparatus, a split-mount device, and a control device, wherein the smart eyewear apparatus implements a better user interaction experience with a linking between online information and offline information and a fusion of virtuality and reality by: establishing a communication connection with a split-mount device based on a communication protocol; transmitting relevant control information to the split-mount device based on the communication protocol; obtaining split-mount feedback data transmitted by the split-mount device based on the communication protocol; and presenting a corresponding augmented reality effect based on the split-mount feedback data, wherein the augmented reality effect includes a virtual image displayed in cooperation with a real scene, a voice effect played, and a vibration effect.

Abstract: A display device includes: a display panel including: a functional area defined by an opening formed in the display panel; and a display area defined as an area where a plurality of light emitting elements are arranged, surrounding the functional area; and a functional module including a lens disposed overlapping the functional area, wherein only some of the plurality of light emitting elements of the display panel are configured to emit light in a low power mode, and wherein all of the plurality of light emitting elements are configured to emit light in a general mode; and wherein the display device is configured to, in the low power mode, provide a predetermined information to a user formed by the boundary between the functional area and the display area and an image formed by light emission of the some of the plurality of light emitting elements.

Abstract: In one embodiment, a method is provided. The method includes determining a set of spheres for a volume of a material. The volume of the material comprises the set of spheres and additional materials. The sizes of the set of spheres are based on a Gaussian mixture model (GMM). The method also includes determining a set of locations for the set of spheres within the volume of the material. The method further includes generating a set of images of the volume of the material based on a first generative adversarial network and a second generative adversarial network. The set of images depict a microstructure of the volume of material.

Abstract: Systems and methods are provided to aid in planning at least a portion of a total knee arthroplasty procedure. The system and method automatically aligns the implant components and the bones according to a desired clinical alignment goal with minimum user input. The system and method further allows the user to adjust the position and orientation of the femur, tibia, or implant in a clinical direction regardless of a pre-adjusted position and orientation of the femur, tibia, or implant. The graphical user interface is provided that includes a three-dimensional (3-D) view window, a view options window, a patient information window, an implant family window, a workflow-specific tasks window, and a limb and knee alignment measures window.

Abstract: A surgical system includes an XR headset and an XR headset controller. The XR headset is configured to be worn by a user during a surgical procedure and includes a see-through display screen configured to display an XR image for viewing by the user. The XR headset controller is configured to receive a plurality of two-dimensional (“2D”) image data associated with an anatomical structure of a patient. The XR headset controller is further configured to generate a first 2D image from the plurality of 2D image data based on a pose of the XR headset. The XR headset controller is further configured to generate a second 2D image from the plurality of 2D image data based on the pose of the XR headset. The XR headset controller is further configured to generate the XR image by displaying the first 2D image in a field of view of a first eye of the user and displaying the second 2D image in a field of view of a second eye of the user.

Abstract: A system and a method are disclosed for post-processing variable pixel rate shader output using gradients in a graphics processing unit. A block of pixels is selected that corresponds to a predetermined kernel size for variable rate shading in a draw call of an application. A pixel shader run is instantiated to generate pixel shading values for at least two pixels located within the block of pixels. A gradient output is generated based on an interpolation of the pixel shading values for the at least two pixels over the block of pixels. The predetermined kernel size may include at least one of a 4×2 block of pixels, a 2×4 block of pixels, a 4×4 block of pixels, an 8×4 block of pixels, a 4×8 block of pixels, and an 8×8 block of pixels or larger. The at least two pixels may be corner pixels of the block of pixels.

Abstract: The present invention provides a method and a system for pixelating a vector graphic into an image. First, a vector graphic defined by an arbitrary curve boundary is approximated as a vector graphic of a Manhattan structure, and then a fast algorithm is used to pixelate the approximate Manhattan vector graphic into a two-dimensional image. A process of moving quadrant planes to superimpose on vertices of the vector graphic is replaced with convolution of an image resulting from a template rectangle image and Dirac pulse functions of the vertices of the vector graphic, and a pixelated image of this vector graphic is obtained through convolution of the template rectangle and a sparse image composed of Dirac pulses located at the positions of the vertices of the vector graphic.

Abstract: Systems, apparatuses and methods may provide away to render augmented reality (AR) and/or virtual reality (VR) sensory enhancements using ray tracing. More particularly, systems, apparatuses and methods may provide a way to normalize environment information captured by multiple capture devices, and calculate, for an observer, the sound sources or sensed events vector paths. The systems, apparatuses and methods may detect and/or manage one or more capture devices and assign one or more the capture devices based on one or more conditions to provide observer an immersive VR/AR experience.