Abstract: Disclosed are an augmented reality-based coloring method and apparatus, an electronic device and a storage medium. The method comprises: acquiring three-dimensional mesh information of a human face, and acquiring position coordinates and texture coordinates of respective vertices of a target portion of the human face based on the three-dimensional mesh information of the human face; constructing a three-dimensional mesh of the target portion according to the position coordinates of respective vertices of the target portion, and determining a normal line of each vertex in the three-dimensional mesh; and coloring the target portion based on the position coordinates, the texture coordinates and a normal line of each vertex in the three-dimensional mesh, a preset shading map of the target portion, and a target RGB value of the target portion, thereby improving the stereoscopic impression and the sense of reality of coloring, and improving the user experience.

Abstract: The present disclosure provides a method of medical procedure using augmented reality for superimposing a patient's medical images (e.g., CT or MRI) over a real-time camera view of the patient. Prior to the medical procedure, the patient's medical images are processed to generate a 3D model that represents a skin contour of the patient's body. The 3D model is further processed to generate a skin marker that comprises only selected portions of the 3D model. At the time of the medical procedure, 3D images of the patient's body are captured using a camera, which are then registered with the skin marker. Then, the patient's medical images can be superimposed over the real-time camera view that is presented to the person performing the medical procedure.

Abstract: An augmented-reality system includes an image capture device that obtains a captured image, an image processing unit that receives and processes the captured image to determine space configuration of a scene; a user interface that receives a position of a fiducial point inputted by a user; and a rendering device that superimposes a fiducial mark at the position of the fiducial point on the captured image. The rendering device superimposes a first computer-generated image representing a sensor at an install height above the fiducial point and on a wall of the captured image.

Abstract: A location-based platform for hosting multiple 3D engines delivering location-based 3D content. An engine platform server system is configured to host one or more engine servers hosting 3D engines and to provide an engine platform comprising digital reality applications developed via the 3D engines. The digital reality applications are configured in pre-determined locations within a persistent virtual world system or in mobile virtual replicas of corresponding real-world objects. Client devices approaching a digital reality application send the viewing position and orientation along with a view request to the engine platform server, which requests virtual frames from respective 3D engines, composites each virtual frame into one view, and sends the resulting media stream to users via the client devices.

Abstract: An application is presented for providing a highly accurate and realistic 3D virtual model of monitored items. The 3D virtual model may include virtual icons that represent a set of sensors. Data visualizations are provided in the 3D virtual model depicting relevant information about the monitored items and the set of sensors. Alarms can be triggered if the sensor exceeds normal operating thresholds and a user directed to a specific sensor that is malfunctioning or subject to the alert. The user can rewind through the virtual model to view events leading up to the alarm or malfunctioning of the monitored item. Further, a 3D virtual model can be reciprocally rendered as a digital twin on a second user's computing device and any extended reality graphical manipulation shared with the second user whether the second user is physically adjacent to a monitored item or remote from a monitored item.

Abstract: Techniques related to synthesizing an image of a person in an unseen pose are discussed. Such techniques include detecting a body part occlusion for a body part in a representation of the person in a first image and, in response to the detected occlusion, projecting a representation of the body part from a second image having a different view into the first image. A geometric transformation based on a source pose of the person and a target pose is then applied to the merged image to generate a synthesized image comprising a representation of the person in the target pose.

Abstract: In some implementations, a method includes: obtaining a representation an environment; determining a plurality of candidate content placement locations within the environment based on the representation of the environment; determining characterization parameters for the plurality of candidate content placement locations; obtaining extended reality (XR) content selected based on a match between content parameters associated with the XR content and characterization parameters for a respective candidate content placement location among the plurality of candidate content placement locations; and displaying, via the display device, the XR content at the respective candidate content placement location within the environment.

Abstract: A technology that streams graphical components and rendering instructions to a client device, for the client device to perform the final rendering and overlaying of that content onto the client's video stream based on the client's most recent tracking of the device's position and orientation. A client device sends a request for augmented reality drawing data to a network device. In response, the network device generates augmented reality drawing data, which can be augmented reality change data based on the augmented reality information and previous client render state information, and sends the augmented reality drawing data to the client device. The client device receives the augmented reality drawing data and renders a visible representation of an augmented reality scene comprising overlaying augmented reality graphics over a current video scene obtained from a camera of the client device.

Abstract: Disclosed herein are methods and systems for registering an Augmented Reality (AR) scene to a 3D model of a certain area, comprising accumulating a geometrical dataset of a certain area based on visual data captured by imaging sensors during an AR session, obtaining a 3D model of the certain area, computing a similarity score for the geometrical dataset compared to the 3D model rotated in several candidate principal rotation angles based on parallelism of corresponding planes identified in the geometrical dataset and in the rotated 3D model, selecting a highest score principal rotation angle, computing a translation vector for each of a plurality of fine-tune rotation angles with respect to the selected principal rotation angle based on a distance between the corresponding planes, selecting a lowest distance fine-tune rotation angle, and registering the geometrical dataset to the 3D model according to the selected fine-tune rotation angle and its translation vector.

Abstract: A system for mixed-reality interaction with a touch device may include a mixed-reality device including a sensor, a mapping device, a simulator, and a display and a touch device including an input device. The input device of the touch device and the sensor of the mixed-reality device may detect a human-touch-device interaction. The mapping device may calibrate, correlate, and map the human-touch-device interaction from a real-world environment to a mixed-reality environment. The input device of the touch device may receive an input based on the human-touch-device interaction. The simulator of the mixed-reality device may render a view of the human-touch-device interaction within the mixed-reality environment and a view of an action within the mixed-reality environment corresponding to the input determined based on the human-touch-device interaction. The display of the mixed-reality device may display the rendered views.

Abstract: Provided is an augmented reality technology configured to combine images in a virtual space in real-time. An augmented reality display device includes an imaging unit that acquires a background image of a real world, a model control unit that controls a two-dimensional or three-dimensional model in a virtual space, and a display unit that displays the model together with the background image. The model includes one or a plurality of joints connectable to another model. The model control unit controls each model to connect a joint of a first model and a joint of a second model in the virtual space.

Abstract: A three-dimensional face model constructing method includes: obtaining front face information to establish a three-dimensional front face model and generate a front planar image corresponding to the three-dimensional front face model, wherein the front planar image includes first feature points; obtaining full face information to establish a three-dimensional full face model and generate a full face planar image corresponding to the three-dimensional full face model, wherein the three-dimensional full face model includes a specific portion and the full face planar image includes second feature points; superimposing a first block of the front plan image to a second block of the full face planar image according to the correspondence between the first feature points and the second feature points, to obtain a complete full face planar image; and generating a three-dimensional face model according to the complete full face planar image.

Abstract: A method of guiding a user when performing a three-dimensional scan of an object includes determining two-dimensional feature points from two-dimensional captured images of the object, determining three-dimensional coordinates for the two-dimensional feature points, determining a cut plane based on the three-dimensional coordinates that divides the object into a target portion and a cut-away portion, and displaying the target portion of the object on a display using a first indicium and the cut-away portion of the object on the display using a second indicium that is different from the first indicium.

Abstract: A firearm system includes a firearm and a computer. Electronics in the firearm determine data that includes a pathway between different points of aim of the firearm as the firearm moves. The computer receives this data and builds an image of the pathway between the different points of aim of the firearm.

Abstract: A system and a method allowing to make changes on a desired model and/or a scene by converting models and/or scenes in different formats into a unique format and by integrating them into Augmented Reality and Virtual Reality/Mixed Reality and other emerging technologies and enables to be displayed of the changes by other users simultaneously are provided.

Abstract: A merged reality system comprises at least one server storing a virtual world system comprising one or more virtual objects, the virtual objects including virtual replicas of at least a first location, a second location, and real world elements in the at least first and second locations. The at least one server is configured to receive, from a plurality of connected devices communicating with the at least one server through a network, real-world data from real-world elements in the first and second locations; use the real-world data from the first and second locations in the virtual world system to enrich and synchronize the virtual replicas with corresponding real-world elements; and overlap and stream at least one portion of the real-world data from the second location onto, e.g., one or more surfaces of the virtual replica of the first location.

Abstract: Mini-Map with Gaze Guidance Line Visualization Module enables new types and classes of Human Computer Interface enhancements where the Humans can easily and simultaneously see where objects are relative within a 2D map in the foreground connected via intuitive gaze guidance lines to the same object with the Augmented Reality view. The Mini-Map is fully manipulatable position, zoom, display angle with defaults being relative to the user.

Abstract: In one implementation, a method includes: instantiating a first objective-effectuator (OE) associated with first attributes and a second OE associated with second attributes into a synthesized reality (SR) setting, wherein the first OE is encapsulated within the second OE; providing a first objective to the first OE based on the first and second attributes; providing a second objective to the second OE based on the second attributes, wherein the first and second objectives are associated with a time period between a first and second temporal points; generating a first set of actions for the first OE based on the first objective and a second set of actions for the second OE based on the second objective; and rendering for display the SR setting for the time period including the first set of actions performed by the first OE and the second set of actions performed by the second OE.

Abstract: A method to efficiently update and manage outputs of real time or offline 3D reconstruction and scanning in a mobile device having limited resource and connection to the Internet is provided. The method makes available to a wide variety of mobile XR applications fresh, accurate and comprehensive 3D reconstruction data, in either single user applications or multi-user applications sharing and updating the same 3D reconstruction data. The method includes a block-based 3D data representation that allows local update and maintains neighbor consistency at the same time, and a multi-layer caching mechanism that retrieves, prefetches, and stores 3D data efficiently for XR applications. Altitude information, which may be expressed as a building floor for indoor environments, may be associated with sparse and/or dense representations of the physical world, to increase the accuracy of localization results and/or in rendering virtual content more realistically.

Abstract: A method relating to ATM repair is provided. The method includes: receiving, by an electronic device associated with a repair provider, diagnostic data from an ATM; capturing, by the electronic device, an image or a video of the ATM; initiating, by the electronic device, a communication session with a remote location computing system; transmitting, by the electronic device and to the remote location computing system, at least one of the image or the video of the ATM and the diagnostic data; receiving, by the electronic device and from the remote location computing system, repair instructions including a modified version of the at least one image or video; outputting, by the electronic device, the repair instructions; receiving, by the electronic device, a user input identifying a component of the modified version of the at least one image or video; and outputting, by the electronic device, diagnostic information for the identified component.