Abstract: A method includes accessing image data and depth data corresponding to image frames to be displayed on an extended reality (XR) display device, and determining sets of feature points corresponding to the image frames based on a multi-layer sampling of the image data and the depth data. The method further includes generating a set of sparse feature points based on an integration of the sets of feature points. The set of sparse feature points are determined based on relative changes in depth data with respect to the sets of feature points. The method further includes generating a set of sparse depth points based on the set of sparse feature points and the depth data and sending the set of sparse depth points to the XR display device for reconstruction of a dense depth map corresponding to the image frames utilizing the set of sparse depth points.

Abstract: Systems and method to implement a geometry processing phase of tile-based rendering. The systems include a plurality of parallel geometry pipelines, a plurality of tiling pipelines and a geometry to tiling arbiter situated between the plurality of geometry pipelines and the plurality of tiling pipelines. Each geometry pipeline is configured to generate one or more geometry blocks for each geometry group of a subset of ordered geometry groups; generate a corresponding primitive position block for each geometry block, and compress each geometry blocks to generate a corresponding compressed geometry block. The tiling pipelines are configured to generate, from the primitive position blocks, a list for each tile indicating primitives that fall within the bounds of that tile. The geometry to tiling arbiter is configured to forward the primitive position blocks generated by the plurality of geometry pipelines to the plurality of tiling pipelines in the correct order based on the order of the geometry groups.

Abstract: Provided is a display apparatus including an image forming optical system configured to form an image to be displayed, an eyepiece optical system configured to provide the image formed by the image forming optical system to a pupil of an observer, and an image shifting optical system disposed on an optical path between the image forming optical system and the eyepiece optical system, the image shifting optical system being configured to shift the image formed by the image forming optical system in a direction perpendicular to an optical axis, wherein the image shifting optical system includes a first optical member having a first focal length and a second optical member having a second focal length, and wherein a distance between the first optical member and the second optical member along the optical axis is equal to a sum of the first focal length and the second focal length.

Abstract: In some implementations, a method performed by data processing apparatuses includes receiving map data that identifies a layout of physical objects within a physical area, identifying contiguous groups of the physical objects as composite objects, selecting bounding algorithms to use for generating graphical shapes for the composite objects, generating graphical shapes by applying the selected bounding algorithms to the composite objects, and testing the graphical shapes against one or more criteria. For candidate graphical shapes that fail a test, a new bounding algorithm can be selected and the generating and the testing can be repeated using the new bounding algorithm. A simplified graphical map can be output that represents the physical objects within the physical area using the graphical shapes.

Abstract: An automatic threat recognition system and method is disclosed for scanning the x-ray CT image of an article to identify the objects of interest (OOIs) contained within the article, which are otherwise not always quickly apparent or discernable to an individual. The system uses a computer to receive information from two-dimensional (2D) image slices from a reconstructed computed tomography (CT) scan image and to produce a plurality of voxels for each slice of the 2D image. The computer analyzes the voxels to create a likelihood map (LM) representing likelihoods that voxels making up the CT image are associated with a material of interest (MOI). The computer further analyzes the LM to construct neighborhoods of voxels within the LM, and classifies each voxel neighborhood based on its features, thereby decluttering the LM to facilitate the process of connecting voxels of a like MOI together to form segments.

Abstract: A method, computer system, and a computer-readable medium for registering one or more structures to a desired orientation for planning and guidance for surgery is provided. The method includes in a preoperative stage, obtaining one or more 3D models of one or more structures from one or more CT images using an image processing segmentation technique or a manual segmentation technique; in the preoperative stage, registering the one or more structures to a template that is adapted to an alternating registration for a patient-specific shape and pose for a desired reduction and corresponding reduction transformations; in an intraoperative stage, mapping the one or more structures to one or more radiographs via a 3D-2D registration that iteratively optimizes a similarity metric between acquired and simulated radiographs; and in the intraoperative stage, providing an output that is representative of a radiograph or a 3D tomographic representation to provide guidance to a user.

Abstract: Disclosed is a method for displaying a panoramic image by an electronic device. According to an example embodiment of the present disclosure, a method for generating a panoramic image may comprise sensing a direction that a first side surface of the electronic device faces through a sensor included in the electronic device, displaying a first partial image of the panoramic image corresponding to the sensed direction of the first side surface, determining a direction corresponding to the reference view information with respect to the first partial image if information regarding the first partial image differs from reference view information indicating a reference view for the panoramic image, and providing information about the determined direction.

Abstract: A method for three-dimensional (3D) inspection of a workpiece for conformance to a specification includes scanning, by a 3D optical scanning device, a hole formed in a workpiece to generate a 3D point cloud of the hole defined in a 3-axis coordinate system of the 3D optical scanning device. The 3D point cloud includes 3D point cloud data that provides a profile of the hole. The method also includes translating the 3D point cloud data to generate translated 3D point cloud data that facilitates analysis of the 3D point cloud. The method further includes performing analysis of the hole using the translated 3D point cloud data to determine conformance of the hole with a specification and to detect anomalies associated with the hole.

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize hemispherical clamping for importance sampling of an image-based light (IBL) to generate a digital image of a virtual environment. For example, the disclosed systems identify a hemispherical portion of an IBL image that corresponds to a reflective surface location on a virtual object. The disclosed systems can then clamp the IBL image using one or more importance sampling algorithms to exclude portions of the IBL image outside of the hemispherical portion that do not contribute direct lighting onto the reflective surface location. The disclosed systems can further utilize the one or more importance sampling algorithms to efficiently sample a ray direction between the reflective surface location and the hemispherical portion of the IBL image. In certain embodiments, the disclosed systems use the sampled ray direction to generate a digital image rendering portraying the virtual object.

Abstract: Example implementations relate to scheduling of jobs for a plurality of graphics processing units (GPUs) providing concurrent processing by a plurality of virtual GPUs. According to an example, a computing system including one or more GPUs receives a request to schedule a new job to be executed by the computing system. The new job is allocated to one or more vGPUs. Allocations of existing jobs are updated to one or more vGPUs. Operational cost of operating the one or more GPUs and migration cost of allocating the new job are minimized and allocations of the existing jobs on the one or more vGPUs is updated. The new job and the existing jobs are processed by the one or more GPUs in the computing system.

Abstract: A computing system, a method, and a computer-readable storage medium for adjusting eye gaze are described. The method includes capturing a video stream including images of a user, detecting the user's face region within the images, and detecting the user's facial feature regions within the images based on the detected face region. The method includes determining whether the user is completely disengaged from the computing system and, if the user is not completely disengaged, detecting the user's eye region within the images based on the detected facial feature regions. The method also includes computing the user's desired eye gaze direction based on the detected eye region, generating gaze-adjusted images based on the desired eye gaze direction, wherein the gaze-adjusted images include a saccadic eye movement, a micro-saccadic eye movement, and/or a vergence eye movement, and replacing the images within the video stream with the gaze-adjusted images.

Abstract: A hybrid rendering HMI terminal device (32) is provided with a web browser (321) and an HMI Web Runtime (322). The web browser (321) displays an HMI screen on which a first part and a second part that represent the state of a monitoring target device (7) are arranged. When signal data from the monitoring target device (7) is analog numerical data, a WebGL rendering processing unit (322i) renders the first part associated with the signal data by WebGL rendering. When signal data from the monitoring target device (7) is data other than analog numerical data, an SVG rendering processing unit (322h) renders the second part associated with the signal data by SVG rendering.

Abstract: The present disclosure relates to methods and devices for graphics processing including an apparatus, e.g., a GPU. The apparatus may determine whether to divide a group of threads into a plurality of sub-groups of threads, each thread of the group of threads being associated with a shader program. The apparatus may also divide, upon determining to divide the group of threads into the plurality of sub-groups of threads, the group of threads into the plurality of sub-groups of threads. Additionally, the apparatus may execute, upon dividing the group of threads into the plurality of sub-groups of threads, a subsection of the shader program for each sub-group of threads of the plurality of sub-groups of threads.

Abstract: A method, computer device, and storage medium for exploring and collecting scene reconstruction data. The method includes: capturing a scene image; performing real-time environment perception on the scene image to detect object to be reconstructed in the scene image and reconstructing three-dimensional representation data of the object to be reconstructed; determining a reconstruction target in the object to be reconstructed based on the three-dimensional representation data of the object to be reconstructed and a preset distance threshold; generating an exploration path according to a preset exploration path planning algorithm and exploring an area to be reconstructed according to the exploration path; and collecting reconstruction data of the reconstruction target according to the preset exploration path planning algorithm when the reconstruction target is reached.

Abstract: The present disclosure is provides a method, an apparatus and a terminal device for constructing parts together. The method includes: determining an assembly progress of an object constructed of parts; determining a currently required part according to the assembly progress; identifying, in a part photo of the object, the currently required part using a pre-trained first neural network model and marking the one or more identified currently required parts in the part photo; and displaying, via a display, a 3D demonstration animation with the marked part photo as a background image. The method is based on the existing 3D demonstration animation, which identifies the currently required part in the captured part photo and marks the identified currently required part in the part photo, and then displays the marked part photo as the background image of the 3D demonstration animation, thereby adding a prompt for the real part.

Abstract: Techniques for composite imagery rendering in a diminished reality environment for medical diagnosis are provided. In one aspect, a method for composite image rendering in a diminished reality environment for medical diagnosis includes: analyzing medical imagery scans for a patient visiting a medical office for consultation with a physician for a diagnosis; obtaining real-time images of the patient who is physically located in the medical office; creating a composite image of the patient comprising relevant portions of the medical imagery scans combined with the real-time images of the patient; selecting one or more portions of the composite image to diminish for the diagnosis; and rendering the composite image in a diminished reality environment.

Abstract: First and second skeleton model data is determined based on first and second surface data of a patient. Each of the skeleton model data describes geometries of rigid anatomic structures of a patient at a different point in time. Skeleton difference data is determined describing differences between the geometries of the rigid anatomic structures. In a next step, movement instruction data is determined which describes movement to be performed by the rigid anatomic structures to minimize the differences, i.e. to correct the posture of the patient. The movement instruction data is for example determined based on anatomy constraint data which describes anatomical movement constraints for the rigid anatomic structures (e.g. range of motion of a joint). An instruction is displayed (e.g. using augmented reality), guiding the user how to move the rigid anatomic structures so as to correct the patients posture.

Abstract: Systems and methods are provided for multi-schema analysis of patient specific anatomical features from medical images. The system may receive medical images of a patient and metadata associated with the medical images indicative of a selected pathology, and automatically classify the medical images using a segmentation algorithm. The system may use an anatomical feature identification algorithm to identify one or more patient specific anatomical features within the medical images by exploring an anatomical knowledge dataset. A 3D surface mesh model may be generated representing the one or more classified patient specific anatomical features, such that information may be extracted from the 3D surface mesh model based on the selected pathology. Physiological information associated with the selected pathology for the 3D surface mesh model may be generated based on the extracted information.

Abstract: A system and method for supporting multi-probe guidance are disclosed. The system comprises: a medical guidance apparatus comprising: a rotatable portion which is rotatable around a first axis; an arc guide attached to or integrally formed with the rotatable portion, and a probe holder movably mounted on the arc guide. The probe holder is rotatable around a second axis perpendicular to the first axis by being translated along an arcuate path defined by the shape of the arc guide. In one embodiment, the probe holder includes a plurality of probe channels to guide a corresponding plurality of probes parallel to each other towards a subject. In other embodiment, the probe holder includes a single probe channel which is offset from a center point of the guidance apparatus by a fixed distance such that each probe can be inserted sequentially through a different insertion point without colliding at the center point.

Abstract: Disclosed is a hybrid approach to rendering transparent or translucent objects, which combines object-space ray tracing with texture-space parametrization and integration. Transparent or translucent objects are first parameterized using two textures: (1) a texture that stores the surface normal at each location on the transparent or translucent object, and (2) a texture that stores the world space coordinates at each location on the transparent or translucent object. Ray tracing can then be used to streamline and unify the computation of light transport inside thick mediums, such as transparent or translucent objects, with the rest of the scene. For each valid (e.g., visible) location on the surface of a transparent or translucent object, the disclosed embodiments trace one or more rays through such objects and compute the resulting lighting in an order-independent fashion. The results are stored in a texture, which is then applied during the final lighting stage.