Abstract: A technique for rendering virtual content to a user stores map data of features in a physical environment of the user and measures the location of the user with stationary sensors placed at respective locations within the environment. A server provides the location of the user and portions of the map data to a headset worn by the user. The headset is thus enabled to render virtual content at apparent locations that are based on the measured location of the user and the features described by the map data.

Abstract: Various embodiments of the present invention relate generally to mechanisms and processes relating to artificially rendering images using viewpoint interpolation and extrapolation. According to particular embodiments, a method includes applying a transform to estimate a path outside the trajectory between a first frame and a second frame, where the first frame includes a first image captured from a first location and the second frame includes a second image captured from a second location. The process also includes generating an artificially rendered image corresponding to a third location positioned on the path.

Abstract: Various embodiments describe systems and processes for capturing and generating multi-view interactive digital media representations (MIDMRs). In one aspect, a method for automatically generating a MIDMR comprises obtaining a first MIDMR and a second MIDMR. The first MIDMR includes a convex or concave motion capture using a recording device and is a general object MIDMR. The second MIDMR is a specific feature MIDMR. The first and second MIDMRs may be obtained using different capture motions. A third MIDMR is generated from the first and second MIDMRs, and is a combined embedded MIDMR. The combined embedded MIDMR may comprise the second MIDMR being embedded in the first MIDMR, forming an embedded second MIDMR. The third MIDMR may include a general view in which the first MIDMR is displayed for interactive viewing by a user on a user device. The embedded second MIDMR may not be viewable in the general view.

Abstract: An apparatus comprising means for: causing display of at least one world-locked volumetric object having at least one non-viewable region; determining a change in a user's viewing position relative to the at least one world-locked volumetric object; determining if the change in the user's viewing position relative to the at least one world-locked volumetric object will cause display of at least a portion of the at least one non-viewable region of the at least one world-locked volumetric object; and if it is determined that the change in the user's viewing position will cause display of at least a portion of the at least one non-viewable region, cause the at least one world-locked volumetric object to be user-locked.

Abstract: A method for color shade matching obtains a 3D tooth surface representation using an intraoral scanner wherein the 3D tooth surface representation comprises surface data and a spatially resolved angular distribution of color vectors, wherein the spatially resolved angular distribution of color vectors associates one or more point positions from the surface data to the corresponding angular distribution of color vectors. The method identifies one or more shade values, where each shade value is associated with one angular distribution of color vectors from the spatially resolved angular distribution of color vectors, by comparing the angular distribution of color vectors to a set of reference angular distributions of color vectors, wherein each reference angular distribution in the set is associated with a corresponding shade value. The method displays, stores, or transmits the surface data with an indication of the one or more shade values.

Abstract: Methods, systems, and apparatuses are described for managing placement of transducer arrays on a subject/patient.

Abstract: A method of generating a composite image includes capturing a video image of a physical scene with a camera, identifying a green-screen region within the video image, identifying a viewpoint and a position and/or orientation of the green-screen region relative to the viewpoint, and generating a modified video image rendered from the viewpoint onto a display surface in which the green-screen region is replaced with an image of a virtual object. The image of the virtual object is generated by projection rendering of a model of the virtual object based on the position and/or orientation of the green-screen region relative to the viewpoint such that the virtual object is constrained within the green-screen region.

Abstract: A method of adjusting a grid spacing of a height map for autonomous driving, may include acquiring a 2D image of a region ahead of a vehicle, generating a depth map using depth information on an object present in the 2D image, converting the generated depth map into a 3D point cloud, generating the height map by mapping the 3D point cloud onto a grid having a predetermined size, and adjusting a grid spacing of the height map in consideration of the driving state of the vehicle relative to the object.

Abstract: An electronic apparatus performs a method of recovering a complete and dense point cloud from a partial point cloud. The method includes: constructing a sparse but complete point cloud from the partial point cloud through a contrastive teacher-student neural network; and transforming the sparse but complete point cloud to the complete and dense point cloud. In some embodiments, the contrastive teacher-student neural network has a dual network structure comprising a teacher network and a student network both sharing the same architecture. The teacher network is a point cloud self-reconstruction network, and the student network is a point cloud completion network.

Abstract: Disclosed is a system for rapid digitization of an article that provides a 3d image of an object, such as a clothing article, wherein the object can be virtually viewed from all sides to be viewed by the user. The system for rapid digitization is configured with two microprocessors for rapidly digitizing the clothing apparel into its equivalent 3d model. The system for rapid digitization is provided with a communication device that displays the status of the system and occurring defects are notified through the same. The system for rapid digitization provides a 3d .obj (object) file of the apparel thereby making it portable in nature.

Abstract: Provided are an image processing apparatus and an image processing method that enable a strobe image using a 3D model to be generated. A strobe model in which 3D models of an object at a plurality of times generated from a plurality of viewpoint images captured from a plurality of viewpoints are disposed in a three-dimensional space is generated. When the strobe model is generated, a target object that is a target in which the 3D model is disposed in the strobe model is set according to a degree of object relevance indicating relevance with a key object serving as a reference for disposition of the 3D model in the strobe model.

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for variably rendering graphical representations of co-users in VR environments based on social graph information between the co-users and a VR user. For example, the disclosed systems can identify a co-user within a VR environment. Furthermore, the disclosed systems can determine a social relevancy (e.g., a social relationship type) between the co-user and a VR user within the VR environment based on social graph information. Then, the disclosed systems can select and/or determine a graphical representation and/or other capabilities of the co-user within the VR environment based on the social relevancy. Additionally, the disclosed systems can display the co-user within the VR environment using the determined graphical representation type (e.g., from the perspective of the VR user).

Abstract: A simulator environment is disclosed. In embodiments, the simulator environment includes graphics generation (GG) processors in communication with one or more display devices. Deep learning neural networks running on the GG processors are configured for run-time generation of photorealistic, geotypical content for display. The DL networks are trained on, and use as input, a combination of image-based input (e.g., imagery relevant to a particular geographical area) and a selection of geo-specific data sources that illustrate specific characteristics of the geographical area. Output images generated by the DL networks include additional data channels corresponding to these geo-specific data characteristics, so the generated images include geotypical representations of land use, elevation, vegetation, and other such characteristics.

Abstract: A method includes receiving, through a network by a computing system associated with an artificial-reality device, video data of a user of a second computing system comprising a first and second image of the user. The first and second images may be captured concurrently by a first camera and a second camera of the second computing system, respectively. The computing system generates a planar proxy for displaying the user and determines a pose for the planar proxy within a three-dimensional virtual environment. The computing system renders a left image for a left-eye display and a right image for a right-eye display of the artificial-reality device based on the planar proxy having the determined pose and the first image, and the planar proxy having the determined pose and the second image, respectively. The computing system displays the rendered left image and right image using the left-eye display and right-eye display, respectively.

Abstract: A graphics processing unit configured to process graphics data using a rendering space which is sub-divided into a plurality of tiles, the graphics processing unit comprising: a plurality of processing cores configured to render graphics data; cost indication logic configured to obtain a cost indication for each of a plurality of sets of one or more tiles of the rendering space, wherein the cost indication for a set of one or more tiles is suggestive of a cost of processing the set of one or more tiles; similarity indication logic configured to obtain similarity indications between sets of one or more tiles of the rendering space, wherein the similarity indication between two sets of one or more tiles is indicative of a level of similarity between the two sets of tiles according to at least one processing metric; and scheduling logic configured to assign the sets of one or more tiles to the processing cores for rendering in dependence on the cost indications and the similarity indications.

Abstract: A technique of the present disclosure includes: an obtaining unit configured to obtain model data indicating a three-dimensional shape of an object in a space; a receiving unit configured to receive viewpoint information from an external apparatus, the viewpoint information indicating a virtual viewpoint for a virtual viewpoint image to be generated by the external apparatus based on data transmitted from the image processing apparatus, the virtual viewpoint image depicting the space as viewed from the virtual viewpoint; a generating unit configured to generate image data by rendering a first portion of the obtained model data, based on the received viewpoint information; and a transmitting unit configured to transmit the generated image data and a second portion of the obtained model data different from the first portion to the external apparatus.

Abstract: The present disclosure is directed to automatically generating a 360 Virtual Photographic Representation (“spin”) of an object using multiple images of the object. The system uses machine learning to automatically differentiate between images of the object taken from different angles. A user supplies multiple images and/or videos of an object and the system automatically analyzes and classifies the images into the proper order before incorporating the images into an interactive spin. The system automatically classifies the images using features identified in the images. The classifications are based on predetermined classifications associated with the object to facilitate proper ordering of the images in the resulting spin.

Abstract: A three-dimensional volume modeling method may include rotating a three-dimensional biological object having a translucent outer surface to different angular positions, capturing different two-dimensional images of the three-dimensional biological object, each of the different two-dimensional images being at a different angular position, and modeling an exterior of the three-dimensional biological object based upon the different two-dimensional images. The method may further involve identifying a point of an internal structure of the three-dimensional biological object each of the two-dimensional images and modeling the internal structure of the three-dimensional biological object in three-dimensional space relative to the exterior of the three-dimensional biological object by triangulating the point amongst the different two-dimensional images using a three-dimensional volumetric template of the three-dimensional biological object.

Abstract: Described herein are techniques for performing ray tracing operations. A command processor executes custom instructions for orchestrating a ray tracing pipeline. The custom instructions cause the command processor to perform a series of loop iterations, each at a particular recursion depth. In a first loop iteration, a ray generation shader is executed that triggers execution of a trace ray operation. In any other iteration, zero or more shaders are executed based on the contents of a shader queue. Any shader may trigger execution of a trace ray operation. The trace ray operation determines whether a ray specified by the shader intersects a triangle. The ray trace operation places shader entries into a shader queue, at the current recursion depth plus 1. The command processor updates the current recursion depth based on whether a trace ray operation is executed. The loop ends when the recursion depth is less than a threshold.

Abstract: A graphics processing engine has a geometry shading stage having two modes of operation. In the first mode of operation, each primitive output by the geometry shading stage is independent, whereas in the second mode of operation, connectivity between input primitives is maintained by the geometry shading stage. The mode of operation of the geometry shading stage can be determined based on the value of control state data which may be generated at compile-time for a geometry shader based on analysis of that geometry shader.