Abstract: A method of rendering an image includes receiving information of a virtual camera, including a camera position and a camera orientation defining a virtual screen; receiving information of a target screen, including a target screen position and a target screen orientation defining a plurality of pixels, each respective pixel corresponding to a respective UV coordinate on the target screen; for each respective pixel of the target screen: determining a respective XY coordinate of a corresponding point on the virtual screen based on the camera position, the camera orientation, the target screen position, the target screen orientation, and the respective UV coordinate; tracing one or more rays from the virtual camera through the corresponding point on the virtual screen toward a virtual scene; and estimating a respective color value for the respective pixel based on incoming light from virtual objects in the virtual scene that intersect the one or more rays.

Abstract: A three-dimensional data encoding method includes: determining a total number of layers; when the total number of the layers is greater than 1, selecting, from pieces of attribute information of three-dimensional points, attribute information of a three-dimensional point based on a sampling period according to a data order of the pieces of attribute information of the three-dimensional points, and classifying the pieces of attribute information of the three-dimensional points into layers by assigning a first layer to pieces of attribute information of three-dimensional points selected and a second layer to pieces of attribute information of three-dimensional points non-selected; encoding the pieces of attribute information of the three-dimensional points for each of the layers; and generating a bitstream including the pieces of attribute information encoded, layer-number information, and sampling period information.

Abstract: The present invention relates to a method of streaming interactive computer graphics from a server to a client device. The method includes the steps of: intercepting graphics instructions transmitted from an application destined for a graphical processing unit (GPU) at the server; processing the graphics instructions to generate graphics data at the server; generating index information for, at least, some of the graphics data at the server; transmitting the index information in place of the graphics data to a client device; extracting corresponding graphics data stored at the client device utilizing the index information; and rendering computer graphics at a graphical processing unit (GPU) at the client device using the corresponding graphics data. A system for streaming interactive computer graphics is also disclosed.

Abstract: Systems and methods of generating a three-dimensional (3D) reconstruction of a scene or environment surrounding a user of a spatial computing system, such as a virtual reality, augmented reality or mixed reality system, using only multiview images comprising, and without the need for depth sensors or depth data from sensors. Features are extracted from a sequence of frames of RGB images and back-projected using known camera intrinsics and extrinsics into a 3D voxel volume wherein each pixel of the voxel volume is mapped to a ray in the voxel volume. The back-projected features are fused into the 3D voxel volume. The 3D voxel volume is passed through a 3D convolutional neural network to refine the and regress truncated signed distance function values at each voxel of the 3D voxel volume.

Abstract: The present disclosure provides to a processing device and a processing method for a ray tracing acceleration structure. The processing device includes a machine-readable storage medium and a processor. The processor executes a descriptor to simulate the interaction between the ray with the scene, and the descriptor includes a first pointer and a second pointer. The processor obtains the TLAS by using the first pointer. The processor traverses the TLAS to find a leaf node in the TLAS that intersects the ray, and the intersected leaf node includes an instance identifier. The processor obtains the intersected instance record from the instance buffer pointed to by the second pointer by using the instance identifier, and the intersected instance record includes a third pointer. The processor obtains the BLAS by using the third pointer. The processor traverses the BLAS to find a primitive node in the BLAS that intersects the ray.

Abstract: A system and method for completing risk object identification that include receiving image data associated with a monocular image of a surrounding environment of an ego vehicle and analyzing the image data and completing semantic waypoint labeling of at least one region of the surrounding environment of the ego vehicle. The system and method also include completing counterfactual scenario augmentation with respect to the at least one region. The system and method further include determining at least one driver intention and at least one driver response associated with the at least one region.

Abstract: A system, method, and computer readable medium for determining at least one feature of at least one lens mounted in a spectacle frame. The system includes a non-point light pattern generating device configured to generate a predefined non-point light pattern, and arranged to have the predefined non-point light pattern reflected on the at least one lens, an image acquisition device configured to acquire an image of the spectacle frame including an image of the reflected non-point light pattern, and a processing unit configured to determine at least the contour of the at least one lens in function of at least one feature of the image of the reflected non-point light pattern.

Abstract: Provided is an input system including an instruction device; a detection device that detects an instruction position of the instruction device; a display device that is provided separately from the detection device and that displays an image of a space; a position detector that detects, in the space, a line-of-sight position in a line-of-sight of the position detector; a processor; and a memory storing instructions, which when executed by the processor, cause the processor to set, in the space, an instruction target area including a reference position of the instruction device, based on the line-of-sight position detected by the position detector at the time of reception of a predetermined operation by the instruction device, and control the display device to display the instruction position of the instruction device in the instruction target area and on the image of the space.

Abstract: A method of generating map images in a computing device includes loading source map data or vector data onto a host; loading an output tile system; determining which output tiles to process by cross-referencing the source map data or the vector data with the output tile system; loading source maps or vector data to a graphics processing unit memory; executing a graphics processing unit kernel to process data and return a map tile to the host; writing the map tile to a file on a database; determining if all tiles have been processed; and generating output map tiles. A map tile generation system includes computing devices; servers connected via a network; non-transitory computer-readable storage media storing instructions and machine-learning graphics processor units coupled to the servers via a kernel interface program.

Abstract: A method of vertical planar surface detection includes identifying, at an electronic device, a plurality of feature points of an environment proximate the electronic device based on imagery captured at the electronic device and based on received non-visual pose information obtained from non-visual sensors. The electronic device queries the plurality of feature points to identify one or more planar surfaces in the imagery captured at the electronic device. Further, the electronic device computes a plurality of oriented point vectors associated with the one or more planar surfaces. A vertical planar surface is detected based at least in part on one or more of the plurality of oriented point vectors being aligned in a direction perpendicular to gravity within a predetermined threshold.

Abstract: Simulating output of a perception system may comprise receiving scenario data indicating a position associated with a simulated sensor and a position and/or identifier of an object, and instantiating a three-dimensional representation of an environment and the object (i.e., a simulated environment). The system may generate depth data indicating distances and/or positions of surfaces in the simulated environment relative to the simulated sensor position and determine a three-dimensional region of interest based at least in part on the depth data associated with at least a portion of the object. In some examples, the three-dimensional region of interest may be smaller than a size of the object, due to an occlusion by topology of the simulated environment and/or another object in the simulated environment.

Abstract: An image processing apparatus includes an acquisition unit configured to acquire a three-dimensional shape data of an object based on images captured by a plurality of cameras, a generation unit configured to generate information based on a relationship between the three-dimensional shape data acquired by the acquisition unit and positions of the plurality of cameras, and a correction unit configured to correct the three-dimensional shape data based on the information generated by the generation unit.

Abstract: A computerized, interactive eyewear display board system, comprising a plurality of display shelves mounted to a display board's support structure, each of the display shelves comprising at least one light that is adapted to selectively illuminate the respective shelf, the at least one light being controlled a computer processor; and an item tracking mechanism that is adapted to determine which particular eyewear is on each particular respective one of the display shelves. In various embodiments, the display board system is adapted to receive eyewear preference criteria from a user, identify particular eyewear that is placed on a particular one of the display shelves that satisfies the eyewear preference criteria, and in response to determining that the particular eyewear satisfies the eyewear preference criteria, illuminate the particular shelf.

Abstract: Disclosed are a method and a system for processing a computer-generated hologram (CGH). The system for processing a CGH includes a CGH generation apparatus and a display apparatus. The CGH generation apparatus repeatedly performs a process of propagating object data from a first depth layer to a second depth layer, changing amplitude data of the object data to second predefined amplitude data, back-propagating the object data from the second depth layer to the first depth layer, and changing the amplitude data of the object data to first predefined amplitude data, and generates a CGH by using the object data.

Abstract: A system and method for fitting virtual eyewear models on face models is described herein. The method includes attaching a first collider to a model of eyewear and attaching a second collider to a model of a user's face. The method also includes positioning the model of eyewear onto the model of the user's face by reducing a distance between the model of eyewear and the model of the user's face until the first collider attached to the model of eyewear collides with the second collider attached to the model of the user's face and anchoring the model of eyewear to the model of the user's face.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for virtual try-on of items such as spectacles. A virtual try-on interface may be implemented at a server or at a user device, and may use collision detection between three-dimensional models of the spectacles and of a user's face and head to determine the correct size and position of the spectacles for virtual try-on. With the determined size and position, a virtual representation of the spectacles is superimposed on an image of the user.

Abstract: A system and method builds triangles from vertices of polygons and stores the pixels overlayed at least in part by each triangle. The system and method then takes pixels overlayed by an even number of times for each polygon and renders them. Antialiasing information may be generated and provided by oring or exclusive oring bits representing each polygon for each of several pixel divisions.

Abstract: A computer device includes a processor configured to simulate a virtual environment based on a set of virtual environment parameters, and perform ray tracing to render a view of the simulated virtual environment. The ray tracing includes generating a plurality of rays for one or more pixels of the rendered view of the simulated virtual environment. The processor is further configured to determine sub-pixel data for each of the plurality of rays based on intersections between the plurality of rays and the simulated virtual environment, and store the determined sub-pixel data for each of the plurality of rays in an image file.

Abstract: Determining whether a ray intersects a 3D axis-aligned box identifies the front-facing plane of the box which intersects the ray at a position furthest along a direction of the ray. Whether the ray intersects the box is determined by whether the ray intersects the identified front-facing plane at a position that is no further along the ray than positions at which the ray intersects the back-facing planes in a subset of the dimensions. The subset of dimensions comprises the two dimensions for which the front-facing plane was not identified, but does not comprise the dimension for which the front-facing plane was identified. Whether the ray intersects the box is determined without performing a test to determine whether the ray intersects the identified front-facing plane at a position that is no further along the ray than a position at which the ray intersects the back-facing plane in the front-facing plane dimension.

Abstract: A three-dimensional data encoding method includes: determining a total number of layers; when the total number of the layers is greater than 1, selecting, from pieces of attribute information of three-dimensional points, attribute information of a three-dimensional point based on a sampling period according to a data order of the pieces of attribute information of the three-dimensional points, and classifying the pieces of attribute information of the three-dimensional points into layers by assigning a first layer to pieces of attribute information of three-dimensional points selected and a second layer to pieces of attribute information of three-dimensional points non-selected; encoding the pieces of attribute information of the three-dimensional points for each of the layers; and generating a bitstream including the pieces of attribute information encoded, layer-number information, and sampling period information.

Abstract: A method and system for performing graphics processing is provided. The method and system includes storing stencil buffer values in a stencil buffer; generating either or both of a reference value and a source value in a fragment shader; comparing the stencil buffer values against the reference value; and processing a fragment based on the comparing the stencil buffer values against the reference value.

Abstract: A method is described which includes receiving a point cloud having a plurality of data points each representing a 3D location in a 3D space, the point cloud being obtained using a detection and ranging (DAR) sensor. For each data point, associating the data point with a 3D volume containing the 3D location of the data point, the 3D volume being defined using a 3D lattice that partitions the 3D space based on spherical coordinates. For at least one 3D volume, the data points are sorted within the 3D volume based on at least one dimension of the 3D lattice; and the sorted data points are stored as a set of ordered data points. The method also includes performing feature extraction on the set of ordered data points to generate a set of ordered feature vectors and providing the set of ordered feature vectors to perform a machine learning inference task.

Abstract: Data structures, methods and primitive block generators for storing primitives in a graphics processing system.

Abstract: A technique for performing a ray intersection test, the method comprising: receiving a request for an early termination ray intersection test for a ray; testing the ray against one or more early termination box nodes and one or more normal box nodes of a bounding volume hierarchy; and based on the test of the ray against the one or more early termination box nodes, determining whether to end traversal of the bounding volume hierarchy and determine whether the ray intersects geometry for the purpose of the ray intersection test.

Abstract: This disclosure provides a method and apparatus for processing occlusion in an image, a device, and a computer storage medium. The method includes: determining a current viewpoint parameter used for drawing a current image frame; obtaining a predicted depth map matching the current viewpoint parameter as a target depth map of the current image frame; and determining an occlusion culling result of an object in the current image frame according to the target depth map.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for virtual fitting of items such as spectacles and/or spectacle frames. A user interface for virtual fitting may be implemented at a server or at a user device, and utilize three-dimensional information for the user and three-dimensional information for each frame, with frame information stored in a frame database, to identify and/or recommend frames that are likely to fit the user. Fit information can be provided for a group of frames or for each individual frame selected by the user. The fit information can be provided with a static image of the frames and/or within a virtual try-on operation in which the frames are virtually placed on a real-time image of the user.

Abstract: Systems, apparatuses and methods may provide for technology that determines a position associated with one or more polygons in unresolved surface data and select an anti-aliasing sample rate based on a state of the one or more polygons with respect to the position. Additionally, the unresolved surface data may be resolved at the position in accordance with the selected anti-aliasing sample rate, wherein the selected anti-aliasing sample rate varies across a plurality of pixels. The position may be a bounding box, a display screen coordinate, and so forth.

Abstract: A three-dimensional (3D) object transmission method and apparatus is disclosed, where the 3D object transmission method includes determining a transmission mesh level for each region of a mesh corresponding to a 3D object that is segmented into regions, identifying boundary regions and non-boundary regions from among the regions, transmitting a vertex based on a transmission mesh level of a corresponding non-boundary region for each of the non-boundary regions, and transmitting a number of vertices in a preset order based on a first transmission mesh level of at least one neighboring region forming a boundary with a corresponding boundary region for each of the boundary regions.

Abstract: Disclosed is a system and method for rendering point clouds via a hybrid data point and construct visualization. The system receives a point cloud of a three-dimensional (“3D”) environment, and differentiates a first set of the point cloud data points from a second set of the data points based on a position of each data point relative to a specified render position. The system generates a first visualization from values of each of the first set of data points, and a second visualization from values of a set of constructs that replace the second set of data points. Each construct has a polygonal shape and a singular set of values defined from the values of two or more of the second set of data points. The system presents a final render of the 3D environment from the render position by combining the first visualization with the second visualization.

Abstract: Systems and methods are disclosed for generating a 3D computer model of an eyewear product, using a computer system, the method including obtaining an inventory comprising a plurality of product frames; scanning a user's anatomy; extracting measurements of the user's anatomy; obtaining a first model of a contour and/or surface of the user's anatomy, based on the extracted measurements of the user's anatomy; identifying, based on the contour and/or the surface of the user's anatomy, a first product frame among the plurality of product frames; determining adjustments to the first product frame based on the contour and/or the surface of the user's anatomy; generating a second model rendering comprising the adjusted first product frame matching the contours and/or the surface of the user's anatomy.

Abstract: The present disclosure relates generally to systems and methods for creating a mixed reality environment. A mixed reality system includes a performance area for generating a mixed reality environment, a motion determination module that determines and tracks the motion of an object within the performance area, a physical article module that generates a physical article within the performance area, and a mixed reality display that displays a virtual article within the performance area, wherein the physical article and the virtual article are correlated to the motion of the object.

Abstract: A system and a method are disclosed that reduce primitive overdraw in a GPU. An occlusion index (OI) for a first tile of a batch of graphical data. In one embodiment, the first tile is bypassed from an early coverage discard (ECD) first-in, first-out (FIFO) if the OI for the first tile is less than a first threshold, otherwise the first tile is entered into the ECD FIFO. The first tile is also bypassed from the ECD FIFO if the OI for the first tile is greater than a second threshold that is greater than the first threshold. In another embodiment, a queue length is logically changed for the first tile in the ECD FIFO if the OI for the first tile is greater than the first threshold and less than or equal to a third threshold that is greater than the first threshold and less than the second threshold.

Abstract: A method for detecting a collision between a cylindrical collider and a convex body in a real-time virtual scenario performed at a computer includes: determining a location of a cylindrical collider corresponding to a virtual object in a virtual scenario in a local coordinate system of a convex body; obtaining a projection of the cylindrical collider on one or more testing axes according to the location of the cylindrical collider in the local coordinate system of the convex body; when the projections of the cylindrical collider and the convex body intersect with each other on each testing axis, determining that there is a collision between the cylindrical collider and the convex body and moving the cylindrical collider away from the convex body in the real-time virtual scenario to avoid the collision.

Abstract: A method, system, and apparatus create a 3D CAD model. Scan data from two or more structured scans of a real-world scene are acquired and each scan processed independently by segmenting the scan data into multiple segments, filtering the scan data, and fitting an initial model that is used as a model candidate. Model candidates are clustered into groups and a refined model is fit onto the model candidates in the same group. A grid of cells representing points is mapped over the refined model. Each of the grid cells is labeled by processing each scan independently, labeling each cell located within the refined model as occupied, utilizing back projection to label remaining cells as occluded or empty. The labels from multiple scans are then combined. Based on the labeling, model details are extracted to further define and complete the refined model.

Abstract: A video processing method includes detecting, as a reference pose, a pose of an individual at a reference time point in an input video sequence; at a second, different, time point in the input video sequence, detecting a second pose of the individual; generating from one or more source images of the individual, a transitional video sequence representing a transition of the individual from the second pose to the reference pose; and associating the transitional video sequence with the input video sequence to generate an output video sequence including at least the transitional video sequence to implement a non-linear replay branch from the second time point to the reference time point.

Abstract: The invention relates to a method for transmitting 3D model data, the 3D model data comprising polygons, from a server to a client for rendering, the method comprising: obtaining the 3D model data by the server; and transmitting the 3D model data from the server to the client. According to the invention, the 3D model data is obtained by the server, based on a given multitude of possible views.

Abstract: In implementations of trajectory-based viewport prediction for 360-degree videos, a video system obtains trajectories of angles of users who have previously viewed a 360-degree video. The angles are used to determine viewports of the 360-degree video, and may include trajectories for a yaw angle, a pitch angle, and a roll angle of a user recorded as the user views the 360-degree video. The video system clusters the trajectories of angles into trajectory clusters, and for each trajectory cluster determines a trend trajectory. When a new user views the 360-degree video, the video system compares trajectories of angles of the new user to the trend trajectories, and selects trend trajectories for a yaw angle, a pitch angle, and a roll angle for the user. Using the selected trend trajectories, the video system predicts viewports of the 360-degree video for the user for future times.

Abstract: A technique for performing ray tracing operations is provided. The technique includes reading descendant-shared type metadata for a non-leaf node of a bounding volume hierarchy; identifying one or more culling types for a ray-intersection test for a ray; and determining whether to treat the non-leaf node as not intersected based on whether the one or more culling types includes at least one type specified by the descendant-shared type metadata.

Abstract: An image dataset is processed with a shadow map generated from objects in a virtual scene that can cast shadows and the scene is rendered independent of the shadows. The shadow might be edited separately, and then applied to a post-render image of the scene to form a shadowed image. Light factor values for pixels of the shadow map might be stored as summed-area table values.

Abstract: Techniques of smoothing surface normals in a texture mapping application involve generating smoothed normals from the perspective of each camera using to capture images for texture mapping. Along these lines, a camera used to capture an image for texture mapping is situated at an orientation relative to the geometrical object onto which a texture mapping computer maps the texture image. The texture mapping computer places a filter window centered at a point on the geometrical object. The texture mapping computer then generates, as the smoothed normal at that point, an average normal over points in the filter window. The average normals thus computed for each camera are then used in the weights of the weighted average that is the image value at that point.

Abstract: Operations of a method include accessing a first object and a second object positioned in virtual three-dimensional (3D) space. The operations further include moving the first object from a first position to a second position, responsive to a user request. The operations include computing a ray extending from a reference point of the first object, in the second position, in a direction of a first dimension. The operations include computing a pixel distance in screen space between a first plane of a first bounding box of the first object and a second plane of a second bounding box of the second object along the ray. Additionally, the method includes snapping the first object into alignment by moving the first object an additional distance to position a face of the first bounding box into the second plane, based on the pixel distance being no greater than an error value.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for virtual try-on of items such as spectacles. A virtual try-on interface may be implemented at a server or at a user device, and may use collision detection between three-dimensional models of the spectacles and of a user's face and head to determine the correct size and position of the spectacles for virtual try-on. With the determined size and position, a virtual representation of the spectacles is superimposed on an image of the user.

Abstract: A texture processor based ray tracing accelerator method and system are described. The system includes a shader, texture processor (TP) and cache, which are interconnected. The TP includes a texture address unit (TA), a texture cache processor (TCP), a filter pipeline unit and a ray intersection engine. The shader sends a texture instruction which contains ray data and a pointer to a bounded volume hierarchy (BVH) node to the TA. The TCP uses an address provided by the TA to fetch BVH node data from the cache. The ray intersection engine performs ray-BVH node type intersection testing using the ray data and the BVH node data. The intersection testing results and indications for BVH traversal are returned to the shader via a texture data return path. The shader reviews the intersection results and the indications to decide how to traverse to the next BVH node.

Abstract: Preparation and execution of quantized scaling may be performed by operations including obtaining an original array and a scaling factor representing a ratio of a size of the original array to a size of a scaled array, determining, for each column of the scaled array, a horizontal coordinate of each of two nearest elements in the horizontal dimension of the original array, and, for each row of the scaled array, a vertical coordinate of each of two nearest elements in the vertical dimension of the original array, calculating, for each row of the scaled array and each column of the scaled array, a linear interpolation coefficient, converting each value of the original array from a floating point number into a quantized number, converting each linear interpolation coefficient from a floating point number into a fixed point number, storing, in a memory, the horizontal coordinates and vertical coordinates as integers, the values as quantized numbers, and the linear interpolation coefficients as fixed point numbers

Abstract: An electronic device is configured to performing a three-dimensional (3D) scan of an interior space. In some cases, the electronic device acquires information and depth measurements relative to the electronic device. The electronic device acquires voxels in a 3D grid that is generated from the 3D scan. The voxels represent portions of the volume of the interior space. The electronic device determines a trajectory and poses of the electronic device concurrently with performing the 3D scan of the interior space. The electronic device labels voxels representing objects in the interior space based on the trajectory and the poses. In some cases, the electronic device uses queries to perform spatial reasoning at an object level of granularity, positions, overlays, or blends virtual objects into an augmented reality representation of the interior space or modifies positions or orientations of the objects by applying a transformation to corresponding connected components.

Abstract: Techniques for improving the visualization of three-dimensional (3D) models of items on computing devices for augmented reality (AR) are described. One technique includes obtaining a virtual representation of an item and detecting a plane within a physical environment. A range of distances to the plane from a computing device for visualizing the virtual representation on a screen of the computing device is determined based on (i) physical attribute(s) of the screen and (ii) physical attribute(s) of the virtual representation. The virtual representation is rendered on the screen for positions on the plane that are within the range of distances from the computing device. The virtual representation is not rendered on the screen for positions on the plane that are outside of the range of distances from the computing device.

Abstract: A method and apparatus for monitoring cache transactions in a cache of a data processing system is provided. Responsive to a cache transaction associated with a transaction address, when a cache controller determines that the cache transaction is selected for monitoring, the cache controller retrieves a pointer stored in a register, determines a location in a log memory from the pointer, and writes a transaction identifier to the determined location in the log memory. The transaction identifier is associated with the transaction address and may be a virtual address, for example. The pointer is updated and stored to the register. The architect of the apparatus may include a mechanism for atomically combining data access instructions with an instruction to commence monitoring.

Abstract: A computer device includes a processor configured to simulate a virtual environment based on a set of virtual environment parameters, and perform ray tracing to render a view of the simulated virtual environment. The ray tracing includes generating a plurality of rays for one or more pixels of the rendered view of the simulated virtual environment. The processor is further configured to determine sub-pixel data for each of the plurality of rays based on intersections between the plurality of rays and the simulated virtual environment, and store the determined sub-pixel data for each of the plurality of rays in an image file.

Abstract: Embodiments of the present disclosure provide a recommendation system based on a user's physical/biometric features. In various embodiments, a system includes a processor configured to determine a physical characteristic of a user based at least in part on an image of the user. The processor is further configured to determine a correlation between the physical characteristic and a product, and generate a product recommendation based at least in part on the determined correlation.

Abstract: A three-dimensional data encoding method includes: determining whether to encode, using an octree structure, a current space unit among a plurality of space units included in three-dimensional data; encoding the current space unit using the octree structure, when it is determined that the current space unit is to be encoded using the octree structure; encoding the current space unit using a different method that is not the octree structure, when it is determined that the current space unit is not to be encoded using the octree structure; and appending, to a bitstream, information that indicates whether the current space unit has been encoded using the octree structure.