Abstract: A method of vertical planar surface detection includes identifying, at an electronic device, a plurality of feature points of an environment proximate the electronic device based on imagery captured at the electronic device and based on received non-visual pose information obtained from non-visual sensors. The electronic device queries the plurality of feature points to identify one or more planar surfaces in the imagery captured at the electronic device. Further, the electronic device computes a plurality of oriented point vectors associated with the one or more planar surfaces. A vertical planar surface is detected based at least in part on one or more of the plurality of oriented point vectors being aligned in a direction perpendicular to gravity within a predetermined threshold.

Abstract: Simulating output of a perception system may comprise receiving scenario data indicating a position associated with a simulated sensor and a position and/or identifier of an object, and instantiating a three-dimensional representation of an environment and the object (i.e., a simulated environment). The system may generate depth data indicating distances and/or positions of surfaces in the simulated environment relative to the simulated sensor position and determine a three-dimensional region of interest based at least in part on the depth data associated with at least a portion of the object. In some examples, the three-dimensional region of interest may be smaller than a size of the object, due to an occlusion by topology of the simulated environment and/or another object in the simulated environment.

Abstract: An image processing apparatus includes an acquisition unit configured to acquire a three-dimensional shape data of an object based on images captured by a plurality of cameras, a generation unit configured to generate information based on a relationship between the three-dimensional shape data acquired by the acquisition unit and positions of the plurality of cameras, and a correction unit configured to correct the three-dimensional shape data based on the information generated by the generation unit.

Abstract: A computerized, interactive eyewear display board system, comprising a plurality of display shelves mounted to a display board's support structure, each of the display shelves comprising at least one light that is adapted to selectively illuminate the respective shelf, the at least one light being controlled a computer processor; and an item tracking mechanism that is adapted to determine which particular eyewear is on each particular respective one of the display shelves. In various embodiments, the display board system is adapted to receive eyewear preference criteria from a user, identify particular eyewear that is placed on a particular one of the display shelves that satisfies the eyewear preference criteria, and in response to determining that the particular eyewear satisfies the eyewear preference criteria, illuminate the particular shelf.

Abstract: Disclosed are a method and a system for processing a computer-generated hologram (CGH). The system for processing a CGH includes a CGH generation apparatus and a display apparatus. The CGH generation apparatus repeatedly performs a process of propagating object data from a first depth layer to a second depth layer, changing amplitude data of the object data to second predefined amplitude data, back-propagating the object data from the second depth layer to the first depth layer, and changing the amplitude data of the object data to first predefined amplitude data, and generates a CGH by using the object data.

Abstract: A system and method for fitting virtual eyewear models on face models is described herein. The method includes attaching a first collider to a model of eyewear and attaching a second collider to a model of a user's face. The method also includes positioning the model of eyewear onto the model of the user's face by reducing a distance between the model of eyewear and the model of the user's face until the first collider attached to the model of eyewear collides with the second collider attached to the model of the user's face and anchoring the model of eyewear to the model of the user's face.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for virtual try-on of items such as spectacles. A virtual try-on interface may be implemented at a server or at a user device, and may use collision detection between three-dimensional models of the spectacles and of a user's face and head to determine the correct size and position of the spectacles for virtual try-on. With the determined size and position, a virtual representation of the spectacles is superimposed on an image of the user.

Abstract: A system and method builds triangles from vertices of polygons and stores the pixels overlayed at least in part by each triangle. The system and method then takes pixels overlayed by an even number of times for each polygon and renders them. Antialiasing information may be generated and provided by oring or exclusive oring bits representing each polygon for each of several pixel divisions.

Abstract: A computer device includes a processor configured to simulate a virtual environment based on a set of virtual environment parameters, and perform ray tracing to render a view of the simulated virtual environment. The ray tracing includes generating a plurality of rays for one or more pixels of the rendered view of the simulated virtual environment. The processor is further configured to determine sub-pixel data for each of the plurality of rays based on intersections between the plurality of rays and the simulated virtual environment, and store the determined sub-pixel data for each of the plurality of rays in an image file.

Abstract: Determining whether a ray intersects a 3D axis-aligned box identifies the front-facing plane of the box which intersects the ray at a position furthest along a direction of the ray. Whether the ray intersects the box is determined by whether the ray intersects the identified front-facing plane at a position that is no further along the ray than positions at which the ray intersects the back-facing planes in a subset of the dimensions. The subset of dimensions comprises the two dimensions for which the front-facing plane was not identified, but does not comprise the dimension for which the front-facing plane was identified. Whether the ray intersects the box is determined without performing a test to determine whether the ray intersects the identified front-facing plane at a position that is no further along the ray than a position at which the ray intersects the back-facing plane in the front-facing plane dimension.

Abstract: A three-dimensional data encoding method includes: determining a total number of layers; when the total number of the layers is greater than 1, selecting, from pieces of attribute information of three-dimensional points, attribute information of a three-dimensional point based on a sampling period according to a data order of the pieces of attribute information of the three-dimensional points, and classifying the pieces of attribute information of the three-dimensional points into layers by assigning a first layer to pieces of attribute information of three-dimensional points selected and a second layer to pieces of attribute information of three-dimensional points non-selected; encoding the pieces of attribute information of the three-dimensional points for each of the layers; and generating a bitstream including the pieces of attribute information encoded, layer-number information, and sampling period information.

Abstract: A method and system for performing graphics processing is provided. The method and system includes storing stencil buffer values in a stencil buffer; generating either or both of a reference value and a source value in a fragment shader; comparing the stencil buffer values against the reference value; and processing a fragment based on the comparing the stencil buffer values against the reference value.

Abstract: A method is described which includes receiving a point cloud having a plurality of data points each representing a 3D location in a 3D space, the point cloud being obtained using a detection and ranging (DAR) sensor. For each data point, associating the data point with a 3D volume containing the 3D location of the data point, the 3D volume being defined using a 3D lattice that partitions the 3D space based on spherical coordinates. For at least one 3D volume, the data points are sorted within the 3D volume based on at least one dimension of the 3D lattice; and the sorted data points are stored as a set of ordered data points. The method also includes performing feature extraction on the set of ordered data points to generate a set of ordered feature vectors and providing the set of ordered feature vectors to perform a machine learning inference task.

Abstract: Data structures, methods and primitive block generators for storing primitives in a graphics processing system.

Abstract: A technique for performing a ray intersection test, the method comprising: receiving a request for an early termination ray intersection test for a ray; testing the ray against one or more early termination box nodes and one or more normal box nodes of a bounding volume hierarchy; and based on the test of the ray against the one or more early termination box nodes, determining whether to end traversal of the bounding volume hierarchy and determine whether the ray intersects geometry for the purpose of the ray intersection test.

Abstract: This disclosure provides a method and apparatus for processing occlusion in an image, a device, and a computer storage medium. The method includes: determining a current viewpoint parameter used for drawing a current image frame; obtaining a predicted depth map matching the current viewpoint parameter as a target depth map of the current image frame; and determining an occlusion culling result of an object in the current image frame according to the target depth map.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for virtual fitting of items such as spectacles and/or spectacle frames. A user interface for virtual fitting may be implemented at a server or at a user device, and utilize three-dimensional information for the user and three-dimensional information for each frame, with frame information stored in a frame database, to identify and/or recommend frames that are likely to fit the user. Fit information can be provided for a group of frames or for each individual frame selected by the user. The fit information can be provided with a static image of the frames and/or within a virtual try-on operation in which the frames are virtually placed on a real-time image of the user.

Abstract: Systems, apparatuses and methods may provide for technology that determines a position associated with one or more polygons in unresolved surface data and select an anti-aliasing sample rate based on a state of the one or more polygons with respect to the position. Additionally, the unresolved surface data may be resolved at the position in accordance with the selected anti-aliasing sample rate, wherein the selected anti-aliasing sample rate varies across a plurality of pixels. The position may be a bounding box, a display screen coordinate, and so forth.

Abstract: Disclosed is a system and method for rendering point clouds via a hybrid data point and construct visualization. The system receives a point cloud of a three-dimensional (“3D”) environment, and differentiates a first set of the point cloud data points from a second set of the data points based on a position of each data point relative to a specified render position. The system generates a first visualization from values of each of the first set of data points, and a second visualization from values of a set of constructs that replace the second set of data points. Each construct has a polygonal shape and a singular set of values defined from the values of two or more of the second set of data points. The system presents a final render of the 3D environment from the render position by combining the first visualization with the second visualization.

Abstract: A three-dimensional (3D) object transmission method and apparatus is disclosed, where the 3D object transmission method includes determining a transmission mesh level for each region of a mesh corresponding to a 3D object that is segmented into regions, identifying boundary regions and non-boundary regions from among the regions, transmitting a vertex based on a transmission mesh level of a corresponding non-boundary region for each of the non-boundary regions, and transmitting a number of vertices in a preset order based on a first transmission mesh level of at least one neighboring region forming a boundary with a corresponding boundary region for each of the boundary regions.

Abstract: Systems and methods are disclosed for generating a 3D computer model of an eyewear product, using a computer system, the method including obtaining an inventory comprising a plurality of product frames; scanning a user's anatomy; extracting measurements of the user's anatomy; obtaining a first model of a contour and/or surface of the user's anatomy, based on the extracted measurements of the user's anatomy; identifying, based on the contour and/or the surface of the user's anatomy, a first product frame among the plurality of product frames; determining adjustments to the first product frame based on the contour and/or the surface of the user's anatomy; generating a second model rendering comprising the adjusted first product frame matching the contours and/or the surface of the user's anatomy.

Abstract: The present disclosure relates generally to systems and methods for creating a mixed reality environment. A mixed reality system includes a performance area for generating a mixed reality environment, a motion determination module that determines and tracks the motion of an object within the performance area, a physical article module that generates a physical article within the performance area, and a mixed reality display that displays a virtual article within the performance area, wherein the physical article and the virtual article are correlated to the motion of the object.

Abstract: A system and a method are disclosed that reduce primitive overdraw in a GPU. An occlusion index (OI) for a first tile of a batch of graphical data. In one embodiment, the first tile is bypassed from an early coverage discard (ECD) first-in, first-out (FIFO) if the OI for the first tile is less than a first threshold, otherwise the first tile is entered into the ECD FIFO. The first tile is also bypassed from the ECD FIFO if the OI for the first tile is greater than a second threshold that is greater than the first threshold. In another embodiment, a queue length is logically changed for the first tile in the ECD FIFO if the OI for the first tile is greater than the first threshold and less than or equal to a third threshold that is greater than the first threshold and less than the second threshold.

Abstract: A method for detecting a collision between a cylindrical collider and a convex body in a real-time virtual scenario performed at a computer includes: determining a location of a cylindrical collider corresponding to a virtual object in a virtual scenario in a local coordinate system of a convex body; obtaining a projection of the cylindrical collider on one or more testing axes according to the location of the cylindrical collider in the local coordinate system of the convex body; when the projections of the cylindrical collider and the convex body intersect with each other on each testing axis, determining that there is a collision between the cylindrical collider and the convex body and moving the cylindrical collider away from the convex body in the real-time virtual scenario to avoid the collision.

Abstract: A method, system, and apparatus create a 3D CAD model. Scan data from two or more structured scans of a real-world scene are acquired and each scan processed independently by segmenting the scan data into multiple segments, filtering the scan data, and fitting an initial model that is used as a model candidate. Model candidates are clustered into groups and a refined model is fit onto the model candidates in the same group. A grid of cells representing points is mapped over the refined model. Each of the grid cells is labeled by processing each scan independently, labeling each cell located within the refined model as occupied, utilizing back projection to label remaining cells as occluded or empty. The labels from multiple scans are then combined. Based on the labeling, model details are extracted to further define and complete the refined model.

Abstract: A video processing method includes detecting, as a reference pose, a pose of an individual at a reference time point in an input video sequence; at a second, different, time point in the input video sequence, detecting a second pose of the individual; generating from one or more source images of the individual, a transitional video sequence representing a transition of the individual from the second pose to the reference pose; and associating the transitional video sequence with the input video sequence to generate an output video sequence including at least the transitional video sequence to implement a non-linear replay branch from the second time point to the reference time point.

Abstract: The invention relates to a method for transmitting 3D model data, the 3D model data comprising polygons, from a server to a client for rendering, the method comprising: obtaining the 3D model data by the server; and transmitting the 3D model data from the server to the client. According to the invention, the 3D model data is obtained by the server, based on a given multitude of possible views.

Abstract: In implementations of trajectory-based viewport prediction for 360-degree videos, a video system obtains trajectories of angles of users who have previously viewed a 360-degree video. The angles are used to determine viewports of the 360-degree video, and may include trajectories for a yaw angle, a pitch angle, and a roll angle of a user recorded as the user views the 360-degree video. The video system clusters the trajectories of angles into trajectory clusters, and for each trajectory cluster determines a trend trajectory. When a new user views the 360-degree video, the video system compares trajectories of angles of the new user to the trend trajectories, and selects trend trajectories for a yaw angle, a pitch angle, and a roll angle for the user. Using the selected trend trajectories, the video system predicts viewports of the 360-degree video for the user for future times.

Abstract: A technique for performing ray tracing operations is provided. The technique includes reading descendant-shared type metadata for a non-leaf node of a bounding volume hierarchy; identifying one or more culling types for a ray-intersection test for a ray; and determining whether to treat the non-leaf node as not intersected based on whether the one or more culling types includes at least one type specified by the descendant-shared type metadata.

Abstract: An image dataset is processed with a shadow map generated from objects in a virtual scene that can cast shadows and the scene is rendered independent of the shadows. The shadow might be edited separately, and then applied to a post-render image of the scene to form a shadowed image. Light factor values for pixels of the shadow map might be stored as summed-area table values.

Abstract: Techniques of smoothing surface normals in a texture mapping application involve generating smoothed normals from the perspective of each camera using to capture images for texture mapping. Along these lines, a camera used to capture an image for texture mapping is situated at an orientation relative to the geometrical object onto which a texture mapping computer maps the texture image. The texture mapping computer places a filter window centered at a point on the geometrical object. The texture mapping computer then generates, as the smoothed normal at that point, an average normal over points in the filter window. The average normals thus computed for each camera are then used in the weights of the weighted average that is the image value at that point.

Abstract: Operations of a method include accessing a first object and a second object positioned in virtual three-dimensional (3D) space. The operations further include moving the first object from a first position to a second position, responsive to a user request. The operations include computing a ray extending from a reference point of the first object, in the second position, in a direction of a first dimension. The operations include computing a pixel distance in screen space between a first plane of a first bounding box of the first object and a second plane of a second bounding box of the second object along the ray. Additionally, the method includes snapping the first object into alignment by moving the first object an additional distance to position a face of the first bounding box into the second plane, based on the pixel distance being no greater than an error value.

Abstract: Various aspects of the subject technology relate to systems, methods, and machine-readable media for virtual try-on of items such as spectacles. A virtual try-on interface may be implemented at a server or at a user device, and may use collision detection between three-dimensional models of the spectacles and of a user's face and head to determine the correct size and position of the spectacles for virtual try-on. With the determined size and position, a virtual representation of the spectacles is superimposed on an image of the user.

Abstract: A texture processor based ray tracing accelerator method and system are described. The system includes a shader, texture processor (TP) and cache, which are interconnected. The TP includes a texture address unit (TA), a texture cache processor (TCP), a filter pipeline unit and a ray intersection engine. The shader sends a texture instruction which contains ray data and a pointer to a bounded volume hierarchy (BVH) node to the TA. The TCP uses an address provided by the TA to fetch BVH node data from the cache. The ray intersection engine performs ray-BVH node type intersection testing using the ray data and the BVH node data. The intersection testing results and indications for BVH traversal are returned to the shader via a texture data return path. The shader reviews the intersection results and the indications to decide how to traverse to the next BVH node.

Abstract: An electronic device is configured to performing a three-dimensional (3D) scan of an interior space. In some cases, the electronic device acquires information and depth measurements relative to the electronic device. The electronic device acquires voxels in a 3D grid that is generated from the 3D scan. The voxels represent portions of the volume of the interior space. The electronic device determines a trajectory and poses of the electronic device concurrently with performing the 3D scan of the interior space. The electronic device labels voxels representing objects in the interior space based on the trajectory and the poses. In some cases, the electronic device uses queries to perform spatial reasoning at an object level of granularity, positions, overlays, or blends virtual objects into an augmented reality representation of the interior space or modifies positions or orientations of the objects by applying a transformation to corresponding connected components.

Abstract: Preparation and execution of quantized scaling may be performed by operations including obtaining an original array and a scaling factor representing a ratio of a size of the original array to a size of a scaled array, determining, for each column of the scaled array, a horizontal coordinate of each of two nearest elements in the horizontal dimension of the original array, and, for each row of the scaled array, a vertical coordinate of each of two nearest elements in the vertical dimension of the original array, calculating, for each row of the scaled array and each column of the scaled array, a linear interpolation coefficient, converting each value of the original array from a floating point number into a quantized number, converting each linear interpolation coefficient from a floating point number into a fixed point number, storing, in a memory, the horizontal coordinates and vertical coordinates as integers, the values as quantized numbers, and the linear interpolation coefficients as fixed point numbers

Abstract: Techniques for improving the visualization of three-dimensional (3D) models of items on computing devices for augmented reality (AR) are described. One technique includes obtaining a virtual representation of an item and detecting a plane within a physical environment. A range of distances to the plane from a computing device for visualizing the virtual representation on a screen of the computing device is determined based on (i) physical attribute(s) of the screen and (ii) physical attribute(s) of the virtual representation. The virtual representation is rendered on the screen for positions on the plane that are within the range of distances from the computing device. The virtual representation is not rendered on the screen for positions on the plane that are outside of the range of distances from the computing device.

Abstract: A method and apparatus for monitoring cache transactions in a cache of a data processing system is provided. Responsive to a cache transaction associated with a transaction address, when a cache controller determines that the cache transaction is selected for monitoring, the cache controller retrieves a pointer stored in a register, determines a location in a log memory from the pointer, and writes a transaction identifier to the determined location in the log memory. The transaction identifier is associated with the transaction address and may be a virtual address, for example. The pointer is updated and stored to the register. The architect of the apparatus may include a mechanism for atomically combining data access instructions with an instruction to commence monitoring.

Abstract: A computer device includes a processor configured to simulate a virtual environment based on a set of virtual environment parameters, and perform ray tracing to render a view of the simulated virtual environment. The ray tracing includes generating a plurality of rays for one or more pixels of the rendered view of the simulated virtual environment. The processor is further configured to determine sub-pixel data for each of the plurality of rays based on intersections between the plurality of rays and the simulated virtual environment, and store the determined sub-pixel data for each of the plurality of rays in an image file.

Abstract: Embodiments of the present disclosure provide a recommendation system based on a user's physical/biometric features. In various embodiments, a system includes a processor configured to determine a physical characteristic of a user based at least in part on an image of the user. The processor is further configured to determine a correlation between the physical characteristic and a product, and generate a product recommendation based at least in part on the determined correlation.

Abstract: A three-dimensional data encoding method includes: determining whether to encode, using an octree structure, a current space unit among a plurality of space units included in three-dimensional data; encoding the current space unit using the octree structure, when it is determined that the current space unit is to be encoded using the octree structure; encoding the current space unit using a different method that is not the octree structure, when it is determined that the current space unit is not to be encoded using the octree structure; and appending, to a bitstream, information that indicates whether the current space unit has been encoded using the octree structure.

Abstract: A method of a virtual X-ray colonoscopy includes scanning (204) a dark-field contrast (144) insufflated colon lumen (140) with an X-ray scanner (110) configured for dark-field-contrast, which generates dark-field-contrasted projection data and attenuation projection data. The dark-field-contrasted projection data and the attenuation projection data are reconstructed (206) into one or more dark-field-contrasted images (148).

Abstract: The present application provides a method of controlling a virtual image. The method may includes acquiring relative spatial position between a terminal device and an interaction device; generating the virtual image based on the relative spatial position, wherein the virtual image includes one or more user interface elements, and a superimposing position of each user interface element in a real space correspond to the interaction device; generating in advance and acquiring a rotational vector corresponding to the interaction device, determining a spatial position of the rotational vector in a virtual space based on the relative spatial position, wherein a direction indicated by the rotational vector may be fixed; and when the spatial position is located with a region corresponding to any user interface element in the virtual space, performing a corresponded control operation based on the user interface element corresponding to the region.

Abstract: In some embodiments, a technique for providing location-based functionality comprises providing functionality, wherein the functionality uses information provided by a location-aware device.

Abstract: A graphics processing system includes a tiling unit for performing tiling calculations and a hidden surface removal (HSR) unit for performing HSR on fragments of the primitives. Primitive depth information is calculated in the tiling unit and forwarded for use by the HSR unit in performing HSR on the fragments. This takes advantage of the tiling unit having access to the primitive data before the HSR unit performs the HSR on the primitives, to determine some depth information which can simplify the HSR performed by the HSR unit. Therefore, the final values of a depth buffer determined in the tiling unit can be used in the HSR unit to determine that a particular fragment will subsequently be hidden by a fragment of a primitive which is yet to be processed in the HSR unit, such that the particular fragment can be culled.

Abstract: Methods, systems, and computer program products for rending an audio object having an apparent size are disclosed. An audio processing system receives audio panning data including a first grid mapping first virtual sound sources in a space and speaker positions to speaker gains. The first grid specifies first speaker gains of the first virtual sound sources in the space. The audio processing system determines a second grid of second virtual sound sources in the space, including mapping the first virtual sound sources into the second virtual sound sources of the second virtual sources. The audio processing system selects at least one of the first grid or second grid for rendering an audio object based on an apparent size of the audio object. The audio processing system renders the audio object based on the selected grid or grids.

Abstract: An information processing apparatus that performs processing to generate a virtual viewpoint image based on a plurality of captured images obtained by a plurality of cameras, comprises an obtaining unit that obtains visibility information which is related to components of a three-dimensional shape model of an object, wherein the visibility information is configured to be used for identifying whether pixels corresponding to the components are included in a captured image for each of not less than two captured images of the plurality of captured images, a selection unit that selects, based on the visibility information, from material data which is based on the captured image obtained by a camera included in the plurality of cameras and is stored in a predetermined storage unit, data to be used to generate the virtual viewpoint image.

Abstract: In one embodiment, a computing system may determine a first orientation of a viewer in a three-dimensional (3D) space based on first sensor data associated with a first time. The system may render one or more first lines of pixels based on the first orientation of the viewer and display the one or more first lines. The system may determine a second orientation of the viewer in the 3D space based on second sensor data associated with a second time that is subsequent to the first time. The system may render one or more second lines of pixels based on the second orientation of the viewer and display the one or more second lines of pixels. The one or more second lines of pixels associated with the second orientation are displayed concurrently with the one or more first lines of pixels associated with the first orientation.

Abstract: In an embodiment, a data processing method provides an improvement in efficient use of computer memory and comprises using a computer, creating in computer memory a glyph memory area that is configured to store a plurality of cached glyphs; using the computer, receiving a request from an application to use a particular glyph; in response to the request, determining whether the particular glyph is in the glyph memory area; in response to determining that the particular glyph is not in the glyph memory area: attempting to store a bitmap of the particular glyph to a next location in the glyph memory area; in response to determining that the next location is not available a first time, reclaiming space in the glyph memory area in an amount sufficient to store the bitmap; attempting a second time to store the bitmap in the next location in the glyph memory area; in response to determining that the next location is not available a second time, clearing the glyph memory area of all previously stored glyphs and stori

Abstract: Image information is often transmitted from one electronic device to another. Such information is typically encoded and/or compressed to reduce the bandwidth required for transmission and/or to decrease the time necessary for transmission. Embodiments are directed to tagging objects or primitives with attribute tags to facilitate the encoding process. Other embodiments are directed to codecs running on hardware and/or software.