Abstract: Systems and methods for generating operating an autonomous vehicle. The methods comprise: obtaining LiDAR point cloud data generated by a LiDAR system of the autonomous vehicle; inspecting the LiDAR point cloud data to infer a health of LiDAR beams; identifying bad quality point cloud data based on the inferred health of the LiDAR beams; removing the bad quality point cloud data from the LiDAR point cloud data to generate modified LiDAR point cloud data; and causing the autonomous vehicle to perform at least one autonomous driving operation or mode change based on the modified LiDAR point cloud data.

Abstract: A method may include receiving at least one camera frame of a first radar target and a second radar target, determining an estimated radar pose based at least in part on the at least one received camera frame, receiving a first radar cross-section (RCS) response from the first radar target and second radar target, determining an estimated elevation angle based at least in part on the first RCS response, and determining an estimated radar angle by refining the estimated radar pose and the estimated elevation angle based on at least in part on the first RCS response.

Abstract: Methods and systems for determining whether a camera of an autonomous vehicle (AV) is calibrated are disclosed. The method includes using image frames captured by a camera of the AV and LIDAR point clouds captured by a LIDAR system of the AV to calculate an AV pose calibration metric. The method also includes: measuring a distance metric between a three-dimensional bounding box around an object and a two-dimensional bounding box in an image captured by the camera; using the AV pose calibration metric and the distance calibration metric to generate a confidence score; and in response to the confidence score being below a threshold, generating a signal indicating that the camera is not calibrated.

Abstract: Systems and methods for generating operating an autonomous vehicle. The methods comprise: obtaining LiDAR point cloud data generated by a LiDAR system of the autonomous vehicle; inspecting the LiDAR point cloud data to infer a health of LiDAR beams; identifying bad quality point cloud data based on the inferred health of the LiDAR beams; removing the bad quality point cloud data from the LiDAR point cloud data to generate modified LiDAR point cloud data; and causing the autonomous vehicle to perform at least one autonomous driving operation or mode change based on the modified LiDAR point cloud data.

Abstract: A method for calibration of vision sensors includes, by a processor: selecting a calibration sequence that has a base calibration pattern and calibration angles, generating a calibrating target that may include the calibration pattern at a selected calibration angle of the angles, and causing a display screen to display a digital image representative of the target at the selected angle relative to an originating border of the selected angle. Each of the calibration angles is associated with a different originating border of the screen. The method includes by a vision sensor capturing at least one image of the calibrating target displayed on the screen. The calibration is repeated for each calibration angle of the sequence. The method includes performing calibration of the vision sensor in response to image signal processing of extracted calibration features in the at least one image of the calibrating targets.

Abstract: An apparatus and method for joint calibration of vision and radar sensors for an autonomous device is disclosed. The apparatus may include a spherical portion and a cutout portion. The cutout portion may be formed within the spherical portion and have three equal surfaces. Additionally, the apparatus may include a trihedral reflector positioned within the cutout portion.

Abstract: An apparatus and method for joint calibration of vision and radar sensors for an autonomous device is disclosed. The apparatus may include a spherical portion and a cutout portion. The cutout portion may be formed within the spherical portion and have three equal surfaces. Additionally, the apparatus may include a trihedral reflector positioned within the cutout portion.

Abstract: One embodiment of the present invention sets forth a technique for estimating a head pose of a user. The technique includes acquiring depth data associated with a head of the user and initializing each particle included in a set of particles with a different candidate head pose. The technique further includes performing one or more optimization passes that include performing at least one iterative closest point (ICP) iteration for each particle and performing at least one particle swarm optimization (PSO) iteration. Each ICP iteration includes rendering the three-dimensional reference model based on the candidate head pose associated with the particle and comparing the three-dimensional reference model to the depth data. Each PSO iteration comprises updating a global best head pose associated with the set of particles and modifying at least one candidate head pose. The technique further includes modifying a shape of the three-dimensional reference model based on depth data.

Abstract: An exemplary camera calibration apparatus includes a movable, e.g., rotatable, support structure which is controllably positioned to allow for image capture of different test patterns and image capture of the same pattern at different distances by a mounted camera. A first test pattern is mounted on a wall, e.g., a pyramid shaped 4 sided wall formed by panels surrounding the camera under calibration. The movable support structure has a first mirror attached to a first side and has a second test pattern attached to a second side. A second mirror mounted on an internal sidewall of the calibration apparatus housing facilities a different image path distance between the camera capturing the image of the first test pattern and the first test pattern. The exemplary camera calibration apparatus is well suited for efficiently calibrating camera devices including a plurality of camera modules, e.g., optical chains, in a relatively small area.

Abstract: System and method of displaying images in temporal superresolution by multiplicative superposition of cascaded display layers integrated in a display device. Using an original video with a target temporal resolution as a priori, a factorization process is performed to derive respective image data for presentation on each display layer. The multiple layers are refreshed in staggered intervals to synthesize a video with an effective refresh rate exceeding that of each individual display layer, e.g., by a factor equal to the number of layers. Further optically averaging neighboring pixels can minimize artifacts.

Abstract: One embodiment of the present invention sets forth a technique for estimating a head pose of a user. The technique includes acquiring depth data associated with a head of the user and initializing each particle included in a set of particles with a different candidate head pose. The technique further includes performing one or more optimization passes that include performing at least one iterative closest point (ICP) iteration for each particle and performing at least one particle swarm optimization (PSO) iteration. Each ICP iteration includes rendering the three-dimensional reference model based on the candidate head pose associated with the particle and comparing the three-dimensional reference model to the depth data. Each PSO iteration comprises updating a global best head pose associated with the set of particles and modifying at least one candidate head pose. The technique further includes modifying a shape of the three-dimensional reference model based on depth data.

Abstract: A computer implemented method of determining a latent image from an observed image is disclosed. The method comprises implementing a plurality of image processing operations within a single optimization framework, wherein the single optimization framework comprises solving a linear minimization expression. The method further comprises mapping the linear minimization expression onto at least one non-linear solver. Further, the method comprises using the non-linear solver, iteratively solving the linear minimization expression in order to extract the latent image from the observed image, wherein the linear minimization expression comprises: a data term, and a regularization term, and wherein the regularization term comprises a plurality of non-linear image priors.

Abstract: System and method of displaying images in spatial/temporal superresolution by multiplicative superposition of cascaded display layers integrated in a display device. Using an original image with a target spatial/temporal resolution as a priori, a factorization process is performed to derive respective image data for presentation on each display layer. The cascaded display layers may be progressive and laterally shifted with each other, resulting in an effective spatial resolution exceeding the native display resolutions of the display layers. Factorized images may be refreshed on respective display layers in synchronization or out of synchronization.

Abstract: One embodiment of the present invention sets forth a technique for estimating a head pose of a user. The technique includes acquiring depth data associated with a head of the user and initializing each particle included in a set of particles with a different candidate head pose. The technique further includes performing one or more optimization passes that include performing at least one iterative closest point (ICP) iteration for each particle and performing at least one particle swarm optimization (PSO) iteration. Each ICP iteration includes rendering the three-dimensional reference model based on the candidate head pose associated with the particle and comparing the three-dimensional reference model to the depth data. Each PSO iteration comprises updating a global best head pose associated with the set of particles and modifying at least one candidate head pose. The technique further includes modifying a shape of the three-dimensional reference model based on depth data.

Abstract: An exemplary camera calibration apparatus includes a movable, e.g., rotatable, support structure which is controllably positioned to allow for image capture of different test patterns and image capture of the same pattern at different distances by a mounted camera. A first test pattern is mounted on a wall, e.g., a pyramid shaped 4 sided wall formed by panels surrounding the camera under calibration. The movable support structure has a first mirror attached to a first side and has a second test pattern attached to a second side. A second mirror mounted on an internal sidewall of the calibration apparatus housing facilities a different image path distance between the camera capturing the image of the first test pattern and the first test pattern. The exemplary camera calibration apparatus is well suited for efficiently calibrating camera devices including a plurality of camera modules, e.g., optical chains, in a relatively small area.

Abstract: One embodiment of the present invention sets forth a technique for estimating a head pose of a user. The technique includes acquiring depth data associated with a head of the user and initializing each particle included in a set of particles with a different candidate head pose. The technique further includes performing one or more optimization passes that include performing at least one iterative closest point (ICP) iteration for each particle and performing at least one particle swarm optimization (PSO) iteration. Each ICP iteration includes rendering the three-dimensional reference model based on the candidate head pose associated with the particle and comparing the three-dimensional reference model to the depth data. Each PSO iteration comprises updating a global best head pose associated with the set of particles and modifying at least one candidate head pose. The technique further includes modifying a shape of the three-dimensional reference model based on depth data.

Abstract: A computer implemented method of determining a latent image from an observed image is disclosed. The method comprises implementing a plurality of image processing operations within a single optimization framework, wherein the single optimization framework comprises solving a linear minimization expression. The method further comprises mapping the linear minimization expression onto at least one non-linear solver. Further, the method comprises using the non-linear solver, iteratively solving the linear minimization expression in order to extract the latent image from the observed image, wherein the linear minimization expression comprises: a data term, and a regularization term, and wherein the regularization term comprises a plurality of non-linear image priors.

Abstract: A computer implemented method of determining a latent image from an observed image is disclosed. The method comprises implementing a plurality of image processing operations within a single optimization framework, wherein the single optimization framework comprises solving a linear minimization expression. The method further comprises mapping the linear minimization expression onto at least one non-linear solver. Further, the method comprises using the non-linear solver, iteratively solving the linear minimization expression in order to extract the latent image from the observed image, wherein the linear minimization expression comprises: a data term, and a regularization term, and wherein the regularization term comprises a plurality of non-linear image priors.

Abstract: System and method of displaying images in temporal superresolution by multiplicative superposition of cascaded display layers integrated in a display device. Using an original video with a target temporal resolution as a priori, a factorization process is performed to derive respective image data for presentation on each display layer. The multiple layers are refreshed in staggered intervals to synthesize a video with an effective refresh rate exceeding that of each individual display layer, e.g., by a factor equal to the number of layers. Further optically averaging neighboring pixels can minimize artifacts.

Abstract: System and method of displaying images in spatial/temporal superresolution by multiplicative superposition of cascaded display layers integrated in a display device. Using an original image with a target spatial/temporal resolution as a priori, a factorization process is performed to derive respective image data for presentation on each display layer. The cascaded display layers may be progressive and laterally shifted with each other, resulting in an effective spatial resolution exceeding the native display resolutions of the display layers. Factorized images may be refreshed on respective display layers in synchronization or out of synchronization.