Abstract: One embodiment of the present invention sets forth a technique for estimating a head pose of a user. The technique includes acquiring depth data associated with a head of the user and initializing each particle included in a set of particles with a different candidate head pose. The technique further includes performing one or more optimization passes that include performing at least one iterative closest point (ICP) iteration for each particle and performing at least one particle swarm optimization (PSO) iteration. Each ICP iteration includes rendering the three-dimensional reference model based on the candidate head pose associated with the particle and comparing the three-dimensional reference model to the depth data. Each PSO iteration comprises updating a global best head pose associated with the set of particles and modifying at least one candidate head pose. The technique further includes modifying a shape of the three-dimensional reference model based on depth data.

Abstract: A computer implemented method of determining a latent image from an observed image is disclosed. The method comprises implementing a plurality of image processing operations within a single optimization framework, wherein the single optimization framework comprises solving a linear minimization expression. The method further comprises mapping the linear minimization expression onto at least one non-linear solver. Further, the method comprises using the non-linear solver, iteratively solving the linear minimization expression in order to extract the latent image from the observed image, wherein the linear minimization expression comprises: a data term, and a regularization term, and wherein the regularization term comprises a plurality of non-linear image priors.

Abstract: System and method of displaying images in spatial/temporal superresolution by multiplicative superposition of cascaded display layers integrated in a display device. Using an original image with a target spatial/temporal resolution as a priori, a factorization process is performed to derive respective image data for presentation on each display layer. The cascaded display layers may be progressive and laterally shifted with each other, resulting in an effective spatial resolution exceeding the native display resolutions of the display layers. Factorized images may be refreshed on respective display layers in synchronization or out of synchronization.

Abstract: A method, computer readable medium, and system are disclosed for classifying video image data. The method includes the steps of processing training video image data by at least a first layer of a convolutional neural network (CNN) to extract a first set of feature maps and generate classification output data for the training video image data. Spatial classification accuracy data is computed based on the classification output data and target classification output data and spatial discrimination factors for the first layer are computed based on the spatial classification accuracies and the first set of feature maps.

Abstract: A method for fabricating custom surface reflectance and spatially-varying bi-directional reflectance distribution functions (BDRFs or svBRDFs). The 3D printing method optimizes micro-geometry to produce a normal distribution function (NDF) that can be printed on surfaces with a 3D printer. Particularly, the method involves optimizing the micro-geometry for a wide range of analytic NDFs and simulating the effective reflectance of the resulting surface. Using the results of the simulation, the appearance of an input svBRDF can be reproduced. To this end, the micro-geometry is optimized in a data-driven fashion and distributed on the surface of the printed object. The methods were demonstrated to allow 3D printing svBRDF on planar samples with current 3D printing technology even with a limited set of printing materials, and the described methods have been shown to be naturally extendable to printing svBRDF on arbitrary shapes or 3D objects.

Abstract: One embodiment of the present invention sets forth a technique for estimating a head pose of a user. The technique includes acquiring depth data associated with a head of the user and initializing each particle included in a set of particles with a different candidate head pose. The technique further includes performing one or more optimization passes that include performing at least one iterative closest point (ICP) iteration for each particle and performing at least one particle swarm optimization (PSO) iteration. Each ICP iteration includes rendering the three-dimensional reference model based on the candidate head pose associated with the particle and comparing the three-dimensional reference model to the depth data. Each PSO iteration comprises updating a global best head pose associated with the set of particles and modifying at least one candidate head pose. The technique further includes modifying a shape of the three-dimensional reference model based on depth data.

Abstract: A system, method and computer program product are provided for generating one or more values for a signal patch using neighboring patches collected based on a distance dynamically computed from a noise distribution of the signal patch. In use, a reference patch is identified from a signal, and a reference distance is computed based on a noise distribution in the reference patch. Neighbor patches are then collected from the signal based on the computed reference distance from the reference patch. Further, the collected neighbor patches are processed with the reference patch to generate one or more values for the reference patch.

Abstract: A method, computer readable medium, and system are disclosed for generating a Gaussian mixture model hierarchy. The method includes the steps of receiving point cloud data defining a plurality of points; defining a Gaussian Mixture Model (GMM) hierarchy that includes a number of mixels, each mixel encoding parameters for a probabilistic occupancy map; and adjusting the parameters for one or more probabilistic occupancy maps based on the point cloud data utilizing a number of iterations of an Expectation-Maximum (EM) algorithm.

Abstract: A lenticular display device that is effective in increasing both perceived angular resolution and spatial resolution. These desirable results are achieved by modifying the lenslet array to better match the content of a given light field. An optimization algorithm or method (which may be implemented with software run on a computing device) is provided that analyzes an input light field and computes an optimal lenslet size, shape, and arrangement of sets of lenslets across the width of the array to better (or even best) match the input light field given a set of output parameters. The resulting lenticular display device (or print) shows higher detail and smoother motion parallax compared with fixed-size lens arrays. The usefulness of these content-adaptive lenticular prints has been demonstrated or proven using rendered simulations, by generating 3D-printed lens arrays according to the present description, and with user studies.

Abstract: A method, computer readable medium, and system are disclosed for detecting and classifying hand gestures. The method includes the steps of receiving an unsegmented stream of data associated with a hand gesture, extracting spatio-temporal features from the unsegmented stream by a three-dimensional convolutional neural network (3DCNN), and producing a class label for the hand gesture based on the spatio-temporal features.

Abstract: One embodiment of the present invention sets forth a technique for estimating a head pose of a user. The technique includes acquiring depth data associated with a head of the user and initializing each particle included in a set of particles with a different candidate head pose. The technique further includes performing one or more optimization passes that include performing at least one iterative closest point (ICP) iteration for each particle and performing at least one particle swarm optimization (PSO) iteration. Each ICP iteration includes rendering the three-dimensional reference model based on the candidate head pose associated with the particle and comparing the three-dimensional reference model to the depth data. Each PSO iteration comprises updating a global best head pose associated with the set of particles and modifying at least one candidate head pose. The technique further includes modifying a shape of the three-dimensional reference model based on depth data.

Abstract: A computer implemented method of determining a latent image from an observed image is disclosed. The method comprises implementing a plurality of image processing operations within a single optimization framework, wherein the single optimization framework comprises solving a linear minimization expression. The method further comprises mapping the linear minimization expression onto at least one non-linear solver. Further, the method comprises using the non-linear solver, iteratively solving the linear minimization expression in order to extract the latent image from the observed image, wherein the linear minimization expression comprises: a data term, and a regularization term, and wherein the regularization term comprises a plurality of non-linear image priors.

Abstract: A computer implemented method of determining a latent image from an observed image is disclosed. The method comprises implementing a plurality of image processing operations within a single optimization framework, wherein the single optimization framework comprises solving a linear minimization expression. The method further comprises mapping the linear minimization expression onto at least one non-linear solver. Further, the method comprises using the non-linear solver, iteratively solving the linear minimization expression in order to extract the latent image from the observed image, wherein the linear minimization expression comprises: a data term, and a regularization term, and wherein the regularization term comprises a plurality of non-linear image priors.

Abstract: A method, computer readable medium, and system are disclosed for generating mixed-primary data for display. The method includes the steps of receiving a source image that includes a plurality of pixels, dividing the source image into a plurality of blocks, analyzing the source image based on an image decomposition algorithm, encoding chroma information and modulation information to generate a video signal, and transmitting the video signal to a mixed-primary display. The chroma information and modulation information correspond with two or more mixed-primary color components and are generated by the image decomposition algorithm to minimize error between a reproduced image and the source image. The two or more mixed-primary colors selected for each block of the source image are not limited to any particular set of colors and each mixed-primary color component may be selected from any color capable of being reproduced by the mixed-primary display.

Abstract: A method for fabricating custom surface reflectance and spatially-varying bi-directional reflectance distribution functions (BDRFs or svBRDFs). The 3D printing method optimizes micro-geometry to produce a normal distribution function (NDF) that can be printed on surfaces with a 3D printer. Particularly, the method involves optimizing the micro-geometry for a wide range of analytic NDFs and simulating the effective reflectance of the resulting surface. Using the results of the simulation, the appearance of an input svBRDF can be reproduced. To this end, the micro-geometry is optimized in a data-driven fashion and distributed on the surface of the printed object. The methods were demonstrated to allow 3D printing svBRDF on planar samples with current 3D printing technology even with a limited set of printing materials, and the described methods have been shown to be naturally extendable to printing svBRDF on arbitrary shapes or 3D objects.

Abstract: A method for fabricating custom surface reflectance and spatially-varying bi-directional reflectance distribution functions (BDRFs or svBRDFs). The 3D printing method optimizes micro-geometry to produce a normal distribution function (NDF) that can be printed on surfaces with a 3D printer. Particularly, the method involves optimizing the micro-geometry for a wide range of analytic NDFs and simulating the effective reflectance of the resulting surface. Using the results of the simulation, the appearance of an input svBRDF can be reproduced. To this end, the micro-geometry is optimized in a data-driven fashion and distributed on the surface of the printed object. The methods were demonstrated to allow 3D printing svBRDF on planar samples with current 3D printing technology even with a limited set of printing materials, and the described methods have been shown to be naturally extendable to printing svBRDF on arbitrary shapes or 3D objects.

Abstract: A three-dimensional relief can be produced from one or more two-dimensional digital (2D) images. A height field is computed from the 2D images and illumination direction information. The height field comprises a multiplicity of geometric surface elements arrayed in a 2D field corresponding to the pixels of the one or more 2D images. Each geometric surface element corresponds to a pixel of each of the digital images and has at least one height parameter representing a displacement from a surface floor. Once the height field is computed, optimizations can be made to the height field including adding and adjusting albedo and glossy surface finishing. The height field can be used to fabricate relief elements in a material, such that each relief element corresponds in shape, position in the height field, and height above the surface floor, to one of the geometric surface elements in the height field.

Abstract: System and method of displaying images in temporal superresolution by multiplicative superposition of cascaded display layers integrated in a display device. Using an original video with a target temporal resolution as a priori, a factorization process is performed to derive respective image data for presentation on each display layer. The multiple layers are refreshed in staggered intervals to synthesize a video with an effective refresh rate exceeding that of each individual display layer, e.g., by a factor equal to the number of layers. Further optically averaging neighboring pixels can minimize artifacts.

Abstract: System and method of displaying images in spatial/temporal superresolution by multiplicative superposition of cascaded display layers integrated in a display device. Using an original image with a target spatial/temporal resolution as a priori, a factorization process is performed to derive respective image data for presentation on each display layer. The cascaded display layers may be progressive and laterally shifted with each other, resulting in an effective spatial resolution exceeding the native display resolutions of the display layers. Factorized images may be refreshed on respective display layers in synchronization or out of synchronization.

Abstract: A computer implemented method of determining a latent image from an observed image is disclosed. The method comprises implementing a plurality of image processing operations within a single optimization framework, wherein the single optimization framework comprises solving a linear minimization expression. The method further comprises mapping the linear minimization expression onto at least one non-linear solver. Further, the method comprises using the non-linear solver, iteratively solving the linear minimization expression in order to extract the latent image from the observed image, wherein the linear minimization expression comprises: a data term, and a regularization term, and wherein the regularization term comprises a plurality of non-linear image priors.