Patents by Inventor Vladlen Koltun
Vladlen Koltun has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11972519Abstract: Described herein are techniques for learning neural reflectance shaders from images. A set of one or more machine learning models can be trained to optimize an illumination latent code and a set of reflectance latent codes for an object within a set of input images. A shader can then be generated based on a machine learning model of the one or more machine learning models. The shader is configured to sample the illumination latent code and the set of reflectance latent codes for the object. A 3D representation of the object can be rendered using the generated shader.Type: GrantFiled: June 24, 2022Date of Patent: April 30, 2024Assignee: Intel CorporationInventors: Benjamin Ummenhofer, Shenlong Wang, Sanskar Agrawal, Yixing Lao, Kai Zhang, Stephan Richter, Vladlen Koltun
-
Patent number: 11928787Abstract: Systems, apparatuses and methods may provide for technology that estimates poses of a plurality of input images, reconstructs a proxy three-dimensional (3D) geometry based on the estimated poses and the plurality of input images, detects a user selection of a virtual viewpoint, encodes, via a first neural network, the plurality of input images with feature maps, warps the feature maps of the encoded plurality of input images based on the virtual viewpoint and the proxy 3D geometry, and blends, via a second neural network, the warped feature maps into a single image, wherein the first neural network is deep convolutional network and the second neural network is a recurrent convolutional network.Type: GrantFiled: September 22, 2020Date of Patent: March 12, 2024Assignee: Intel CorporationInventors: Gernot Riegler, Vladlen Koltun
-
Patent number: 11875252Abstract: Some embodiments are directed to a neural network training device for training a neural network. At least one layer of the neural network layers is a projection layer. The projection layer projects a layer input vector (x) of the projection layer to a layer output vector (y). The output vector (y) sums to the summing parameter (k).Type: GrantFiled: May 17, 2019Date of Patent: January 16, 2024Inventors: Brandon David Amos, Vladlen Koltun, Jeremy Zieg Kolter, Frank Rüdiger Schmidt
-
Patent number: 11816784Abstract: Systems, apparatuses and methods may provide for technology that generates, by a first neural network, an initial set of model weights based on input data and iteratively generates, by a second neural network, an updated set of model weights based on residual data associated with the initial set of model weights and the input data. Additionally, the technology may output a geometric model of the input data based on the updated set of model weights. In one example, the first neural network and the second neural network reduce the dependence of the geometric model on the number of data points in the input data.Type: GrantFiled: June 15, 2022Date of Patent: November 14, 2023Assignee: Intel CorporationInventors: Rene Ranftl, Vladlen Koltun
-
Publication number: 20230343014Abstract: Described herein are techniques for learning neural reflectance shaders from images. A set of one or more machine learning models can be trained to optimize an illumination latent code and a set of reflectance latent codes for an object within a set of input images. A shader can then be generated based on a machine learning model of the one or more machine learning models. The shader is configured to sample the illumination latent code and the set of reflectance latent codes for the object. A 3D representation of the object can be rendered using the generated shader.Type: ApplicationFiled: June 24, 2022Publication date: October 26, 2023Applicant: Intel CorporationInventors: Benjamin Ummenhofer, Shenlong Wang, Sanskar Agrawal, Yixing Lao, Kai Zhang, Stephan Richter, Vladlen Koltun
-
Publication number: 20230113271Abstract: Methods, apparatus, systems and articles of manufacture disclosed herein perform dense prediction of an input image using transformers at an encoder stage and at a reassembly stage of an image processing system. A disclosed apparatus includes an encoder with an embedder to convert an input image to a plurality of tokens representing features extracted from the input image. The tokens are embedded with a learnable position embedding. The encoder also includes one or more transformers configured in a sequence of stages to relate the tokens to each other. The apparatus further includes a decoder that includes one or more of reassemblers to assemble the tokens into feature representations, one or more of fusion blocks to combine the feature representations to generate a final feature representation, and an output head to generate a dense prediction based on the final feature representation and based on an output task.Type: ApplicationFiled: June 30, 2022Publication date: April 13, 2023Inventors: Renee Ranftl, Alexey Bochkovskiy, Vladlen Koltun
-
Publication number: 20230102866Abstract: Systems and methods for operating a deep equilibrium (DEQ) model in a neural network are disclosed. DEQs solve for a fixed point of a single nonlinear layer, which enables decoupling the internal structure of the layer from how the fixed point is actually computed. This disclosure discloses that such decoupling can be exploited while substantially enhancing this fixed point computation using a custom neural solver.Type: ApplicationFiled: September 27, 2022Publication date: March 30, 2023Inventors: Shaojie BAI, Vladlen KOLTUN, Jeremy KOLTER, Devin T. WILLMOTT, João D. SEMEDO
-
Patent number: 11610129Abstract: A computer-implemented method for a classification and training a neural network includes receiving input at the neural network, wherein the input includes a plurality of resolution inputs of varying resolutions, outputting a plurality of feature tensors for each corresponding resolution of the plurality of resolution inputs, fusing the plurality of feature tensors utilizing upsampling or down sampling for the vary resolutions, utilizing an equilibrium solver to identify one or more prediction vectors from the plurality of feature tensors, and outputting a loss in response to the one or more prediction vectors.Type: GrantFiled: June 8, 2020Date of Patent: March 21, 2023Inventors: Shaojie Bai, Jeremy Kolter, Vladlen Koltun, Devin T. Willmott
-
Publication number: 20220398480Abstract: Regularized training of a Deep Equilibrium Model (DEQ) is provided. A regularization term is computed using a predefined quantity of random samples and the Jacobian matrix of the DEQ, the regularization term penalizing the spectral radius of the Jacobian matrix. The regularization term is included in an original loss function of the DEQ to form a regularized loss function. A gradient of the regularized loss function is computed with respect to model parameters of the DEQ. The gradient is used to update the model parameters.Type: ApplicationFiled: June 9, 2021Publication date: December 15, 2022Inventors: Shaojie BAI, Vladlen KOLTUN, J. Zico KOLTER, Devin T. WILLMOTT, João D. SEMEDO
-
Publication number: 20220343521Abstract: Methods, apparatus, systems, and articles of manufacture are disclosed for metric depth estimation using a monocular visual-inertial system. An example apparatus for metric depth estimation includes at least one memory, instructions in the apparatus, and processor circuitry to execute the instructions to access a globally-aligned depth prediction, the globally-aligned depth prediction generated based on a monocular depth estimator, access a dense scale map scaffolding, the dense scale map scaffolding generated based on visual-inertial odometry, regress a dense scale residual map determined using the globally-aligned depth prediction and the dense scale map scaffolding, and apply the dense scale residual map to the globally-aligned depth prediction.Type: ApplicationFiled: June 30, 2022Publication date: October 27, 2022Inventors: Diana Wofk, Rene Ranftl, Matthias Mueller, Vladlen Koltun
-
Publication number: 20220309739Abstract: Systems, apparatuses and methods may provide for technology that generates, by a first neural network, an initial set of model weights based on input data and iteratively generates, by a second neural network, an updated set of model weights based on residual data associated with the initial set of model weights and the input data. Additionally, the technology may output a geometric model of the input data based on the updated set of model weights. In one example, the first neural network and the second neural network reduce the dependence of the geometric model on the number of data points in the input data.Type: ApplicationFiled: June 15, 2022Publication date: September 29, 2022Applicant: Intel CorporationInventors: Rene Ranftl, Vladlen Koltun
-
Patent number: 11393160Abstract: Systems, apparatuses and methods may provide for technology that generates, by a first neural network, an initial set of model weights based on input data and iteratively generates, by a second neural network, an updated set of model weights based on residual data associated with the initial set of model weights and the input data. Additionally, the technology may output a geometric model of the input data based on the updated set of model weights. In one example, the first neural network and the second neural network reduce the dependence of the geometric model on the number of data points in the input data.Type: GrantFiled: March 23, 2018Date of Patent: July 19, 2022Assignee: Intel CorporationInventors: Rene Ranftl, Vladlen Koltun
-
Publication number: 20220075555Abstract: Systems, apparatuses and methods may provide for technology that selects elements of a multi-scale kernel according to resolutions in an adaptive grid, conducts convolutions on the adaptive grid with the selected elements of the multi-scale kernel, and generates a signed distance field based on the convolutions.Type: ApplicationFiled: November 17, 2021Publication date: March 10, 2022Applicant: Intel CorporationInventors: Benjamin Ummenhofer, Vladlen Koltun
-
Publication number: 20220028026Abstract: Apparatus and method for enhancing graphics rendering photorealism. For example, one embodiment of a graphics processor comprises: a graphics processing pipeline comprising a plurality of graphics processing stages to render a graphics image; a local storage to store intermediate rendering data to generate the graphics image; and machine-learning hardware logic to perform a refinement operation on the graphics image using at least a portion of the intermediate rendering data to generate a translated image.Type: ApplicationFiled: July 27, 2020Publication date: January 27, 2022Inventors: Stephan Richter, Vladlen Koltun, Hassan Abu Alhaija
-
Publication number: 20220012848Abstract: Methods, apparatus, systems and articles of manufacture disclosed herein perform dense prediction of an input image using transformers at an encoder stage and at a reassembly stage of an image processing system. A disclosed apparatus includes an encoder with an embedder to convert an input image to a plurality of tokens representing features extracted from the input image. The tokens are embedded with a learnable position embedding. The encoder also includes one or more transformers configured in a sequence of stages to relate the tokens to each other. The apparatus further includes a decoder that includes one or more of reassemblers to assemble the tokens into feature representations, one or more of fusion blocks to combine the feature representations to generate a final feature representation, and an output head to generate a dense prediction based on the final feature representation and based on an output task.Type: ApplicationFiled: September 25, 2021Publication date: January 13, 2022Inventors: Rene Ranftl, Alexey Bochkovskiy, Vladlen Koltun
-
Publication number: 20210383234Abstract: A computer-implemented method for a classification and training a neural network includes receiving input at the neural network, wherein the input includes a plurality of resolution inputs of varying resolutions, outputting a plurality of feature tensors for each corresponding resolution of the plurality of resolution inputs, fusing the plurality of feature tensors utilizing upsampling or down sampling for the vary resolutions, utilizing an equilibrium solver to identify one or more prediction vectors from the plurality of feature tensors, and outputting a loss in response to the one or more prediction vectors.Type: ApplicationFiled: June 8, 2020Publication date: December 9, 2021Inventors: Shaojie BAI, Jeremy KOLTER, Vladlen KOLTUN, Devin T. WILLMOTT
-
Publication number: 20210319324Abstract: Systems, apparatuses and methods may provide for technology that trains a reversible graph neural network (GNN) by partitioning an input vertex feature matrix into a plurality of groups, generating, via a block of the reversible GNN, outputs for the plurality of groups based on an adjacency matrix and an edge feature matrix, wherein the outputs are generated during one or more forward propagations, conducting a reconstruction of the input feature matrix during one or more backward propagations, and excluding the adjacency matrix and the edge feature matrix from the reconstruction. The technology also trains a deep equilibrium GNN.Type: ApplicationFiled: June 25, 2021Publication date: October 14, 2021Applicant: Intel CorporationInventors: Matthias Mueller, Vladlen Koltun, Guohao Li
-
Publication number: 20210319319Abstract: Methods, apparatus, systems, and articles of manufacture are disclosed to implement parallel architectures for neural network classifiers. An example non-transitory computer readable medium comprises instructions that, when executed, cause a machine to at least: process a first stream using first neural network blocks, the first stream based on an input image; process a second stream using second neural network blocks, the second stream based on the input image; fuse a result of the first neural network blocks and the second neural network blocks; perform average pooling on the fused result; process a fully connected layer based on the result of the average pooling; and classify the image based on the output of the fully connected layer.Type: ApplicationFiled: June 25, 2021Publication date: October 14, 2021Inventors: Ankit Goyal, Alexey Bochkovkiy, Vladlen Koltun
-
Publication number: 20210012576Abstract: Systems, apparatuses and methods may provide for technology that estimates poses of a plurality of input images, reconstructs a proxy three-dimensional (3D) geometry based on the estimated poses and the plurality of input images, detects a user selection of a virtual viewpoint, encodes, via a first neural network, the plurality of input images with feature maps, warps the feature maps of the encoded plurality of input images based on the virtual viewpoint and the proxy 3D geometry, and blends, via a second neural network, the warped feature maps into a single image, wherein the first neural network is deep convolutional network and the second neural network is a recurrent convolutional network.Type: ApplicationFiled: September 22, 2020Publication date: January 14, 2021Inventors: Gernot Riegler, Vladlen Koltun
-
Publication number: 20200364553Abstract: Some embodiments are directed to a neural network training device for training a neural network. At least one layer of the neural network layers is a projection layer. The projection layer projects a layer input vector (x) of the projection layer to a layer output vector (y). The output vector (y) sums to the summing parameter (k).Type: ApplicationFiled: May 17, 2019Publication date: November 19, 2020Inventors: Brandon David Amos, Vladlen Koltun, Jeremy Zieg Kolter, Frank Rüdiger Schmidt