Patents by Inventor Jonathan Tilton Barron
Jonathan Tilton Barron has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240005590Abstract: Techniques of image synthesis using a neural radiance field (NeRF) includes generating a deformation model of movement experienced by a subject in a non-rigidly deforming scene. For example, when an image synthesis system uses NeRFs, the system takes as input multiple poses of subjects for training data. In contrast to conventional NeRFs, the technical solution first expresses the positions of the subjects from various perspectives in an observation frame. The technical solution then involves deriving a deformation model, i.e., a mapping between the observation frame and a canonical frame in which the subject's movements are taken into account. This mapping is accomplished using latent deformation codes for each pose that are determined using a multilayer perceptron (MLP). A NeRF is then derived from positions and casted ray directions in the canonical frame using another MLP. New poses for the subject may then be derived using the NeRF.Type: ApplicationFiled: January 14, 2021Publication date: January 4, 2024Inventors: Ricardo Martin Brualla, Keunhong Park, Utkarsh Sinha, Sofien Bouaziz, Daniel Goldman, Jonathan Tilton Barron, Steven Maxwell Seitz
-
Publication number: 20230360182Abstract: Apparatus and methods related to applying lighting models to images of objects are provided. An example method includes applying a geometry model to an input image to determine a surface orientation map indicative of a distribution of lighting on an object based on a surface geometry. The method further includes applying an environmental light estimation model to the input image to determine a direction of synthetic lighting to be applied to the input image. The method also includes applying, based on the surface orientation map and the direction of synthetic lighting, a light energy model to determine a quotient image indicative of an amount of light energy to be applied to each pixel of the input image. The method additionally includes enhancing, based on the quotient image, a portion of the input image. One or more neural networks can be trained to perform one or more of the aforementioned aspects.Type: ApplicationFiled: May 17, 2021Publication date: November 9, 2023Inventors: Sean Ryan Francesco Fanello, Yun-Ta Tsai, Rohit Kumar Pandey, Paul Debevec, Michael Milne, Chloe LeGendre, Jonathan Tilton Barron, Christoph Rhemann, Sofien Bouaziz, Navin Padman Sarma
-
Publication number: 20230306655Abstract: Provided are systems and methods for synthesizing novel views of complex scenes (e.g., outdoor scenes). In some implementations, the systems and methods can include or use machine-learned models that are capable of learning from unstructured and/or unconstrained collections of imagery such as, for example, “in the wild” photographs. In particular, example implementations of the present disclosure can learn a volumetric scene density and radiance represented by a machine-learned model such as one or more multilayer perceptrons (MLPs).Type: ApplicationFiled: June 1, 2023Publication date: September 28, 2023Inventors: Daniel Christopher Duckworth, Alexey Dosovitskiy, Ricardo Martin-Brualla, Jonathan Tilton Barron, Noha Radwan, Seyed Mohammad Mehdi Sajjadi
-
Publication number: 20230281913Abstract: Systems and methods for view synthesis and three-dimensional reconstruction can learn an environment by utilizing a plurality of images of an environment and depth data. The use of depth data can be helpful when the quantity of images and different angles may be limited. For example, large outdoor environments can be difficult to learn due to the size, the varying image exposures, and the limited variance in view direction changes. The systems and methods can leverage a plurality of panoramic images and corresponding lidar data to accurately learn a large outdoor environment to then generate view synthesis outputs and three-dimensional reconstruction outputs. Training may include the use of an exposure correction network to address lighting exposure differences between training images.Type: ApplicationFiled: March 4, 2022Publication date: September 7, 2023Inventors: Konstantinos Rematas, Thomas Allen Funkhouser, Vittorio Carlo Ferrari, Andrew Huaming Liu, Andrea Tagliasacchi, Pratul Preeti Srinivasan, Jonathan Tilton Barron
-
Publication number: 20230230275Abstract: Provided are systems and methods that invert a trained NeRF model, which stores the structure of a scene or object, to estimate the 6D pose from an image taken with a novel view. 6D pose estimation has a wide range of applications, including visual localization and object pose estimation for robot manipulation.Type: ApplicationFiled: November 15, 2021Publication date: July 20, 2023Inventors: Tsung-Yi Lin, Peter Raymond Florence, Yen-Chen Lin, Jonathan Tilton Barron
-
Patent number: 11704844Abstract: Provided are systems and methods for synthesizing novel views of complex scenes (e.g., outdoor scenes). In some implementations, the systems and methods can include or use machine-learned models that are capable of learning from unstructured and/or unconstrained collections of imagery such as, for example, “in the wild” photographs. In particular, example implementations of the present disclosure can learn a volumetric scene density and radiance represented by a machine-learned model such as one or more multilayer perceptrons (MLPs).Type: GrantFiled: April 18, 2022Date of Patent: July 18, 2023Assignee: GOOGLE LLCInventors: Daniel Christopher Duckworth, Alexey Dosovitskiy, Ricardo Martin Brualla, Jonathan Tilton Barron, Noha Waheed Ahmed Radwan, Seyed Mohammad Mehdi Sajjadi
-
Publication number: 20230177822Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for rendering a new image that depicts a scene from a perspective of a camera at a new camera viewpoint.Type: ApplicationFiled: December 2, 2022Publication date: June 8, 2023Inventors: Vincent Michael Casser, Henrik Kretzschmar, Matthew Justin Tancik, Sabeek Mani Pradhan, Benjamin Joseph Mildenhall, Pratul Preeti Srinivasan, Jonathan Tilton Barron
-
Publication number: 20220237834Abstract: Provided are systems and methods for synthesizing novel views of complex scenes (e.g., outdoor scenes). In some implementations, the systems and methods can include or use machine-learned models that are capable of learning from unstructured and/or unconstrained collections of imagery such as, for example, “in the wild” photographs. In particular, example implementations of the present disclosure can learn a volumetric scene density and radiance represented by a machine-learned model such as one or more multilayer perceptrons (MLPs).Type: ApplicationFiled: April 18, 2022Publication date: July 28, 2022Inventors: Daniel Christopher Duckworth, Alexey Dosovitskiy, Ricardo Martin Brualla, Jonathan Tilton Barron, Noha Waheed Ahmed Radwan, Seyed Mohammad Mehdi Sajjadi
-
Patent number: 11308659Abstract: Provided are systems and methods for synthesizing novel views of complex scenes (e.g., outdoor scenes). In some implementations, the systems and methods can include or use machine-learned models that are capable of learning from unstructured and/or unconstrained collections of imagery such as, for example, “in the wild” photographs. In particular, example implementations of the present disclosure can learn a volumetric scene density and radiance represented by a machine-learned model such as one or more multilayer perceptrons (MLPs).Type: GrantFiled: July 30, 2021Date of Patent: April 19, 2022Assignee: GOOGLE LLCInventors: Daniel Christopher Duckworth, Seyed Mohammad Mehdi Sajjadi, Jonathan Tilton Barron, Noha Radwan, Alexey Dosovitskiy, Ricardo Martin-Brualla
-
Publication number: 20220036602Abstract: Provided are systems and methods for synthesizing novel views of complex scenes (e.g., outdoor scenes). In some implementations, the systems and methods can include or use machine-learned models that are capable of learning from unstructured and/or unconstrained collections of imagery such as, for example, “in the wild” photographs. In particular, example implementations of the present disclosure can learn a volumetric scene density and radiance represented by a machine-learned model such as one or more multilayer perceptrons (MLPs).Type: ApplicationFiled: July 30, 2021Publication date: February 3, 2022Inventors: Daniel Christopher Duckworth, Seyed Mohammad Mehdi Sajjadi, Jonathan Tilton Barron, Noha Waheed Ahmed Radwan, Alexey Dosovitskiy, Ricardo Martin-Brualla
-
Patent number: 10897609Abstract: The present disclosure relates to methods and systems that may improve and/or modify images captured using multiscopic image capture systems. In an example embodiment, burst image data is captured via a multiscopic image capture system. The burst image data may include at least one image pair. The at least one image pair is aligned based on at least one rectifying homography function. The at least one aligned image pair is warped based on a stereo disparity between the respective images of the image pair. The warped and aligned images are then stacked and a denoising algorithm is applied. Optionally, a high dynamic range algorithm may be applied to at least one output image of the aligned, warped, and denoised images.Type: GrantFiled: November 11, 2019Date of Patent: January 19, 2021Assignee: Google LLCInventors: Jonathan Tilton Barron, Stephen Joseph DiVerdi, Ryan Geiss
-
Publication number: 20200084429Abstract: The present disclosure relates to methods and systems that may improve and/or modify images captured using multiscopic image capture systems. In an example embodiment, burst image data is captured via a multiscopic image capture system. The burst image data may include at least one image pair. The at least one image pair is aligned based on at least one rectifying homography function. The at least one aligned image pair is warped based on a stereo disparity between the respective images of the image pair. The warped and aligned images are then stacked and a denoising algorithm is applied. Optionally, a high dynamic range algorithm may be applied to at least one output image of the aligned, warped, and denoised images.Type: ApplicationFiled: November 11, 2019Publication date: March 12, 2020Inventors: Jonathan Tilton Barron, Stephen Joseph DiVerdi, Ryan Geiss
-
Patent number: 10477185Abstract: The present disclosure relates to methods and systems that may improve and/or modify images captured using multiscopic image capture systems. In an example embodiment, burst image data is captured via a multiscopic image capture system. The burst image data may include at least one image pair. The at least one image pair is aligned based on at least one rectifying homography function. The at least one aligned image pair is warped based on a stereo disparity between the respective images of the image pair. The warped and aligned images are then stacked and a denoising algorithm is applied. Optionally, a high dynamic range algorithm may be applied to at least one output image of the aligned, warped, and denoised images.Type: GrantFiled: December 20, 2018Date of Patent: November 12, 2019Assignee: Google LLCInventors: Jonathan Tilton Barron, Stephen Joseph DiVerdi, Ryan Geiss
-
Publication number: 20190124319Abstract: The present disclosure relates to methods and systems that may improve and/or modify images captured using multiscopic image capture systems. In an example embodiment, burst image data is captured via a multiscopic image capture system. The burst image data may include at least one image pair. The at least one image pair is aligned based on at least one rectifying homography function. The at least one aligned image pair is warped based on a stereo disparity between the respective images of the image pair. The warped and aligned images are then stacked and a denoising algorithm is applied. Optionally, a high dynamic range algorithm may be applied to at least one output image of the aligned, warped, and denoised images.Type: ApplicationFiled: December 20, 2018Publication date: April 25, 2019Inventors: Jonathan Tilton Barron, Stephen Joseph DiVerdi, Ryan Geiss
-
Patent number: 10187628Abstract: The present disclosure relates to methods and systems that may improve and/or modify images captured using multiscopic image capture systems. In an example embodiment, burst image data is captured via a multiscopic image capture system. The burst image data may include at least one image pair. The at least one image pair is aligned based on at least one rectifying homography function. The at least one aligned image pair is warped based on a stereo disparity between the respective images of the image pair. The warped and aligned images are then stacked and a denoising algorithm is applied. Optionally, a high dynamic range algorithm may be applied to at least one output image of the aligned, warped, and denoised images.Type: GrantFiled: August 14, 2017Date of Patent: January 22, 2019Assignee: Google LLCInventors: Jonathan Tilton Barron, Stephen Joseph DiVerdi, Ryan Geiss
-
Patent number: 9892496Abstract: Example embodiments may allow for the efficient, edge-preserving filtering, upsampling, or other processing of image data with respect to a reference image. A cost-minimization problem to generate an output image from the input array is mapped onto regularly-spaced vertices in a multidimensional vertex space. This mapping is based on an association between pixels of the reference image and the vertices, and between elements of the input array and the pixels of the reference image. The problem is them solved to determine vertex disparity values for each of the vertices. Pixels of the output image can be determined based on determined vertex disparity values for respective one or more vertices associated with each of the pixels. This fast, efficient image processing method can be used to enable edge-preserving image upsampling, image colorization, semantic segmentation of image contents, image filtering or de-noising, or other applications.Type: GrantFiled: November 4, 2016Date of Patent: February 13, 2018Assignee: Google LLCInventors: Jonathan Tilton Barron, Benjamin Michael Poole
-
Publication number: 20170347088Abstract: The present disclosure relates to methods and systems that may improve and/or modify images captured using multiscopic image capture systems. In an example embodiment, burst image data is captured via a multiscopic image capture system. The burst image data may include at least one image pair. The at least one image pair is aligned based on at least one rectifying homography function. The at least one aligned image pair is warped based on a stereo disparity between the respective images of the image pair. The warped and aligned images are then stacked and a denoising algorithm is applied. Optionally, a high dynamic range algorithm may be applied to at least one output image of the aligned, warped, and denoised images.Type: ApplicationFiled: August 14, 2017Publication date: November 30, 2017Inventors: Jonathan Tilton Barron, Stephen Joseph DiVerdi, Ryan Geiss
-
Patent number: 9762893Abstract: The present disclosure relates to methods and systems that may improve and/or modify images captured using multiscopic image capture systems. In an example embodiment, burst image data is captured via a multiscopic image capture system. The burst image data may include at least one image pair. The at least one image pair is aligned based on at least one rectifying homography function. The at least one aligned image pair is warped based on a stereo disparity between the respective images of the image pair. The warped and aligned images are then stacked and a denoising algorithm is applied. Optionally, a high dynamic range algorithm may be applied to at least one output image of the aligned, warped, and denoised images.Type: GrantFiled: December 7, 2015Date of Patent: September 12, 2017Assignee: Google Inc.Inventors: Jonathan Tilton Barron, Stephen Joseph DiVerdi, Ryan Geiss
-
Patent number: 9736451Abstract: Example embodiments may allow for the efficient determination of disparity information for a stereo image pair by embedding pixels of the image pair in a multidimensional dimensional vertex space. Regularly-spaced vertices in the vertex space are associated with pixels of the stereo image pair and disparity loss functions are determined for each of the vertices based on disparity loss functions of the associated pixels. The determined vertex-disparity loss functions can be used to determine vertex disparity values for each of the vertices. Disparity values for pixels of the stereo image pair can be determined based on determined vertex disparity values for respective one or more vertices associated with each of the pixels. The determined pixel disparity values can be used to enable depth-selective image processing, determination of pixel depth maps, mapping and/or navigation of an environment, human-computer interfacing, biometrics, augmented reality, or other applications.Type: GrantFiled: December 29, 2016Date of Patent: August 15, 2017Assignee: Google IncInventor: Jonathan Tilton Barron
-
Publication number: 20170163966Abstract: The present disclosure relates to methods and systems that may improve and/or modify images captured using multiscopic image capture systems. In an example embodiment, burst image data is captured via a multiscopic image capture system. The burst image data may include at least one image pair. The at least one image pair is aligned based on at least one rectifying homography function. The at least one aligned image pair is warped based on a stereo disparity between the respective images of the image pair. The warped and aligned images are then stacked and a denoising algorithm is applied. Optionally, a high dynamic range algorithm may be applied to at least one output image of the aligned, warped, and denoised images.Type: ApplicationFiled: December 7, 2015Publication date: June 8, 2017Inventors: Jonathan Tilton Barron, Stephen Joseph DiVerdi, Ryan Geiss