Patents by Inventor Jonathan T. Barron
Jonathan T. Barron has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20230351560Abstract: Systems and methods described herein may relate to potential methods of training a machine learning model to be implemented on a mobile computing device configured to capture, adjust, and/or store image frames. An example method includes supplying a first image frame of a subject in a setting lit within a first lighting environment and supplying a second image frame of the subject lit within a second lighting environment. The method further includes determining a mask. Additionally, the method includes combining the first image frame and the second image frame according to the mask to generate a synthetic image and assigning a score to the synthetic image. The method also includes training a machine learning model based on the assigned score to adjust a captured image based on the synthetic image.Type: ApplicationFiled: December 23, 2019Publication date: November 2, 2023Inventors: David Jacobs, Yun-Ta Tsai, Jonathan T. Barron, Xuaner Zhang
-
Publication number: 20220375042Abstract: A method includes obtaining dual-pixel image data that includes a first sub-image and a second sub-image, and generating an in-focus image, a first kernel corresponding to the first sub-image, and a second kernel corresponding to the second sub-image. A loss value may be determined using a loss function that determines a difference between (i) a convolution of the first sub-image with the second kernel and (ii) a convolution of the second sub-image with the first kernel, and/or a sum of (i) a difference between the first sub-image and a convolution of the in-focus image with the first kernel and (ii) a difference between the second sub-image and a convolution of the in-focus image with the second kernel. Based on the loss value and the loss function, the in-focus image, the first kernel, and/or the second kernel, may be updated and displayed.Type: ApplicationFiled: November 13, 2020Publication date: November 24, 2022Inventors: Rahul Garg, Neal Wadhwa, Pratul Preeti Srinivasan, Tianfan Xue, Jiawen Chen, Shumian Xin, Jonathan T. Barron
-
Publication number: 20220375045Abstract: A method includes obtaining an input image that contains a particular representation of lens flare, and processing the input image by a machine learning model to generate a de-flared image that includes the input image with at least part of the particular representation of lens flare removed. The machine learning (ML) model may be trained by generating training images that combine respective baseline images with corresponding lens flare images. For each respective training image, a modified image may be determined by processing the respective training image by the ML model, and a loss value may be determined based on a loss function comparing the modified image to a corresponding baseline image used to generate the respective training image. Parameters of the ML model may be adjusted based on the loss value determined for each respective training image and the loss function.Type: ApplicationFiled: November 9, 2020Publication date: November 24, 2022Inventors: Yicheng Wu, Qiurui He, Tianfan Xue, Rahul Garg, Jiawen Chen, Jonathan T. Barron
-
Publication number: 20220327769Abstract: Examples relate to implementations of a neural light transport. A computing system may obtain data indicative of a plurality of UV texture maps and a geometry of an object. Each UV texture map depicts the object from a perspective of a plurality of perspectives. The computing system may train a neural network to learn a light transport function using the data. The light transport function may be a continuous function that specifies how light interacts with the object when the object is viewed from the plurality of perspectives. The computing system may generate an output UV texture map that depicts the object from a synthesized perspective based on an application of the light transport function by the trained neural network.Type: ApplicationFiled: May 4, 2020Publication date: October 13, 2022Inventors: Yun-Ta TSAI, Xiuming ZHANG, Jonathan T. BARRON, Sean FANELLO, Tiancheng SUN, Tianfan XUE
-
Patent number: 11145075Abstract: A handheld user device includes a monocular camera to capture a feed of images of a local scene and a processor to select, from the feed, a keyframe and perform, for a first image from the feed, stereo matching using the first image, the keyframe, and a relative pose based on a pose associated with the first image and a pose associated with the keyframe to generate a sparse disparity map representing disparities between the first image and the keyframe. The processor further is to determine a dense depth map from the disparity map using a bilateral solver algorithm, and process a viewfinder image generated from a second image of the feed with occlusion rendering based on the depth map to incorporate one or more virtual objects into the viewfinder image to generate an AR viewfinder image. Further, the processor is to provide the AR viewfinder image for display.Type: GrantFiled: October 4, 2019Date of Patent: October 12, 2021Assignee: Google LLCInventors: Julien Valentin, Onur G. Guleryuz, Mira Leung, Maksym Dzitsiuk, Jose Pascoal, Mirko Schmidt, Christoph Rhemann, Neal Wadhwa, Eric Turner, Sameh Khamis, Adarsh Prakash Murthy Kowdle, Ambrus Csaszar, João Manuel Castro Afonso, Jonathan T. Barron, Michael Schoenberg, Ivan Dryanovski, Vivek Verma, Vladimir Tankovich, Shahram Izadi, Sean Ryan Francesco Fanello, Konstantine Nicholas John Tsotsos
-
Publication number: 20210004979Abstract: A handheld user device includes a monocular camera to capture a feed of images of a local scene and a processor to select, from the feed, a keyframe and perform, for a first image from the feed, stereo matching using the first image, the keyframe, and a relative pose based on a pose associated with the first image and a pose associated with the keyframe to generate a sparse disparity map representing disparities between the first image and the keyframe. The processor further is to determine a dense depth map from the disparity map using a bilateral solver algorithm, and process a viewfinder image generated from a second image of the feed with occlusion rendering based on the depth map to incorporate one or more virtual objects into the viewfinder image to generate an AR viewfinder image. Further, the processor is to provide the AR viewfinder image for display.Type: ApplicationFiled: October 4, 2019Publication date: January 7, 2021Inventors: Jullien VALENTIN, Onur G. GULERYUZ, Mira LEUNG, Maksym DZITSIUK, Jose PASCOAL, Mirko SCHMIDT, Christoph RHEMANN, Neal WADHWA, Eric TURNER, Sameh KHAMIS, Adarsh Prakash Murthy KOWDLE, Ambrus CSASZAR, João Manuel Castro AFONSO, Jonathan T. BARRON, Michael SCHOENBERG, Ivan DRYANOVSKI, Vivek VERMA, Vladimir TANKOVICH, Shahram IZADI, Sean Ryan Francesco FANELLO, Konstantine Nicholas John TSOTSOS
-
Patent number: 10237527Abstract: A computing device may obtain an input image. The input image may have a white point represented by chrominance values that define white color in the input image. Possibly based on colors of the input image, the computing device may generate a two-dimensional chrominance histogram of the input image. The computing device may convolve the two-dimensional chrominance histogram with a filter to create a two-dimensional heat map. Entries in the two-dimensional heat map may represent respective estimates of how close respective tints corresponding to the respective entries are to the white point of the input image. The computing device may select an entry in the two-dimensional heat map that represents a particular value that is within a threshold of a maximum value in the heat map, and based on the selected entry, tint the input image to form an output image.Type: GrantFiled: August 23, 2018Date of Patent: March 19, 2019Assignee: Google LLCInventors: Jonathan T. Barron, Yun-Ta Tsai
-
Publication number: 20180367774Abstract: A computing device may obtain an input image. The input image may have a white point represented by chrominance values that define white color in the input image. Possibly based on colors of the input image, the computing device may generate a two-dimensional chrominance histogram of the input image. The computing device may convolve the two-dimensional chrominance histogram with a filter to create a two-dimensional heat map. Entries in the two-dimensional heat map may represent respective estimates of how close respective tints corresponding to the respective entries are to the white point of the input image. The computing device may select an entry in the two-dimensional heat map that represents a particular value that is within a threshold of a maximum value in the heat map, and based on the selected entry, tint the input image to form an output image.Type: ApplicationFiled: August 23, 2018Publication date: December 20, 2018Inventors: Jonathan T. Barron, Yun-Ta Tsai
-
Patent number: 10091479Abstract: A computing device may obtain an input image. The input image may have a white point represented by chrominance values that define white color in the input image. Possibly based on colors of the input image, the computing device may generate a two-dimensional chrominance histogram of the input image. The computing device may convolve the two-dimensional chrominance histogram with a filter to create a two-dimensional heat map. Entries in the two-dimensional heat map may represent respective estimates of how close respective tints corresponding to the respective entries are to the white point of the input image. The computing device may select an entry in the two-dimensional heat map that represents a particular value that is within a threshold of a maximum value in the heat map, and based on the selected entry, tint the input image to form an output image.Type: GrantFiled: September 13, 2017Date of Patent: October 2, 2018Assignee: Google LLCInventors: Jonathan T. Barron, Yun-Ta Tsai
-
Publication number: 20180007337Abstract: A computing device may obtain an input image. The input image may have a white point represented by chrominance values that define white color in the input image. Possibly based on colors of the input image, the computing device may generate a two-dimensional chrominance histogram of the input image. The computing device may convolve the two-dimensional chrominance histogram with a filter to create a two-dimensional heat map. Entries in the two-dimensional heat map may represent respective estimates of how close respective tints corresponding to the respective entries are to the white point of the input image. The computing device may select an entry in the two-dimensional heat map that represents a particular value that is within a threshold of a maximum value in the heat map, and based on the selected entry, tint the input image to form an output image.Type: ApplicationFiled: September 13, 2017Publication date: January 4, 2018Inventors: Jonathan T. Barron, Yun-Ta Tsai
-
Patent number: 9860441Abstract: A computing device may obtain a first captured image of a scene and a second captured image of the scene. For a plurality of m×n pixel tiles of the first captured image, the computing device may determine respective distance matrixes. The distance matrixes may represent respective fit confidences between the m×n pixel tiles and pluralities of target p×q pixel tiles in the second captured image. The computing device may approximate the distance matrixes with respective bivariate surfaces. The computing device may upsample the bivariate surfaces to obtain respective offsets for pixels in the plurality of m×n pixel tiles. The respective offsets, when applied to pixels in the plurality of m×n pixel tiles, may cause parts of the first captured image to estimate locations in the second captured image.Type: GrantFiled: July 12, 2017Date of Patent: January 2, 2018Assignee: Google LLCInventors: Jonathan T. Barron, Dillon Sharlet
-
Patent number: 9794540Abstract: A computing device may obtain an input image. The input image may have a white point represented by chrominance values that define white color in the input image. Possibly based on colors of the input image, the computing device may generate a two-dimensional chrominance histogram of the input image. The computing device may convolve the two-dimensional chrominance histogram with a filter to create a two-dimensional heat map. Entries in the two-dimensional heat map may represent respective estimates of how close respective tints corresponding to the respective entries are to the white point of the input image. The computing device may select an entry in the two-dimensional heat map that represents a particular value that is within a threshold of a maximum value in the heat map, and based on the selected entry, tint the input image to form an output image.Type: GrantFiled: May 1, 2017Date of Patent: October 17, 2017Assignee: Google Inc.Inventors: Jonathan T. Barron, Yun-Ta Tsai
-
Publication number: 20170237961Abstract: A computing device may obtain an input image. The input image may have a white point represented by chrominance values that define white color in the input image. Possibly based on colors of the input image, the computing device may generate a two-dimensional chrominance histogram of the input image. The computing device may convolve the two-dimensional chrominance histogram with a filter to create a two-dimensional heat map. Entries in the two-dimensional heat map may represent respective estimates of how close respective tints corresponding to the respective entries are to the white point of the input image. The computing device may select an entry in the two-dimensional heat map that represents a particular value that is within a threshold of a maximum value in the heat map, and based on the selected entry, tint the input image to form an output image.Type: ApplicationFiled: May 1, 2017Publication date: August 17, 2017Inventors: Jonathan T. Barron, Yun-Ta Tsai
-
Patent number: 9736366Abstract: A computing device may obtain a first captured image of a scene and a second captured image of the scene. For a plurality of m×n pixel tiles of the first captured image, the computing device may determine respective distance matrixes. The distance matrixes may represent respective fit confidences between the m×n pixel tiles and pluralities of target p×q pixel tiles in the second captured image. The computing device may approximate the distance matrixes with respective bivariate quadratic surfaces. The computing device may upsample the bivariate quadratic surfaces to obtain respective offsets for pixels in the plurality of m×n pixel tiles. The respective offsets, when applied to pixels in the plurality of m×n pixel tiles, may cause parts of the first captured image to estimate locations in the second captured image.Type: GrantFiled: May 23, 2015Date of Patent: August 15, 2017Assignee: Google Inc.Inventors: Jonathan T. Barron, Dillon Sharlet
-
Patent number: 9672604Abstract: A computing device may obtain an input image. The input image may have a white point represented by chrominance values that define white color in the input image. Possibly based on colors of the input image, the computing device may generate a two-dimensional chrominance histogram of the input image. The computing device may convolve the two-dimensional chrominance histogram with a filter to create a two-dimensional heat map. Entries in the two-dimensional heat map may represent respective estimates of how close respective tints corresponding to the respective entries are to the white point of the input image. The computing device may select an entry in the two-dimensional heat map that represents a particular value that is within a threshold of a maximum value in the heat map, and based on the selected entry, tint the input image to form an output image.Type: GrantFiled: April 12, 2016Date of Patent: June 6, 2017Assignee: Google Inc.Inventor: Jonathan T. Barron
-
Publication number: 20160350900Abstract: A computing device may obtain an input image. The input image may have a white point represented by chrominance values that define white color in the input image. Possibly based on colors of the input image, the computing device may generate a two-dimensional chrominance histogram of the input image. The computing device may convolve the two-dimensional chrominance histogram with a filter to create a two-dimensional heat map. Entries in the two-dimensional heat map may represent respective estimates of how close respective tints corresponding to the respective entries are to the white point of the input image. The computing device may select an entry in the two-dimensional heat map that represents a particular value that is within a threshold of a maximum value in the heat map, and based on the selected entry, tint the input image to form an output image.Type: ApplicationFiled: April 12, 2016Publication date: December 1, 2016Inventor: Jonathan T. Barron
-
Patent number: 9342873Abstract: A computing device may determine a per-tile motion estimate between a first m×n pixel tile from a first captured image of a scene and a second m×n pixel tile from a second captured image of the scene. A per-tile confidence of the per-tile motion estimate may be obtained. The per-tile motion estimate and the per-tile confidence may be upsampled to obtain respective per-pixel motion estimates and associated per-pixel confidences for pixels of the first m×n pixel tile. The respective per-pixel motion estimates and associated per-pixel confidences may be iteratively filtered. The iterative filtering may involve multiplying the respective per-pixel motion estimates and associated per-pixel confidences by an affinity matrix. The iterative filtering may also smooth the respective per-pixel motion estimates and associated per-pixel confidences.Type: GrantFiled: May 23, 2015Date of Patent: May 17, 2016Assignee: Google Inc.Inventor: Jonathan T. Barron
-
Patent number: 9336582Abstract: A computing device may obtain an input image. The input image may have a white point represented by chrominance values that define white color in the input image. Possibly based on colors of the input image, the computing device may generate a two-dimensional chrominance histogram of the input image. The computing device may convolve the two-dimensional chrominance histogram with a filter to create a two-dimensional heat map. Entries in the two-dimensional heat map may represent respective estimates of how close respective tints corresponding to the respective entries are to the white point of the input image. The computing device may select an entry in the two-dimensional heat map that represents a particular value that is within a threshold of a maximum value in the heat map, and based on the selected entry, tint the input image to form an output image.Type: GrantFiled: April 17, 2015Date of Patent: May 10, 2016Assignee: Google Inc.Inventor: Jonathan T. Barron