Patents by Inventor Carsten Curt Eckard Rother
Carsten Curt Eckard Rother has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20220196840Abstract: Detecting material properties such reflectivity, true color and other properties of surfaces in a real world environment is described in various examples using a single hand-held device. For example, the detected material properties are calculated using a photometric stereo system which exploits known relationships between lighting conditions, surface normals, true color and image intensity. In examples, a user moves around in an environment capturing color images of surfaces in the scene from different orientations under known lighting conditions. In various examples, surfaces normals of patches of surfaces are calculated using the captured data to enable fine detail such as human hair, netting, textured surfaces to be modeled. In examples, the modeled data is used to render images depicting the scene with realism or to superimpose virtual graphics on the real world in a realistic manner.Type: ApplicationFiled: December 29, 2021Publication date: June 23, 2022Inventors: Otmar HILLIGES, Malte Hanno WEISS, Shahram IZADI, David KIM, Carsten Curt Eckard ROTHER
-
Patent number: 11215711Abstract: Detecting material properties such reflectivity, true color and other properties of surfaces in a real world environment is described in various examples using a single hand-held device. For example, the detected material properties are calculated using a photometric stereo system which exploits known relationships between lighting conditions, surface normals, true color and image intensity. In examples, a user moves around in an environment capturing color images of surfaces in the scene from different orientations under known lighting conditions. In various examples, surfaces normals of patches of surfaces are calculated using the captured data to enable fine detail such as human hair, netting, textured surfaces to be modeled. In examples, the modeled data is used to render images depicting the scene with realism or to superimpose virtual graphics on the real world in a realistic manner.Type: GrantFiled: December 18, 2017Date of Patent: January 4, 2022Assignee: Microsoft Technology Licensing, LLCInventors: Otmar Hilliges, Malte Hanno Weiss, Shahram Izadi, David Kim, Carsten Curt Eckard Rother
-
Publication number: 20180106905Abstract: Detecting material properties such reflectivity, true color and other properties of surfaces in a real world environment is described in various examples using a single hand-held device. For example, the detected material properties are calculated using a photometric stereo system which exploits known relationships between lighting conditions, surface normals, true color and image intensity. In examples, a user moves around in an environment capturing color images of surfaces in the scene from different orientations under known lighting conditions. In various examples, surfaces normals of patches of surfaces are calculated using the captured data to enable fine detail such as human hair, netting, textured surfaces to be modeled. In examples, the modeled data is used to render images depicting the scene with realism or to superimpose virtual graphics on the real world in a realistic manner.Type: ApplicationFiled: December 18, 2017Publication date: April 19, 2018Inventors: Otmar HILLIGES, Malte Hanno WEISS, Shahram IZADI, David KIM, Carsten Curt Eckard ROTHER
-
Patent number: 9857470Abstract: Detecting material properties such reflectivity, true color and other properties of surfaces in a real world environment is described in various examples using a single hand-held device. For example, the detected material properties are calculated using a photometric stereo system which exploits known relationships between lighting conditions, surface normals, true color and image intensity. In examples, a user moves around in an environment capturing color images of surfaces in the scene from different orientations under known lighting conditions. In various examples, surfaces normals of patches of surfaces are calculated using the captured data to enable fine detail such as human hair, netting, textured surfaces to be modeled. In examples, the modeled data is used to render images depicting the scene with realism or to superimpose virtual graphics on the real world in a realistic manner.Type: GrantFiled: December 28, 2012Date of Patent: January 2, 2018Assignee: Microsoft Technology Licensing, LLCInventors: Otmar Hilliges, Malte Hanno Weiss, Shahram Izadi, David Kim, Carsten Curt Eckard Rother
-
Patent number: 9430817Abstract: Blind image deblurring with a cascade architecture is described, for example, where photographs taken on a camera phone are deblurred in a process which revises blur estimates and estimates a blur function as a combined process. In various examples the estimates of the blur function are computed using first trained machine learning predictors arranged in a cascade architecture. In various examples a revised blur estimate is calculated at each level of the cascade using a latest deblurred version of a blurred image. In some examples the revised blur estimates are calculated using second trained machine learning predictors interleaved with the first trained machine learning predictors.Type: GrantFiled: November 12, 2013Date of Patent: August 30, 2016Assignee: Microsoft Technology Licensing, LLCInventors: Kevin Schelten, Reinhard Sebastian Bernhard Nowozin, Jeremy Jancsary, Carsten Curt Eckard Rother
-
Patent number: 9396523Abstract: Image restoration cascades are described, for example, where digital photographs containing noise are restored using a cascade formed from a plurality of layers of trained machine learning predictors connected in series. For example, noise may be from sensor noise, motion blur, dust, optical low pass filtering, chromatic aberration, compression and quantization artifacts, down sampling or other sources. For example, given a noisy image, each trained machine learning predictor produces an output image which is a restored version of the noisy input image; each trained machine learning predictor in a given internal layer of the cascade also takes input from the previous layer in the cascade. In various examples, a loss function expressing dissimilarity between input and output images of each trained machine learning predictor is directly minimized during training. In various examples, data partitioning is used to partition a training data set to facilitate generalization.Type: GrantFiled: July 24, 2013Date of Patent: July 19, 2016Assignee: Microsoft Technology Licensing, LLCInventors: Jeremy Jancsary, Reinhard Sebastian Bernhard Nowozin, Carsten Curt Eckard Rother
-
Patent number: 9070047Abstract: A tractable model solves certain labeling problems by providing potential functions having arbitrary dependencies upon an observed dataset (e.g., image data). The model uses decision trees corresponding to various factors to map dataset content to a set of parameters used to define the potential functions in the model. Some factors define relationships among multiple variable nodes. When making label predictions on a new dataset, the leaf nodes of the decision tree determine the effective weightings for such potential functions. In this manner, decision trees define non-parametric dependencies and can represent rich, arbitrary functional relationships if sufficient training data is available. Decision trees training is scalable, both in the training set size and by parallelization. Maximum pseudolikelihood learning can provide for joint training of aspects of the model, including feature test selection and ordering, factor weights, and the scope of the interacting variable nodes used in the graph.Type: GrantFiled: December 27, 2011Date of Patent: June 30, 2015Assignee: Microsoft Technology Licensing, LLCInventors: Reinhard Sebastian Bernhard Nowozin, Carsten Curt Eckard Rother, Bangpeng Yao, Toby Leonard Sharp, Pushmeet Kohli
-
Publication number: 20150131898Abstract: Blind image deblurring with a cascade architecture is described, for example, where photographs taken on a camera phone are deblurred in a process which revises blur estimates and estimates a blur function as a combined process. In various examples the estimates of the blur function are computed using first trained machine learning predictors arranged in a cascade architecture. In various examples a revised blur estimate is calculated at each level of the cascade using a latest deblurred version of a blurred image. In some examples the revised blur estimates are calculated using second trained machine learning predictors interleaved with the first trained machine learning predictors.Type: ApplicationFiled: November 12, 2013Publication date: May 14, 2015Applicant: Microsoft CorporationInventors: Kevin Schelten, Reinhard Sebastian Bernhard Nowozin, Jeremy Jancsary, Carsten Curt Eckard Rother
-
Patent number: 8971612Abstract: Learning image processing tasks from scene reconstructions is described where the tasks may include but are not limited to: image de-noising, image in-painting, optical flow detection, interest point detection. In various embodiments training data is generated from a 2 or higher dimensional reconstruction of a scene and from empirical images of the same scene. In an example a machine learning system learns at least one parameter of a function for performing the image processing task by using the training data. In an example, the machine learning system comprises a random decision forest. In an example, the scene reconstruction is obtained by moving an image capture apparatus in an environment where the image capture apparatus has an associated dense reconstruction and camera tracking system.Type: GrantFiled: December 15, 2011Date of Patent: March 3, 2015Assignee: Microsoft CorporationInventors: Jamie Daniel Joseph Shotton, Pushmeet Kohli, Stefan Johannes Josef Holzer, Shahram Izadi, Carsten Curt Eckard Rother, Sebastian Nowozin, David Kim, David Molyneaux, Otmar Hilliges
-
Publication number: 20150030237Abstract: Image restoration cascades are described, for example, where digital photographs containing noise are restored using a cascade formed from a plurality of layers of trained machine learning predictors connected in series. For example, noise may be from sensor noise, motion blur, dust, optical low pass filtering, chromatic aberration, compression and quantization artifacts, down sampling or other sources. For example, given a noisy image, each trained machine learning predictor produces an output image which is a restored version of the noisy input image; each trained machine learning predictor in a given internal layer of the cascade also takes input from the previous layer in the cascade. In various examples, a loss function expressing dissimilarity between input and output images of each trained machine learning predictor is directly minimized during training. In various examples, data partitioning is used to partition a training data set to facilitate generalization.Type: ApplicationFiled: July 24, 2013Publication date: January 29, 2015Applicant: Microsoft CorporationInventors: Jeremy Jancsary, Reinhard Sebastian Bernhard Nowozin, Carsten Curt Eckard Rother
-
Publication number: 20150016717Abstract: A computing device is described herein that is configured to select a pixel pair including a foreground pixel of an image and a background pixel of the image from a global set of pixels based at least on spatial distances from an unknown pixel and color distances from the unknown pixel. The computing device is further configured to determine an opacity measure for the unknown pixel based at least on the selected pixel pair.Type: ApplicationFiled: September 29, 2014Publication date: January 15, 2015Inventors: Kaiming He, Jian Sun, Carsten Curt Eckard Rother, Xiao-ou Tang
-
Patent number: 8891884Abstract: A new tractable model solves labeling problems using regression tree fields, which represent non-parametric Gaussian conditional random fields. Regression tree fields are parameterized by non-parametric regression trees, allowing universal specification of interactions between image observations and variables. The new model uses regression trees corresponding to various factors to map dataset content (e.g., image content) to a set of parameters used to define the potential functions in the model. Some factors define relationships among multiple variable nodes. Further, the training of regression trees is scalable, both in the training set size and in the fact that the training can be parallelized. In one implementation, maximum pseudolikelihood learning provides for joint training of various aspects of the model, including feature test selection and ordering (i.e., the structure of the regression trees), parameters of each factor in the graph, and the scope of the interacting variable nodes used in the graph.Type: GrantFiled: December 27, 2011Date of Patent: November 18, 2014Assignee: Microsoft CorporationInventors: Reinhard Sebastian Bernhard Nowozin, Carsten Curt Eckard Rother, Jeremy Martin Jancsary
-
Publication number: 20140307950Abstract: Image deblurring is described, for example, to remove blur from digital photographs captured at a handheld camera phone and which are blurred due to camera shake. In various embodiments an estimate of blur in an image is available from a blur estimator and a trained machine learning system is available to compute parameter values of a blur function from the blurred image. In various examples the blur function is obtained from a probability distribution relating a sharp image, a blurred image and a fixed blur estimate. For example, the machine learning system is a regression tree field trained using pairs of empirical sharp images and blurred images calculated from the empirical images using artificially generated blur kernels.Type: ApplicationFiled: April 13, 2013Publication date: October 16, 2014Applicant: Microsoft CorporationInventors: Jeremy Jancsary, Uwe Johann Schmidt, Reinhard Sebastian Bernhard Nowozin, Carsten Curt Eckard Rother
-
Patent number: 8855411Abstract: A computing device is described herein that is configured to select a pixel pair including a foreground pixel of an image and a background pixel of the image from a global set of pixels based at least on spatial distances from an unknown pixel and color distances from the unknown pixel. The computing device is further configured to determine an opacity measure for the unknown pixel based at least on the selected pixel pair.Type: GrantFiled: May 16, 2011Date of Patent: October 7, 2014Assignee: Microsoft CorporationInventors: Kaiming He, Jian Sun, Carsten Curt Eckard Rother, Xiao-Ou Tang
-
Publication number: 20140241612Abstract: Real-time stereo matching is described, for example, to find depths of objects in an environment from an image capture device capturing a stream of stereo images of the objects. For example, the depths may be used to control augmented reality, robotics, natural user interface technology, gaming and other applications. Streams of stereo images, or single stereo images, obtained with or without patterns of illumination projected onto the environment are processed using a parallel-processing unit to obtain depth maps. In various embodiments a parallel-processing unit propagates values related to depth in rows or columns of a disparity map in parallel. In examples, the values may be propagated according to a measure of similarity between two images of a stereo pair; propagation may be temporal between disparity maps of frames of a stream of stereo images and may be spatial within a left or right disparity map.Type: ApplicationFiled: February 23, 2013Publication date: August 28, 2014Applicant: MICROSOFT CORPORATIONInventors: Christoph Rhemann, Carsten Curt Eckard Rother, Christopher Zach, Shahram Izadi, Adam Garnet Kirk, Oliver Whyte, Michael Bleyer
-
Patent number: 8787658Abstract: Methods of image segmentation using reduced foreground training data are described. In an embodiment, the foreground and background training data for use in segmentation of an image is determined by optimization of a modified energy function. The modified energy function is the energy function used in image segmentation with an additional term comprising a scalar value. The optimization is performed for different values of the scalar to produce multiple initial segmentations and one of these segmentations is selected based on pre-defined criteria. The training data is then used in segmenting the image. In other embodiments further methods are described: one places an ellipse inside the user-defined bounding box to define the background training data and another uses a comparison of properties of neighboring image elements, where one is outside the user-defined bounding box, to reduce the foreground training data.Type: GrantFiled: March 19, 2013Date of Patent: July 22, 2014Assignee: Microsoft CorporationInventors: Carsten Curt Eckard Rother, Toby Sharp, Andrew Blake, Vladimir Kolmogorov
-
Publication number: 20140184749Abstract: Detecting material properties such reflectivity, true color and other properties of surfaces in a real world environment is described in various examples using a single hand-held device. For example, the detected material properties are calculated using a photometric stereo system which exploits known relationships between lighting conditions, surface normals, true color and image intensity. In examples, a user moves around in an environment capturing color images of surfaces in the scene from different orientations under known lighting conditions. In various examples, surfaces normals of patches of surfaces are calculated using the captured data to enable fine detail such as human hair, netting, textured surfaces to be modeled. In examples, the modeled data is used to render images depicting the scene with realism or to superimpose virtual graphics on the real world in a realistic manner.Type: ApplicationFiled: December 28, 2012Publication date: July 3, 2014Applicant: MICROSOFT CORPORATIONInventors: Otmar Hilliges, Malte Hanno Weiss, Shahram Izadi, David Kim, Carsten Curt Eckard Rother
-
Patent number: 8655069Abstract: Methods of updating image segmentation following user input are described. In an embodiment, the properties used in computing the different portions of the image are updated as a result of one or more user inputs. Image elements which have been identified by a user input are given more weight when updating the properties than other image elements which have already been assigned to a particular portion of the image. In another embodiment, an updated segmentation is post-processed such that only regions which are connected to an appropriate user input are updated.Type: GrantFiled: March 5, 2010Date of Patent: February 18, 2014Assignee: Microsoft CorporationInventors: Carsten Curt Eckard Rother, Toby Sharp, Andrew Blake, Vladimir Kolmogorov
-
Patent number: 8644609Abstract: A method of up-sampling binary images for segmentation is described. In an embodiment, digital images are down-sampled before segmentation. The resulting initial binary segmentation, which has a lower resolution than the original image, is then up-sampled and smoothed to generate an interim non-binary solution which has a higher resolution than the initial binary segmentation. The final binary segmentation for the image is then computed from the interim non-binary solution based on a threshold. This method does not use the original image data in inferring the final binary segmentation solution from the initial binary segmentation. In an embodiment, the method may be applied to all images and in another embodiment, the method may be used for images which comprise a large number of pixels in total or in single dimension and smaller images may not be down-sampled before segmentation.Type: GrantFiled: March 19, 2013Date of Patent: February 4, 2014Assignee: Microsoft CorporationInventors: Carsten Curt Eckard Rother, Toby Sharp
-
Patent number: 8605992Abstract: Image completion using scene geometry is described, for example, to remove marks from digital photographs or complete regions which are blank due to editing. In an embodiment an image depicting, from a viewpoint, a scene of textured objects has regions to be completed. In an example, geometry of the scene is estimated from a depth map and the geometry used to warp the image so that at least some surfaces depicted in the image are fronto-parallel to the viewpoint. An image completion process is guided using distortion applied during the warping. For example, patches used to fill the regions are selected on the basis of distortion introduced by the warping. In examples where the scene comprises regions having only planar surfaces the warping process comprises rotating the image. Where the scene comprises non-planar surfaces, geodesic distances between image elements may be scaled to flatten the non-planar surfaces.Type: GrantFiled: April 8, 2011Date of Patent: December 10, 2013Assignee: Microsoft CorporationInventors: Pushmeet Kohli, Toby Sharp, Carsten Curt Eckard Rother