Patents by Inventor Carsten Rother

Carsten Rother has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 9135732
    Abstract: Systems and methods for editing digital images using information about objects in those images are described. For example, the information about objects comprises depth ordering information and/or information about the class each object is a member of. Examples of classes include sky, building, aeroplane, grass and person. This object-level information is used to provide new and/or improved editing functions such as cut and paste, filling-in image regions using tiles or patchworks, digital tapestry, alpha matte generation, super resolution, auto cropping, auto color balance, object selection, depth of field manipulation, and object replacement. In addition improvements to user interfaces for image editing systems are described which use object-level information.
    Type: Grant
    Filed: February 8, 2007
    Date of Patent: September 15, 2015
    Assignee: Microsoft Corporation
    Inventors: John Winn, Carsten Rother
  • Patent number: 8824801
    Abstract: A method and apparatus for processing video is disclosed. In an embodiment, image features of an object within a frame of video footage are identified and the movement of each of these features is tracked throughout the video footage to determine its trajectory (track). The tracks are analyzed, the maximum separation of the tracks is determined and used to determine a texture map, which is in turn interpolated to provide an unwrap mosaic for the object. The process may be iterated to provide an improved mosaic. Effects or artwork can be overlaid on this mosaic and the edited mosaic can be warped via the mapping, and combined with layers of the original footage. The effect or artwork may move with the object's surface.
    Type: Grant
    Filed: May 16, 2008
    Date of Patent: September 2, 2014
    Assignee: Microsoft Corporation
    Inventors: Andrew Fitzgibbon, Alexander Rav-Acha, Pushmeet Kohli, Carsten Rother
  • Publication number: 20140192158
    Abstract: The description relates to stereo image matching to determine depth of a scene as captured by images. More specifically, the described implementations can involve a two-stage approach where the first stage can compute depth at highly accurate but sparse feature locations. The second stage can compute a dense depth map using the first stage as initialization. This improves accuracy and robustness of the dense depth map.
    Type: Application
    Filed: January 4, 2013
    Publication date: July 10, 2014
    Applicant: Microsoft Corporation
    Inventors: Oliver Whyte, Adam G. Kirk, Shahram Izadi, Carsten Rother, Michael Bleyer, Christoph Rhemann
  • Patent number: 8587670
    Abstract: An image capture device is described which is operable in any one of a number of capture modes. The device comprises a camera, a memory and a processor. The memory stores a plurality of sets of capture triggers, with each set of capture triggers being associated with one of the plurality of capture modes. The processor selects one of the plurality of capture modes, such that the device is operable in the selected capture mode. In the selected capture mode, an image is captured automatically when a capture trigger within the associated set of capture triggers is satisfied.
    Type: Grant
    Filed: November 16, 2006
    Date of Patent: November 19, 2013
    Assignee: Microsoft Corporation
    Inventors: Kenneth Wood, Stephen Hodges, Lyndsay Williams, James Srinivasan, Carsten Rother, Antonio Criminisi, John Chiloyan
  • Patent number: 8558917
    Abstract: A method of transferring images from a first device to a second device and computer program code for performing this method is described. A connection characteristic for a connection between the first & second devices is determined and at least one image is selected from a plurality of images on the first device for transfer dependent upon both the connection characteristic and image selection criteria. The selected image(s) are then transferred over the connection from the first device to the second device.
    Type: Grant
    Filed: November 24, 2006
    Date of Patent: October 15, 2013
    Assignee: Microsoft Corporation
    Inventors: Kenneth Wood, Stephen Hodges, Lyndsay Williams, Mitch Goldberg, Carsten Rother, Antonio Criminisi, James Srinivasan
  • Patent number: 8503783
    Abstract: A method of labeling pixels in an image is described where the pixel label is selected from a set of three or more labels. The pixel labeling problem is reduced to a sequence of binary optimizations by representing the label value for each pixel as a binary word and then optimizing the value of each bit within the word, starting with the most significant bit. Data which has been learned from one or more training images is used in the optimization to provide information about the less significant bits within the word.
    Type: Grant
    Filed: March 17, 2011
    Date of Patent: August 6, 2013
    Assignee: Microsoft Corporation
    Inventors: Carsten Rother, Victor Lempitsky, Andrew Blake
  • Patent number: 8498481
    Abstract: Image segmentation using star-convexity constraints is described. In an example, user input specifies positions of one or more star centers in a foreground to be segmented from a background of an image. In embodiments, an energy function is used to express the problem of segmenting the image and that energy function incorporates a star-convexity constraint which limits the number of possible solutions. For example, the star-convexity constraint may be that, for any point p inside the foreground, all points on a shortest path (which may be geodesic or Euclidean) between the nearest star center and p also lie inside the foreground. In some examples continuous star centers such as lines are used. In embodiments a user may iteratively edit the star centers by adding brush strokes to the image in order to progressively change the star-convexity constraints and obtain an accurate segmentation.
    Type: Grant
    Filed: May 7, 2010
    Date of Patent: July 30, 2013
    Assignee: Microsoft Corporation
    Inventors: Andrew Blake, Varun Gulshan, Carsten Rother, Antonio Criminisi
  • Patent number: 8436852
    Abstract: Image editing which is consistent with geometry of a scene depicted in the image is described. In an embodiment a graphical user interface (GUI) is provided to enable a user to simply and quickly specify four corners of a rectangular frame drawn onto a source image using the GUI. In embodiments, the four corners are used to compute parameters of a virtual camera assumed to capture the image of the drawn frame. Embodiments of an image processing system are described which use the virtual camera parameters to control editing of the source image in ways consistent with the 3D geometry of the scene depicted in that image. In some embodiments out of bounds images are formed and/or realistic-looking shadows are synthesized. In examples, users are able to edit images and the virtual camera parameters are dynamically recomputed and used to update the edited image.
    Type: Grant
    Filed: February 9, 2009
    Date of Patent: May 7, 2013
    Assignee: Microsoft Corporation
    Inventors: Antonio Criminisi, Carsten Rother, Gavin Smyth, Amit Shesh
  • Patent number: 8249349
    Abstract: An image processing system is described which automatically labels image elements of a digital image. In an embodiment an energy function describing the quality of possible labelings of an image is globally optimized to find an output labeled image. In the embodiment, the energy function comprises terms that depend on at least one non-local parameter. For example, the non-local parameter describes characteristics of image elements having the same label. In an embodiment the global optimization is achieved in a practical, efficient manner by using a tree structure to represent candidate values of the non-local parameter and by using a branch and bound process. In some embodiments, the branch and bound process comprises evaluating a lower bound of the energy function by using a min-cut process. For example, the min-cut process enables the lower bound to be evaluated efficiently using a graphical data structure to represent the lower bound.
    Type: Grant
    Filed: November 25, 2008
    Date of Patent: August 21, 2012
    Assignee: Microsoft Corporation
    Inventors: Andrew Blake, Carsten Rother, Victor Lempitsky
  • Patent number: 8249397
    Abstract: A method of displaying digital images is described in which a display length indicator is received and digital images are accessed. A set of digital images are selected from the accessed digital images in accordance with the display length indicator and displayed in a predetermined order. The method may be performed by a computer program, which may be embodied on a computer readable medium.
    Type: Grant
    Filed: November 16, 2006
    Date of Patent: August 21, 2012
    Assignee: Microsoft Corporation
    Inventors: Kenneth Wood, Stephen Hodges, Lyndsay Williams, James Srinivasan, Carsten Rother, Antonio Criminisi
  • Patent number: 8213726
    Abstract: Multi-scale processing may be used to reduce the memory and computational requirements of optimization algorithms for image labeling, for example, for object segmentation, 3D reconstruction, stereo correspondence, optical flow and other applications. For example, in order to label a large image (or 3D volume) a multi-scale process first solves the problem at a low resolution, obtaining a coarse labeling of an original high resolution problem. This labeling is refined by solving another optimization on a subset of the image elements. In examples, an energy function for a coarse level version of an input image is formed directly from an energy function of the input image. In examples, the subset of image elements may be selected using a measure of confidence in the labeling.
    Type: Grant
    Filed: June 19, 2009
    Date of Patent: July 3, 2012
    Assignee: Microsoft Corporation
    Inventors: Pushmeet Kohli, Carsten Rother, Victor Lempitsky
  • Publication number: 20110274352
    Abstract: Image segmentation using star-convexity constraints is described. In an example, user input specifies positions of one or more star centers in a foreground to be segmented from a background of an image. In embodiments, an energy function is used to express the problem of segmenting the image and that energy function incorporates a star-convexity constraint which limits the number of possible solutions. For example, the star-convexity constraint may be that, for any point p inside the foreground, all points on a shortest path (which may be geodesic or Euclidean) between the nearest star center and p also lie inside the foreground. In some examples continuous star centers such as lines are used. In embodiments a user may iteratively edit the star centers by adding brush strokes to the image in order to progressively change the star-convexity constraints and obtain an accurate segmentation.
    Type: Application
    Filed: May 7, 2010
    Publication date: November 10, 2011
    Applicant: Microsoft Corporation
    Inventors: Andrew Blake, Varun Gulshan, Carsten Rother, Antonio Criminisi
  • Patent number: 8041114
    Abstract: Computer vision applications often require each pixel within an image to be assigned one of a set of labels. A method of improving the labels assigned to pixels is described which uses the quadratic pseudoboolean optimization (QPBO) algorithm. Starting with a partially labeled solution, an unlabeled pixel is assigned a value from a fully labeled reference solution and the energy of the partially labeled solution plus this additional pixel is calculated. The calculated energy is then used to generate a revised partially labeled solution using QPBO.
    Type: Grant
    Filed: June 15, 2007
    Date of Patent: October 18, 2011
    Assignee: Microsoft Corporation
    Inventors: Carsten Rother, Victor Lempitsky, Vladimir Kolmogorov
  • Patent number: 8019177
    Abstract: Previously, Poisson blending has been used for image blending including cloning an object onto a target background and blending pairs of source images together. Such Poisson blending works well in many situations. However, whilst this method is always workable, we have found that discolorations sometimes occur. We realized that these discolorations occur when the gradient of the source image is preserved too insistently, at the expense of preserving object and background color. In some situations object outlines become smeared or blurred. We develop a color preservation term and a fragility measure to address these problems. This gives a user additional control to obtain smooth compositions and reduce discoloration artifacts.
    Type: Grant
    Filed: July 28, 2006
    Date of Patent: September 13, 2011
    Assignee: Microsoft Corporation
    Inventors: Carsten Rother, Vladimir Kolmogorov, Andrew Blake
  • Publication number: 20110164819
    Abstract: A method of labeling pixels in an image is described where the pixel label is selected from a set of three or more labels. The pixel labeling problem is reduced to a sequence of binary optimizations by representing the label value for each pixel as a binary word and then optimizing the value of each bit within the word, starting with the most significant bit. Data which has been learned from one or more training images is used in the optimization to provide information about the less significant bits within the word.
    Type: Application
    Filed: March 17, 2011
    Publication date: July 7, 2011
    Applicant: Microsoft Corporation
    Inventors: Carsten Rother, Victor Lempitsky, Andrew Blake
  • Patent number: 7925089
    Abstract: A method of labeling pixels in an image is described where the pixel label is selected from a set of three or more labels. The pixel labeling problem is reduced to a sequence of binary optimizations by representing the label value for each pixel as a binary word and then optimizing the value of each bit within the word, starting with the most significant bit. Data which has been learned from one or more training images is used in the optimization to provide information about the less significant bits within the word.
    Type: Grant
    Filed: September 18, 2007
    Date of Patent: April 12, 2011
    Assignee: Microsoft Corporation
    Inventors: Carsten Rother, Victor Lempitsky, Andrew Blake
  • Publication number: 20110064303
    Abstract: Given an image of structured and/or unstructured objects, semantically meaningful areas are automatically partitioned from the image, each area labeled with a specific object class. Shape filters are used to enable capturing of some or all of the shape, texture, and/or appearance context information. A shape filter comprises one or more regions of arbitrary shape, size, and/or position within a bounding area of an image, paired with a specified texton. A texton comprises information describing the texture of a patch of surface of an object. In a training process a sub-set of possible shape filters is selected and incorporated into a conditional random field model of object classes. The conditional random field model is then used for object detection and recognition.
    Type: Application
    Filed: November 11, 2010
    Publication date: March 17, 2011
    Applicant: Microsoft Corporation
    Inventors: John Winn, Carsten Rother, Antonio Criminisi, Jamie Shotton
  • Publication number: 20100322525
    Abstract: Multi-scale processing may be used to reduce the memory and computational requirements of optimization algorithms for image labeling, for example, for object segmentation, 3D reconstruction, stereo correspondence, optical flow and other applications. For example, in order to label a large image (or 3D volume) a multi-scale process first solves the problem at a low resolution, obtaining a coarse labeling of an original high resolution problem. This labeling is refined by solving another optimization on a subset of the image elements. In examples, an energy function for a coarse level version of an input image is formed directly from an energy function of the input image. In examples, the subset of image elements may be selected using a measure of confidence in the labeling.
    Type: Application
    Filed: June 19, 2009
    Publication date: December 23, 2010
    Applicant: Microsoft Corporation
    Inventors: Pushmeet Kohli, Carsten Rother, Victor Lempitsky
  • Patent number: 7840059
    Abstract: Given an image of structured and/or unstructured objects we automatically partition it into semantically meaningful areas each labeled with a specific object class. We use a novel type of feature which we refer to as a shape filter. Shape filters enable us to capture some or all of shape, texture and appearance context information. A shape filter comprises one or more regions of arbitrary shape, size and position within a bounding area of an image, paired with a specified texton. A texton comprises information describing the texture of a patch of surface of an object. In a training process we select a sub-set of possible shape filters and incorporate those into a conditional random field model of object classes. That model is then used for object detection and recognition.
    Type: Grant
    Filed: September 21, 2006
    Date of Patent: November 23, 2010
    Assignee: Microsoft Corporation
    Inventors: John Winn, Carsten Rother, Antonio Criminisi, Jamie Shotton
  • Publication number: 20100201681
    Abstract: Image editing which is consistent with geometry of a scene depicted in the image is described. In an embodiment a graphical user interface (GUI) is provided to enable a user to simply and quickly specify four corners of a rectangular frame drawn onto a source image using the GUI. In embodiments, the four corners are used to compute parameters of a virtual camera assumed to capture the image of the drawn frame. Embodiments of an image processing system are described which use the virtual camera parameters to control editing of the source image in ways consistent with the 3D geometry of the scene depicted in that image. In some embodiments out of bounds images are formed and/or realistic-looking shadows are synthesized. In examples, users are able to edit images and the virtual camera parameters are dynamically recomputed and used to update the edited image.
    Type: Application
    Filed: February 9, 2009
    Publication date: August 12, 2010
    Applicant: Microsoft Corporation
    Inventors: Antonio Criminisi, Carsten Rother, Gavin Smyth, Amit Shesh