Patents by Inventor Rolf Toft
Rolf Toft has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 9342871Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.Type: GrantFiled: September 30, 2014Date of Patent: May 17, 2016Assignee: Apple Inc.Inventors: Claus Molgaard, Marius Tico, Rolf Toft, Paul M. Hubel
-
Patent number: 9344636Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.Type: GrantFiled: May 30, 2014Date of Patent: May 17, 2016Assignee: Apple Inc.Inventors: Marius Tico, Rolf Toft, Paul M. Hubel
-
Publication number: 20160094801Abstract: Traditionally, time-lapse videos are constructed from images captured at time intervals called “temporal points of interests” or “temporal POIs.” Disclosed herein are systems and methods of constructing improved, motion-stabilized time-lapse videos using temporal points of interest and image similarity comparisons. According to some embodiments, a “burst” of images may be captured, centered around the aforementioned temporal points of interest. Then, each burst sequence of images may be analyzed, e.g., by performing an image similarity comparison between each image in the burst sequence and the image selected at the previous temporal point of interest.Type: ApplicationFiled: February 3, 2015Publication date: March 31, 2016Inventors: Sebastien X. Beysserie, Jason Klivington, Rolf Toft, Frank Doepke
-
Patent number: 9300871Abstract: An apparatus, method, and computer-readable medium for motion sensor-based video stabilization. A motion sensor may capture motion data of a video sequence. A controller may compute instantaneous motion of the camera for a current frame of the video sequence and accumulated motion of the camera corresponding to motion of a plurality of frames of the video sequence. The controller may compare the instantaneous motion to a first threshold value, compare the accumulated motion to a second threshold value, and set a video stabilization strength parameter for the current frame based on the results of the comparison. A video stabilization unit may perform video stabilization on the current frame according to the frame's strength parameter.Type: GrantFiled: June 8, 2012Date of Patent: March 29, 2016Assignee: Apple Inc.Inventors: Jianping Zhou, Sebastien X. Beysserie, George E. Williams, Rolf Toft
-
Patent number: 9262684Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. Novel approaches for fusing non-reference images with a pre-selected reference frame in a set of commonly captured images are disclosed. The fusing approach may use a soft transition by using a weighted average for ghost/non-ghost pixels to avoid sudden transition between neighborhood and almost similar pixels. Additionally, the ghost/non-ghost decision can be made based on a set of neighboring pixels rather than independently for each pixel. An alternative approach may involve performing a multi-resolution decomposition of all the captured images, using temporal fusion, spatio-temporal fusion, or combinations thereof, at each level and combining the different levels to generate an output image.Type: GrantFiled: June 6, 2013Date of Patent: February 16, 2016Assignee: Apple Inc.Inventors: Marius Tico, Jianping Zhou, Anita Nariani Schulze, Rolf Toft, Paul Hubel, Wei Sun
-
Publication number: 20150350509Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.Type: ApplicationFiled: May 30, 2014Publication date: December 3, 2015Applicant: Apple Inc.Inventors: Marius Tico, Rolf Toft, Paul M. Hubel
-
Publication number: 20150348242Abstract: Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a “spatial difference map.” Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.Type: ApplicationFiled: September 30, 2014Publication date: December 3, 2015Inventors: Claus Molgaard, Marius Tico, Rolf Toft, Paul M. Hubel
-
Publication number: 20150350550Abstract: In some embodiments, a method for compensating for lens motion includes estimating a starting position of a lens assembly associated with captured pixel data. The captured pixel data is captured from an image sensor. In some embodiments, the method further includes calculating from the starting position and position data received from the one or more position sensors lens movement associated with the captured pixel data. The lens movement is mapped into pixel movement associated with the captured pixel data. A transform matrix is adjusted to reflect at least the pixel movement. A limit factor associated with the position data is calculated. The captured pixel data is recalculated using the transform matrix and the limit factor.Type: ApplicationFiled: November 18, 2014Publication date: December 3, 2015Applicant: Apple Inc.Inventors: Damien J. Thivent, Rolf Toft, Jianping Zhou
-
Publication number: 20140363087Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. Novel approaches for fusing non-reference images with a pre-selected reference frame in a set of commonly captured images are disclosed. The fusing approach may use a soft transition by using a weighted average for ghost/non-ghost pixels to avoid sudden transition between neighborhood and almost similar pixels. Additionally, the ghost/non-ghost decision can be made based on a set of neighboring pixels rather than independently for each pixel. An alternative approach may involve performing a multi-resolution decomposition of all the captured images, using temporal fusion, spatio-temporal fusion, or combinations thereof, at each level and combining the different levels to generate an output image.Type: ApplicationFiled: June 6, 2013Publication date: December 11, 2014Inventors: Marius Tico, Jianping Zhou, Anita Nariani Schulze, Rolf Toft, Paul Hubel, Wei Sun
-
Publication number: 20140362256Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel combination of image quality and commonality metrics are used to identify a reference frame from a set of commonly captured images which, when the set's other images are combined with it, results in a quality stabilized image. The disclosed image quality and commonality metrics may also be used to optimize the use of a limited amount of image buffer memory during image capture sequences that return more images that the memory may accommodate at one time. Image quality and commonality metrics may also be used to effect the combination of multiple relatively long-exposure images which, when combined with a one or more final (relatively) short-exposure images, yields images exhibiting motion-induced blurring in interesting and visually pleasing ways.Type: ApplicationFiled: June 6, 2013Publication date: December 11, 2014Inventors: Anita Nariani Schulze, Rolf Toft, Paul M. Hubel, Marius Tico, Jianping Zhou, Ralph Brunner, Claus Molgaard
-
Publication number: 20140363096Abstract: Systems, methods, and computer readable media to improve image stabilization operations are described. A novel approach to pixel-based registration of non-reference images to a reference frame in a set of commonly captured images is disclosed which makes use of pyramid decomposition to more efficiently detect corners. The disclosed pixel-based registration operation may also be combined with motion sensor data-based registration approaches to register non-reference images with respect to the reference frame. When the registered non-reference images are combined with the pre-selected reference image, the resulting image is a quality stabilized image.Type: ApplicationFiled: June 6, 2013Publication date: December 11, 2014Inventors: Jianping Zhou, Paul Hubel, Marius Tico, Anita Nariani Schulze, Rolf Toft
-
Publication number: 20140354842Abstract: Systems, methods, and computer readable media to improve image stabilization operations and other image processing operations are described. A novel combination of interleaved image capture and image processing operations, e.g., image registration operations, may be employed on a bracketed capture of still images. Such techniques may result in improved camera performance and processing efficiency, as well as decreased shot-to-shot time intervals. In another embodiment, an image fusion portion of an image post-processing pipeline may also be performed in an interleaved fashion, such that each image in the sequence of obtained bracketed images may be incrementally added to an output composite image after it has been aligned with the preceding image or images from the sequence.Type: ApplicationFiled: May 28, 2013Publication date: December 4, 2014Inventors: Russell Pflughaupt, Rolf Toft, Paul Hubel, Anita Nariani Schulze
-
Publication number: 20140267802Abstract: Techniques are disclosed for selectively capturing, retaining, and combining multiple sub-exposure images or brackets to yield a final image having diminished motion-induced blur and good noise characteristics. More specifically, after or during the capture of N brackets, the M best may be identified for combining into a single output image, (N>M). As used here, the term “best” means those brackets that exhibit the least amount of relative motion with respect to one another—with one caveat: integer pixel shifts may be preferred over sub-pixel shifts.Type: ApplicationFiled: June 4, 2013Publication date: September 18, 2014Inventors: Todd Sachs, Marius Tico, Rolf Toft, Paul Hubel, George Williams
-
Publication number: 20140189729Abstract: A method of processing a boot sequence for a processor forming a part of a module apparatus involves interrogating a host television device to determine if the host television device requires use of an application execution engine residing on the module apparatus; receiving a reply to the interrogation; and responsive to the reply, establishing a mode of operation wherein the application execution engine is used to execute application code for the host television device. This abstract is not to be considered limiting, since other embodiments may deviate from the features described in this abstract.Type: ApplicationFiled: March 10, 2014Publication date: July 3, 2014Applicants: SONY ELECTRONICS INC., SONY CORPORATIONInventors: George Williams, Xudong Yu, Toshiro Ozawa, Rolf Toft, Leo M. Pedlow, JR., Brant L. Candelore, David Boyden, Robert Blanchard, Christopher Jensen Read
-
Patent number: 8718198Abstract: A method of processing a boot sequence for a processor forming a part of a module apparatus involves interrogating a host television device to determine if the host television device requires use of an application execution engine residing on the module apparatus; receiving a reply to the interrogation; and responsive to the reply, establishing a mode of operation wherein the application execution engine is used to execute application code for the host television device. This abstract is not to be considered limiting, since other embodiments may deviate from the features described in this abstract.Type: GrantFiled: December 1, 2009Date of Patent: May 6, 2014Assignees: Sony Corporation, Sony Electronics Inc.Inventors: George Williams, Xudong Yu, Toshiro Ozawa, Rolf Toft, Leo M. Pedlow, Jr., Brant L. Candelore, David Boyden, Robert Blanchard, Christopher Jensen Read
-
Publication number: 20130329062Abstract: An apparatus, method, and computer-readable medium for motion sensor-based video stabilization. A motion sensor may capture motion data of a video sequence. A controller may compute instantaneous motion of the camera for a current frame of the video sequence and accumulated motion of the camera corresponding to motion of a plurality of frames of the video sequence. The controller may compare the instantaneous motion to a first threshold value, compare the accumulated motion to a second threshold value, and set a video stabilization strength parameter for the current frame based on the results of the comparison. A video stabilization unit may perform video stabilization on the current frame according to the frame's strength parameter.Type: ApplicationFiled: June 8, 2012Publication date: December 12, 2013Applicant: APPLE INC.Inventors: Jianping Zhou, Sebastien X. Beysserie, George E. Williams, Rolf Toft
-
Patent number: 8544047Abstract: A TV-set is equipped with HDMI and USB connections that allow it to display and run audio-video content from a variety of conventional consumer devices. The TV-set is further equipped to provide a secure HDMI-USB interface that will allow the transfer of licensed high definition content and Internet subscriber services. Such secure HDMI-USB interface also enables a selection of proprietary application modules to be attached. Downloadable user interface templates, much like XML style sheets, are rendered to a user interface displayed on the screen. These are associated with corresponding thumbnails and URI's that allow a user to surf through lists and catalogs of materials, and then to play them in the appropriate formats and provide the machine with a customized controller. A remote commander is simplified, yet expanded to control all the attached devices through interactions with the user interface.Type: GrantFiled: July 14, 2011Date of Patent: September 24, 2013Assignees: Sony Corporation, Sony Electronics Inc.Inventors: Edgar Tu, David Boyden, Takashi Hironaka, Thomas Dawson, George Williams, Ludovic Douillet, Rajesh Rao, Peter Rae Shintani, Djung Nguyen, Milton Frazier, Ian Charles Matthews, Behram Mario Dacosta, Robert Hardacker, Nicholas James Colsey, Mark Hanson, Jason R. Meerbergen, Leo Mark Pedlow, Rolf Toft
-
Publication number: 20110271296Abstract: A TV-set is equipped with HDMI and USB connections that allow it to display and run audio-video content from a variety of conventional consumer devices. The TV-set is further equipped to provide a secure HDMI-USB interface that will allow the transfer of licensed high definition content and Internet subscriber services. Such secure HDMI-USB interface also enables a selection of proprietary application modules to be attached. Downloadable user interface templates, much like XML style sheets, are rendered to a user interface displayed on the screen. These are associated with corresponding thumbnails and URI's that allow a user to surf through lists and catalogs of materials, and then to play them in the appropriate formats and provide the machine with a customized controller. A remote commander is simplified, yet expanded to control all the attached devices through interactions with the user interface.Type: ApplicationFiled: July 14, 2011Publication date: November 3, 2011Applicants: SONY ELECTRONICS INC., SONY CORPORATIONInventors: Edgar Tu, David Boyden, Takashi Hironaka, Thomas Dawson, George Williams, Ludovic Douillet, Rajesh Rao, Peter Rae Shintani, Djung Nguyen, Milton Frazier, Ian Charles Matthews, Behram Mario Dacosta, Robert Hardacker, Nicholas James Colsey, Mark Hanson, Jason R. Meerbergen, Leo Mark Pedlow, Rolf Toft
-
Patent number: 7996869Abstract: A TV-set is equipped with HDMI and USB connections that allow it to display and run audio-video content from a variety of conventional consumer devices. The TV-set is further equipped to provide a secure HDMI-USB interface that will allow the transfer of licensed high definition content and Internet subscriber services. Such secure HDMI-USB interface also enables a selection of proprietary application modules to be attached. Downloadable user interface templates, much like XML style sheets, are rendered to a user interface displayed on the screen. These are associated with corresponding thumbnails and URI's that allow a user to surf through lists and catalogs of materials, and then to play them in the appropriate formats and provide the machine with a customized controller. A remote commander is simplified, yet expanded to control all the attached devices through interactions with the user interface.Type: GrantFiled: August 16, 2007Date of Patent: August 9, 2011Assignees: Sony Corporation, Sony Electronics, Inc.Inventors: Edgar Tu, David Boyden, Takashi Hironaka, Thomas Dawson, George Williams, Ludovic Douillet, Rajesh Rao, Peter Rae Shintani, Djung Nguyen, Milton Frazier, Ian Charles Matthews, Behram Mario Dacosta, Robert Hardacker, Nicholas James Colsey, Mark Hanson, Jason R. Meerbergen, Leo Mark Pedlow, Rolf Toft
-
Patent number: 7797713Abstract: An Internet-enabled TV can display thumbnails of audio-video content available from an Internet server for selecting content to be displayed on the TV by a user manipulating a remote control. Internet information loading rules and thumbnail navigation rules are disclosed.Type: GrantFiled: September 5, 2007Date of Patent: September 14, 2010Assignees: Sony Corporation, Sony Electronics Inc.Inventors: Thomas Patrick Dawson, Ted Mark Dunn, Seth Hill, Yuko Nishikawa, Rolf Toft, George Williams