Patents by Inventor Simon Heinzle
Simon Heinzle has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 10095953Abstract: Techniques of depth modification for display applications are disclosed. A multiscopic video frame, including at least first and second images depicting a scene, is received. Depth information pertaining to the multiscopic video frame is received. A depth mapping function is determined based on one or more image characteristics of the multiscopic video frame, other than depth. The depth information is modified based on the depth mapping function. At least a third image is generated based on the modified depth information and based further on at least one of the first and second images, where the third image is output for display.Type: GrantFiled: May 28, 2014Date of Patent: October 9, 2018Assignees: Disney Enterprises, Inc., ETH Zurich (Eidgenoessische Technische Hochschule Zurich)Inventors: Aljosa Aleksej Andrej Smolic, Tunc Ozan Aydin, Steven Charles Poulakos, Simon Heinzle, Alexandre Chapiro
-
Patent number: 9843776Abstract: Methods and systems for generating stereoscopic content with granular control over binocular disparity based on multi-perspective imaging from representations of light fields are provided. The stereoscopic content is computed as piecewise continuous cuts through a representation of a light field, minimizing an energy reflecting prescribed parameters such as depth budget, maximum binocular disparity gradient, desired stereoscopic baseline. The methods and systems may be used for efficient and flexible stereoscopic post-processing, such as reducing excessive binocular disparity while preserving perceived depth or retargeting of already captured scenes to various view settings. Moreover, such methods and systems are highly useful for content creation in the context of multi-view autostereoscopic displays and provide a novel conceptual approach to stereoscopic image processing and post-production.Type: GrantFiled: July 15, 2015Date of Patent: December 12, 2017Assignee: Disney Enterprises, Inc.Inventors: Changil Kim, Alexander Hornung, Simon Heinzle, Wojciech Matusik, Markus Gross
-
Patent number: 9791706Abstract: A multi-planar plenoptic display assembly is provided that includes multiple spatially-varying light emitting and light modulating planes. The display assembly includes at least one light emitting device and may include, but does not require, a modulating device used in conjunction according to display methods taught herein to display light field data. A display assembly controller may be used to render a light field with depth into a multi-planar plenoptic display assembly by assigning decomposed portions of the light field to the display assembly for display or presentation by differing ones of the emitting elements and by operating a modulating device to provide a parallax barrier. In one embodiment, a dynamic parallax barrier and a number of bi-state screens. Another embodiment uses a beam splitter to co-locate two pairs of autostereoscopic displays each including a projector projecting 3D content, a parallax barrier, and an emissive/projector element.Type: GrantFiled: January 9, 2014Date of Patent: October 17, 2017Assignee: DISNEY ENTERPRISES, INC.Inventors: Nicola V. Ranieri, Simon Heinzle, Peter C. Barnum, Wojciech Matusik, Marcus Gross
-
Patent number: 9720245Abstract: A lenticular display device that is effective in increasing both perceived angular resolution and spatial resolution. These desirable results are achieved by modifying the lenslet array to better match the content of a given light field. An optimization algorithm or method (which may be implemented with software run on a computing device) is provided that analyzes an input light field and computes an optimal lenslet size, shape, and arrangement of sets of lenslets across the width of the array to better (or even best) match the input light field given a set of output parameters. The resulting lenticular display device (or print) shows higher detail and smoother motion parallax compared with fixed-size lens arrays. The usefulness of these content-adaptive lenticular prints has been demonstrated or proven using rendered simulations, by generating 3D-printed lens arrays according to the present description, and with user studies.Type: GrantFiled: November 27, 2012Date of Patent: August 1, 2017Assignee: Disney Enterprises, Inc.Inventors: Jan Kautz, James Tompkin, Wojciech Matusik, Simon Heinzle
-
Patent number: 9565414Abstract: Approaches are described for generating a multiview autostereoscopic image from a stereo three-dimensional input image pair. A stereo to multiview rendering system receives a stereo three-dimensional image including a left image and a right image at a first view position and a second view position, respectively. The system generates a first input warp and a second input warp that maps the left image and the right image to a third and fourth positions, respectively, where the third and fourth positions lie between the first and second view positions. The system generates a plurality of output warps based on the first warp and the second warp. The system resamples each output warp in the plurality of output warps to create a plurality of partial output images. The system interleaves the plurality of partial output images to generate a composite output image.Type: GrantFiled: December 18, 2013Date of Patent: February 7, 2017Assignees: Disney Enterprises, Inc., ETH Zurich (Eidgenoessische Technische Hochschule Zurich)Inventors: Aljosa Aleksej Andrej Smolic, Pierre Greisen, Simon Heinzle, Michael Schaffner, Frank Ka{hacek over (g)}an Gürkaynak
-
Patent number: 9363427Abstract: A device and method incorporates features of a temporal contrast sensor with a camera sensor in an imager. The method includes registering the camera sensor with the temporal contrast sensor as a function of a calibration target. The method includes receiving camera sensor data from the camera sensor and temporal contrast sensor data from the temporal contrast sensor. The method includes generating a plurality of images as a function of incorporating the temporal contrast sensor data with the camera sensor data.Type: GrantFiled: August 28, 2013Date of Patent: June 7, 2016Assignee: DISNEY ENTERPRISES, INC.Inventors: Simon Heinzle, Aljoscha Smolic, Oliver Wang, Alexander Sorkine Hornung, Yael Pritch, Henning Zimmer
-
Patent number: 9237331Abstract: A closed-loop control system for stereoscopic video capture is provided. At least two motorized lenses are positioned in accordance with specified parameters to capture spatially-disparate images of a scene. The motorized lenses focus light on a corresponding one of the at least two sensors, which generate image streams. One or more processors execute instructions to provide a stream analyzer and a control module. The stream analyzer receives the image streams from the sensors and analyzes the image streams and the specified parameters in real time; the stream analyzer then modifies the image streams and generates metadata. The control module then receives and analyzes the image streams and metadata and transmits updated parameters to a control mechanism that is coupled to the at least two motorized lenses. The control mechanism then modifies operation of the at least two motorized lenses in real time in accordance with the updated parameters.Type: GrantFiled: April 8, 2011Date of Patent: January 12, 2016Assignees: DISNEY ENTERPRISES, INC., ETH ZÜRICH (EIDGENÖESSISCHE TECHNISCHE HOCHSCHULE ZÜRICH)Inventors: Simon Heinzle, Pierre Greisen, Aljoscha Smolic, Wojciech Matusik, Markus Gross
-
Patent number: 9215440Abstract: Techniques are disclosed for rendering images. The techniques include receiving an input image associated with a source space, the input image comprising a plurality of source pixels, and applying an adaptive transformation to a source pixel, where the adaptive transformation maps the source pixel to a target space associated with an output image comprising a plurality of target pixels. The techniques further include determining a target pixel affected by the source pixel based on the adaptive transformation. The techniques further include writing the transformed source pixel into a location in the output image associated with the target pixel.Type: GrantFiled: October 17, 2012Date of Patent: December 15, 2015Assignee: Disney Enterprises, Inc.Inventors: Pierre Greisen, Simon Heinzle, Michael Schaffner, Aljosa Aleksej Andrej Smolic
-
Publication number: 20150348273Abstract: Techniques of depth modification for display applications are disclosed. A multiscopic video frame, including at least first and second images depicting a scene, is received. Depth information pertaining to the multiscopic video frame is received. A depth mapping function is determined based on one or more image characteristics of the multiscopic video frame, other than depth. The depth information is modified based on the depth mapping function. At least a third image is generated based on the modified depth information and based further on at least one of the first and second images, where the third image is output for display.Type: ApplicationFiled: May 28, 2014Publication date: December 3, 2015Applicant: Disney Enterprises, Inc.Inventors: Alexandre CHAPIRO, Tunc Ozan AYDIN, Steven Charles POULAKOS, Simon HEINZLE, Aljosa Aleksej Andrej SMOLIC
-
Patent number: 9202258Abstract: Techniques are disclosed for retargeting images. The techniques include receiving one or more input images, computing a two-dimensional saliency map based on the input images in order to determine one or more visually important features associated with the input images, projecting the saliency map horizontally and vertically to create at least one of a horizontal and vertical saliency profile, and scaling at least one of the horizontal and vertical saliency profiles. The techniques further include creating an output image based on the scaled saliency profiles. Low saliency areas are scaled non-uniformly while high saliency areas are scaled uniformly. Temporal stability is achieved by filtering the horizontal resampling pattern and the vertical resampling pattern over time. Image retargeting is achieved with greater efficiency and lower compute power, resulting in a retargeting architecture that may be implemented in a circuit suitable for mobile applications such as mobile phones and tablet computers.Type: GrantFiled: October 17, 2012Date of Patent: December 1, 2015Assignee: Disney Enterprises, Inc.Inventors: Pierre Greisen, Aljosa Aleksej Andrej Smolic, Simon Heinzle, Manuel Lang
-
Publication number: 20150319423Abstract: Methods and systems for generating stereoscopic content with granular control over binocular disparity based on multi-perspective imaging from representations of light fields are provided. The stereoscopic content is computed as piecewise continuous cuts through a representation of a light field, minimizing an energy reflecting prescribed parameters such as depth budget, maximum binocular disparity gradient, desired stereoscopic baseline. The methods and systems may be used for efficient and flexible stereoscopic post-processing, such as reducing excessive binocular disparity while preserving perceived depth or retargeting of already captured scenes to various view settings. Moreover, such methods and systems are highly useful for content creation in the context of multi-view autostereoscopic displays and provide a novel conceptual approach to stereoscopic image processing and post-production.Type: ApplicationFiled: July 15, 2015Publication date: November 5, 2015Applicant: DISNEY ENTERPRISES, INC.Inventors: CHANGIL KIM, ALEXANDER HORNUNG, SIMON HEINZLE, WOJCIECH MATUSIK, MARKUS GROSS
-
Patent number: 9179134Abstract: A multi-planar plenoptic display assembly with multiple spatially-varying light emitting and modulating planes. The display assembly includes at least one light emitting device and may include a modulating device used in conjunction according to display methods taught herein to display light field data. A display assembly controller may be used to render a light field with depth into a multi-planar plenoptic display assembly by assigning decomposed portions of the light field to the display assembly for display or presentation by differing ones of the emitting elements and by operating a modulating device to provide a parallax barrier. In one embodiment, a projector is used with bi-state screens. In another embodiment, two automultiscopic displays (either parallax barrier or lenticular lenses) are overlaid with a beam splitter. In a further embodiment, an oscillating mirror is used to temporally and optically move one automultiscopic layer (either parallax barrier or lenses) through space.Type: GrantFiled: November 27, 2012Date of Patent: November 3, 2015Assignee: DISNEY ENTERPRISES, INC.Inventors: Nicola V. Ranieri, Simon Heinzle, Quinn Smithwick, Markus Gross, Wojciech Matusik, Lanny Smoot, Daniel Reetz
-
Patent number: 9148658Abstract: A method for performing light-based calibration of optics with caustic surfaces. The method includes mapping a light detecting device to a programmable light source. Then, the method includes operating a calibration light source to direct light onto one or more caustic surfaces of an optical assembly, e.g., an assembly of one or more lenses, facets, lenticules, and lenslets. The method may then involve, with the light detecting device, capturing an image of a projection surface of the optical assembly, which is opposite the one or more caustic surfaces in the optical assembly, as the projection surface is illuminated by the light from the light source. Further, the method includes processing the captured image, along with the mapping of the light detecting device to the programmable light source, to generate a calibration map of the optical assembly including the caustic surfaces.Type: GrantFiled: January 15, 2014Date of Patent: September 29, 2015Assignee: Disney Enterprises, Inc.Inventors: Yael Pritch, Quinn Smithwick, Alexander Sorkine Hornung, Nicola Ranieri, Anselm Grundhofer, Lanny S. Smoot, Simon Heinzle, Markus Gross
-
Patent number: 9113043Abstract: Methods and systems for generating stereoscopic content with granular control over binocular disparity based on multi-perspective imaging from representations of light fields are provided. The stereoscopic content is computed as piecewise continuous cuts through a representation of a light field, minimizing an energy reflecting prescribed parameters such as depth budget, maximum binocular disparity gradient, desired stereoscopic baseline. The methods and systems may be used for efficient and flexible stereoscopic post-processing, such as reducing excessive binocular disparity while preserving perceived depth or retargeting of already captured scenes to various view settings. Moreover, such methods and systems are highly useful for content creation in the context of multi-view autostereoscopic displays and provide a novel conceptual approach to stereoscopic image processing and post-production.Type: GrantFiled: October 24, 2011Date of Patent: August 18, 2015Assignee: DISNEY ENTERPRISES, INC.Inventors: Changil Kim, Alexander Hornung, Simon Heinzle, Wojciech Matusik, Markus Gross
-
Publication number: 20150201188Abstract: A method for performing light-based calibration of optics with caustic surfaces. The method includes mapping a light detecting device to a programmable light source. Then, the method includes operating a calibration light source to direct light onto one or more caustic surfaces of an optical assembly, e.g., an assembly of one or more lenses, facets, lenticules, and lenslets. The method may then involve, with the light detecting device, capturing an image of a projection surface of the optical assembly, which is opposite the one or more caustic surfaces in the optical assembly, as the projection surface is illuminated by the light from the light source. Further, the method includes processing the captured image, along with the mapping of the light detecting device to the programmable light source, to generate a calibration map of the optical assembly including the caustic surfaces.Type: ApplicationFiled: January 15, 2014Publication date: July 16, 2015Applicant: DISNEY ENTERPRISES, INC,Inventors: YAEL PRITCH, QUINN SMITHWICK, ALEXANDER SORKINE HORNUNG, NICOLA RANIERI, ANSELM GRUNDHOFER, LANNY S. SMOOT, SIMON HEINZLE, MARKUS GROSS
-
Publication number: 20150062351Abstract: A device and method incorporates features of a temporal contrast sensor with a camera sensor in an imager. The method includes registering the camera sensor with the temporal contrast sensor as a function of a calibration target. The method includes receiving camera sensor data from the camera sensor and temporal contrast sensor data from the temporal contrast sensor. The method includes generating a plurality of images as a function of incorporating the temporal contrast sensor data with the camera sensor data.Type: ApplicationFiled: August 28, 2013Publication date: March 5, 2015Applicant: Disney Enterprises Inc.Inventors: Simon HEINZLE, Aljoscha SMOLIC, Oliver WANG, Alexander Sorkine HORNUNG, Yael PRITCH, Henning ZIMMER
-
Publication number: 20140347452Abstract: Approaches are described for generating a multiview autostereoscopic image from a stereo three-dimensional input image pair. A stereo to multiview rendering system receives a stereo three-dimensional image including a left image and a right image at a first view position and a second view position, respectively. The system generates a first input warp and a second input warp that maps the left image and the right image to a third and fourth positions, respectively, where the third and fourth positions lie between the first and second view positions. The system generates a plurality of output warps based on the first warp and the second warp. The system resamples each output warp in the plurality of output warps to create a plurality of partial output images. The system interleaves the plurality of partial output images to generate a composite output image.Type: ApplicationFiled: December 18, 2013Publication date: November 27, 2014Applicant: DISNEY ENTERPRISES, INC.Inventors: Aljosa Aleksej Andrej SMOLIC, Pierre GREISEN, Simon HEINZLE, Michael SCHAFFNER, Frank KAGAN GÜRKAYNAK
-
Publication number: 20140146388Abstract: A lenticular display device is described that is effective in increasing both perceived angular resolution and spatial resolution. These desirable results are achieved by modifying the lenslet array to better match the content of a given light field. An optimization algorithm or method (which may be implemented with software run on a computing device) is provided that analyzes an input light field and computes an optimal lenslet size, shape, and arrangement of sets of lenslets across the width of the array to better (or even best) match the input light field given a set of output parameters. The resulting lenticular display device (or print) shows higher detail and smoother motion parallax compared with fixed-size lens arrays. The usefulness of these content-adaptive lenticular prints has been demonstrated or proven using rendered simulations, by generating 3D-printed lens arrays according to the present description, and with user studies.Type: ApplicationFiled: November 27, 2012Publication date: May 29, 2014Applicant: DISNEY ENTERPRISES, INC.Inventors: Jan Kautz, James Tompkin, Wojciech Matusik, Simon Heinzle
-
Publication number: 20140125894Abstract: A multi-planar plenoptic display assembly is provided that includes multiple spatially-varying light emitting and light modulating planes. The display assembly includes at least one light emitting device and may include, but does not require, a modulating device used in conjunction according to display methods taught herein to display light field data. A display assembly controller may be used to render a light field with depth into a multi-planar plenoptic display assembly by assigning decomposed portions of the light field to the display assembly for display or presentation by differing ones of the emitting elements and by operating a modulating device to provide a parallax barrier. In one embodiment, a dynamic parallax barrier and a number of bi-state screens. Another embodiment uses a beam splitter to co-locate two pairs of autostereoscopic displays each including a projector projecting 3D content, a parallax barrier, and an emissive/projector element.Type: ApplicationFiled: January 9, 2014Publication date: May 8, 2014Applicant: DISNEY ENTERPRISES, INC.Inventors: NICOLA V. RANIERI, Simon Heinzle, Peter C. Barnum, Wojciech Matusik, Marcus Gross
-
Publication number: 20140104380Abstract: Techniques are disclosed for rendering images. The techniques include receiving an input image associated with a source space, the input image comprising a plurality of source pixels, and applying an adaptive transformation to a source pixel, where the adaptive transformation maps the source pixel to a target space associated with an output image comprising a plurality of target pixels. The techniques further include determining a target pixel affected by the source pixel based on the adaptive transformation. The techniques further include writing the transformed source pixel into a location in the output image associated with the target pixel.Type: ApplicationFiled: October 17, 2012Publication date: April 17, 2014Applicant: DISNEY ENTERPRISES, INC.Inventors: Pierre GREISEN, Simon HEINZLE, Michael SCHAFFNER, Aljosa Aleksej Andrej SMOLIC