Motion Correction Plus Resolution Enhancement Patents (Class 348/208.13)
  • Patent number: 11750937
    Abstract: In a high speed image capturing state, a camera signal processing circuit is not needed to perform a signal process at a high screen rate, but at a regular screen rate. In the high speed image capturing mode, raw data of 240 fps received from an image sensor 101 are recorded on a recording device 111 through a conversion processing section 201 and a recording device controlling circuit 210. Raw data that have been decimated and size-converted are supplied to a camera signal processing circuit 203 through a pre-processing circuit 202 and an image being captured is displayed on a display section 112 with a signal for which a camera process has been performed.
    Type: Grant
    Filed: May 17, 2022
    Date of Patent: September 5, 2023
    Assignee: Sony Group Corporation
    Inventors: Ryota Kosakai, Katsutoshi Aiki, Nobuyuki Sato, Hiroki Nagahama, Masatoshi Sase, Yutaka Yoneda
  • Patent number: 11729513
    Abstract: An electronic device and a high dynamic range (HDR) image generation method therefore are provided. The electronic device includes an image sensor and a processor, wherein the processor can be configured to adjust the exposure of the image sensor so as to acquire a first image having a first brightness and a plurality of second images having a second brightness, perform, on the first image, brightness conversion and noise attenuation of at least a first intensity so as to provide a third image having the second brightness, and generate a second HDR image on the basis of the first image and the third image, and generate a second HDR image on the basis of the first HDR image and the plurality of second images.
    Type: Grant
    Filed: August 3, 2022
    Date of Patent: August 15, 2023
    Assignee: Samsung Electronics Co., Ltd.
    Inventors: Sanghyeon Lim, Junhyeon Kim, Dongyoung Choi, Dongo Kim, Youngmin Kim, Junki Cho, Jiyoon Park, Kihuk Lee
  • Patent number: 11558558
    Abstract: According to some embodiments, a camera captures a plurality of images of substantially the same scene at different exposure levels for each video frame to be captured. These images are then deghosted, merged or fused, and processed using a spatially, and optionally also temporally, varying tonemapping operator, and the resulting image after some further finishing is written into a video stream. In some embodiments, this processing is performed in real time on a graphics processing unit present in the device containing the camera. Other applications are shown and discussed.
    Type: Grant
    Filed: May 31, 2021
    Date of Patent: January 17, 2023
    Inventor: Oliver Markus Haynold
  • Patent number: 11393125
    Abstract: An image capture device may be calibrated using calibration parameters. Optical medium through which the image capture device captures images may be changed. The calibration parameters of the image capture device may be dynamically modified to account for the change in the optical medium. The image capture device may be operated in accordance with the modified calibration parameters to capture images through the optical medium.
    Type: Grant
    Filed: December 9, 2019
    Date of Patent: July 19, 2022
    Assignee: GoPro, Inc.
    Inventors: Nicolas Rahmouni, Maxim Karpushin, Thomas Derbanne
  • Patent number: 11388380
    Abstract: In a high speed image capturing state, a camera signal processing circuit is not needed to perform a signal process at a high screen rate, but at a regular screen rate. In the high speed image capturing mode, raw data of 240 fps received from an image sensor 101 are recorded on a recording device 111 through a conversion processing section 201 and a recording device controlling circuit 210. Raw data that have been decimated and size-converted are supplied to a camera signal processing circuit 203 through a pre-processing circuit 202 and an image being captured is displayed on a display section 112 with a signal for which a camera process has been performed.
    Type: Grant
    Filed: March 16, 2021
    Date of Patent: July 12, 2022
    Assignee: Sony Corporation
    Inventors: Ryota Kosakai, Katsutoshi Aiki, Nobuyuki Sato, Hiroki Nagahama, Masatoshi Sase, Yutaka Yoneda
  • Patent number: 11232598
    Abstract: A method and a signal processor for receiving a data stream comprising at least two distinct sets of encoded data, at least one set of which is relative to transient/stochastic components of a signal. Based at least in part on the distinct sets of encoded data, the signal processor decodes and reconstructs a corresponding rendition of signal for each set of the encoded data. The distinct sets of renditions of signal are then combined into a single rendition of reconstructed signal.
    Type: Grant
    Filed: December 3, 2019
    Date of Patent: January 25, 2022
    Assignee: V-NOVA INTERNATIONAL LIMITED
    Inventors: Luca Rossato, Guido Meardi
  • Patent number: 11120675
    Abstract: A smart motion detection device includes an image sensor, an infrared sensor and a processor. The image sensor captures a monitoring image. The infrared sensor detects a thermal motion condition and provides an alarm signal accordingly. The processor executes an image recording mode of the image sensor only according to the alarm signal when an image quality of the monitoring image is unacceptable.
    Type: Grant
    Filed: July 24, 2019
    Date of Patent: September 14, 2021
    Assignee: Pix Art Imaging Inc.
    Inventors: Han-Chang Lin, Shiue-Shin Liu, Shuen-Yin Bai
  • Patent number: 11082754
    Abstract: Various aspects of a system and method to generate a multi-dimensional video are disclosed herein. In accordance with an embodiment, the system includes an application server, which receives a plurality of videos feeds from a plurality of image-capturing devices that capture a pre-defined area. Further, sensor data is received from a plurality of sensors associated with one or more subjects in the pre-defined area. Thereafter, a location-of-occurrence of an upcoming activity-of-interest is predicted in the predefined area based on the received plurality of video feeds, sensor data, and/or pre-stored statistical data of the historical performance of the one or more subjects. One or more control instructions are communicated to one or more of the plurality of image-capturing devices to focus towards the predicted location-of-occurrence to enable the generation of the multi-dimensional video.
    Type: Grant
    Filed: August 18, 2016
    Date of Patent: August 3, 2021
    Assignee: SONY CORPORATION
    Inventor: Nabarun Goswami
  • Patent number: 11012630
    Abstract: An image processor includes: an image sensor which outputs a short exposure image and a long exposure image; a sensor controller which controls first and second exposure sensitivities; a motion blending ratio calculator which calculates a motion blending ratio based on a motion amount of a subject; a motion-adapted image synthesizer which generates a motion-adapted image based on the motion blending ratio; and an HDR image synthesizer which synthesizes the motion-adapted image and the short exposure image, to generate an HDR image. The sensor controller controls the first exposure sensitivity: so that the first exposure time changes from increasing to constant and the first sensor gain changes from constant to increasing, when the brightness of the subject decreases from a first subject brightness; and so that the first exposure time changes to increasing and the first sensor gain changes to constant, when the brightness decreases from a second subject brightness.
    Type: Grant
    Filed: September 24, 2020
    Date of Patent: May 18, 2021
    Assignee: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
    Inventors: Yoshimitsu Noguchi, Tomokuni Iijima, Katsuyuki Morita, Shigeru Furumiya
  • Patent number: 10572457
    Abstract: Appearance of a forming condition of data constituting a database is promoted when work to construct the database is made efficient. The database construction device includes an original file storage, a configuration data specification information storage, and a database construction unit. The database construction unit stores one of pieces of original data as follows in the database storage device as at least a part of the database while correlating the one of pieces of original data with configuration data specification information. The original data is correlated with original data specification information stored in the configuration data specification information storage in the pieces of original data specification information of an original file. The original data specification information indicates a plurality of kinds of original data forming conditions. The configuration data specification information indicates a plurality of kinds of database configuration data forming conditions.
    Type: Grant
    Filed: February 13, 2017
    Date of Patent: February 25, 2020
    Assignee: IWASAKI ELECTRIC MFG. CO., LTD.
    Inventor: Kenji Tatani
  • Patent number: 10567675
    Abstract: An imaging apparatus includes an imaging unit, a first image processor, and a second image processor. The imaging unit sequentially acquires image frames whose photography conditions have been changed. The first image processor generates moving image data by performing image processing to maintain continuity between the image frames obtained by changing the photography conditions. The second image processor composes the image frames whose photography conditions have been changed, to generate condition-changed still image data.
    Type: Grant
    Filed: September 7, 2017
    Date of Patent: February 18, 2020
    Assignee: Olympus Corporation
    Inventors: Koichi Shintani, Masaomi Tomizawa, Osamu Nonaka
  • Patent number: 10404911
    Abstract: An image pickup apparatus includes an image data production unit which produces data of images of a plurality of resolutions, from an image frame obtained by picking up an image of a target object as a moving picture for each pixel string which configures a row, and an image sending unit which extracts, from the data of the images of the plurality of resolutions, pixel strings of a region requested from a host terminal and connect the extracted pixel strings for each number of pixels determined in accordance with a given rule to produce a stream and then transmit the stream to the host terminal. The image sending unit adjusts a connection pixel number of data of an image of a particular resolution such that, where N is an integer, data for 1/(N+1) frame are transmitted with respect to one frame of data of an image of the other resolution.
    Type: Grant
    Filed: January 11, 2018
    Date of Patent: September 3, 2019
    Assignee: SONY INTERACTIVE ENTERTAINMENT INC.
    Inventors: Akio Ohba, Hidehiko Ogasawara
  • Patent number: 10362276
    Abstract: A transmission apparatus that transmits an image to be distributed to a reception apparatus includes a holding unit configured to hold a plurality of settings that include resolution of a captured image and that are used for generating the image to be distributed, a reception unit configured to receive, from the reception apparatus, specification information for specifying one of the plurality of held settings in relation to superimposition of the mask image and superimposition information indicating a position at which the mask image is superimposed upon the image to be distributed generated in accordance with the one of the settings specified by the specification information, and a setting unit configured to set a position at which the mask image is superimposed upon the captured image on the basis of the specified one of the settings and the superimposition information received by the reception unit.
    Type: Grant
    Filed: November 22, 2016
    Date of Patent: July 23, 2019
    Assignee: Canon Kabushiki Kaisha
    Inventor: Takahiro Iwasaki
  • Patent number: 10163967
    Abstract: An imaging device with low power consumption. The imaging device includes a plurality of pixels arranged in a matrix, a first circuit, a second circuit, a third circuit, and a fourth circuit. The first circuit has a function of converting an analog signal into a digital signal. The second circuit has a function of detecting a difference between image data of a first frame and image data of a second frame. The third circuit has a function of controlling the frequency of a clock signal. The fourth circuit has a function of generating clock signals of a plurality of frequencies.
    Type: Grant
    Filed: June 10, 2016
    Date of Patent: December 25, 2018
    Assignee: Semiconductor Energy Laboratory Co., Ltd.
    Inventors: Yuki Okamoto, Seiichi Yoneda, Yoshiyuki Kurokawa
  • Patent number: 10003735
    Abstract: Highly accurate focus adjustment device can be provided, for example, that includes a focus detection area setting unit that sets a focus detection area to a position of a face detected in an image pickup field angle, and sets a luminance addition number and a correlation addition number in the focus detection area according to a size of the face, and performs focus detection for the face, thereby to suppress a decrease in accuracy.
    Type: Grant
    Filed: August 24, 2016
    Date of Patent: June 19, 2018
    Assignee: Canon Kabushiki Kaisha
    Inventor: Naoyuki Ohara
  • Patent number: 9979945
    Abstract: Systems, methods, and media for reconstructing a space-time volume from a coded image are provided. In accordance with some embodiments, systems for reconstructing a space-time volume from a coded image are provided, the systems comprising: an image sensor that outputs image data; and at least one processor that: causes a projection of the space-time volume to be captured in a single image of the image data in accordance with a coded shutter function; receives the image data; and performs a reconstruction process on the image data to provide a space-time volume corresponding to the image data.
    Type: Grant
    Filed: January 13, 2017
    Date of Patent: May 22, 2018
    Assignee: Sony Corporation
    Inventors: Yasunobu Hitomi, Jinwei Gu, Mohit Gupta, Tomoo Mitsunaga, Shree K. Nayar
  • Patent number: 9955072
    Abstract: An imaging device for producing a high resolution image of a target with an imaging sensor and a method using the same is provided. The method is comprised of the steps of: determining, from the target size, resolution requirements of the image to be produced; capturing multiple individual low resolution images of the target, a minimum number of individual images captured being based upon the resolution requirements determined in the determining resolution requirements step; moving the OIS module to specific positions between the capturing of the individual low resolution images in the capturing multiple low resolution images step, the specific positions being based upon the resolution requirements in the determining resolution requirements step; and processing the multiple low resolution images to produce a high resolution image.
    Type: Grant
    Filed: June 2, 2017
    Date of Patent: April 24, 2018
    Assignee: Hand Held Products, Inc.
    Inventors: Nianyi Li, Scott McCloskey
  • Patent number: 9916644
    Abstract: A method for removing a ghost artifact from a multiple-exposure image of a scene method includes steps of generating and segmenting a difference mask, determining a lower threshold and an upper threshold, generating a refined mask, and generating a corrected image. The difference mask includes a plurality of absolute differences in luminance-values between the multiple-exposure image and a first image of the scene. The segmenting step involves segmenting the difference mask into a plurality of blocks. The lower and upper thresholds are based on statistical properties of the blocks. The method generates the refined mask by mapping each absolute difference to a respective one of a plurality refined values, of the refined mask, equal to a function of the absolute difference, the lower threshold, and the upper threshold. The corrected image is a weighted sum of the first image and the multiple-exposure image, weights being based on the refined mask.
    Type: Grant
    Filed: September 9, 2016
    Date of Patent: March 13, 2018
    Assignee: OmniVision Technologies, Inc.
    Inventors: Sarvesh Swami, Donghui Wu, Timofey Uvarov
  • Patent number: 9838659
    Abstract: An image processing device includes a processor including hardware. The processor is configured to implement an image acquisition process that acquires an image captured by an image sensor that includes a first-color filter having first transmittance characteristics, a second-color filter having second transmittance characteristics, and a third-color filter having third transmittance characteristics, and a correction process that multiplies a pixel value that corresponds to a second color by a first coefficient to calculate a component value that corresponds to the overlapping region of the first transmittance characteristics and the third transmittance characteristics, and corrects a pixel value that corresponds to a first color, the pixel value that corresponds to the second color, and a pixel value that corresponds to a third color based on the component value that corresponds to the overlapping region.
    Type: Grant
    Filed: December 10, 2015
    Date of Patent: December 5, 2017
    Assignee: OLYMPUS CORPORATION
    Inventor: Shinichi Imade
  • Patent number: 9784822
    Abstract: A time-of-flight sensor device generates and analyzes a high-resolution depth map frame from a high-resolution image to determine a mode of operation for the time-of-flight sensor and an illuminator and to control the time-of-flight sensor and illuminator according to the mode of operation. A binned depth map frame can be created from a binned image from the time-of-flight sensor and combined with the high-resolution depth map frame to create a compensated depth map frame.
    Type: Grant
    Filed: September 14, 2015
    Date of Patent: October 10, 2017
    Assignee: TEXAS INSTRUMENTS INCORPORATED
    Inventors: Werner Adam Metz, Dong-Ik Ko
  • Patent number: 9672650
    Abstract: An image processing apparatus includes: a digital processing unit that performs digital processing on accepted one or at least two images, thereby acquiring one or at least two processed images; a physical property information acquiring unit that acquires physical property information, which is information relating to a physical property that has been lost compared with one or more physical properties of a target contained in the images, in the one or at least two processed images; a physical property processing unit that performs physical property processing, which is processing for adding a physical property corresponding to the physical property information, using the one or at least two processed images; and an output unit that outputs a processed image subjected to the physical property processing. Accordingly, it is possible to reproduce lost physical properties of a target expressed in an image, to the fullest extent possible.
    Type: Grant
    Filed: February 10, 2014
    Date of Patent: June 6, 2017
    Assignee: I-CUBED RESEARCH CENTER INC.
    Inventor: Tetsujiro Kondo
  • Patent number: 9674430
    Abstract: An imaging device for producing a high resolution image of a target with an imaging sensor and a method using the same is provided. The method is comprised of the steps of: determining, from the target size, resolution requirements of the image to be produced; capturing multiple individual low resolution images of the target, a minimum number of individual images captured being based upon the resolution requirements determined in the determining resolution requirements step; moving the OIS module to specific positions between the capturing of the individual low resolution images in the capturing multiple low resolution images step, the specific positions being based upon the resolution requirements in the determining resolution requirements step; and processing the multiple low resolution images to produce a high resolution image.
    Type: Grant
    Filed: March 9, 2016
    Date of Patent: June 6, 2017
    Assignee: Hand Held Products, Inc.
    Inventors: Nianyi Li, Scott McCloskey
  • Patent number: 9648286
    Abstract: It is an objective of the present invention to reduce processing loads for video analysis utilizing information throughout the facility. A video monitoring system according to the present invention simulates a flow of a moving object within a video captured by multiple monitoring cameras, calculates a parameter correlated with a processing load for movement analysis of the moving object according to the simulation result, and specifies a processing scheme that is capable of reducing the processing load according to a correspondence relationship between the parameter and the simulation result (refer to FIG. 2).
    Type: Grant
    Filed: August 28, 2014
    Date of Patent: May 9, 2017
    Assignee: Hitachi, Ltd.
    Inventors: Kenichi Yoneji, Tsukasa Yamauchi, Tomoaki Yoshinaga, Daisuke Matsubara
  • Patent number: 9648251
    Abstract: The present invention relates in general to a method of high dynamic range (HDR) digital imaging and determining and setting exposure parameters for detection of image data by an imaging device, such as a digital camera, to produce a high dynamic range (HDR) image of a scene. The method includes setting an exposure (Ei+1) for a next frame (fi+1) as a function of the current exposure (Ei), of a first brightness distribution (Hfulli) of colors of pixels of the current frame (fi) and of a second brightness distribution (Hmotioni) of colors of those pixels weighted by the motion of those pixels, such that motion is evaluated for those pixels in comparison with a previous frame (fi), capturing the next frame (fi+1) under the set exposure (Ei+1) and merging the next frame (fi+1) with at least the current frame (fi) into an HDR image.
    Type: Grant
    Filed: January 19, 2016
    Date of Patent: May 9, 2017
    Assignee: Thomson Licensing
    Inventors: Foteini Tania Pouli, Erik Reinhard, Yulia Gryaditskaya
  • Patent number: 9478010
    Abstract: A method, device, system, and article of manufacture are provided for generating an enhanced image of a predetermined scene from images. In one embodiment, a method comprises receiving, by a computing device, a first indication associated with continuous image capture of a predetermined scene being enabled; in response to the continuous image capture being enabled, receiving, by the computing device, from an image sensor, a reference image and a first image, wherein each of the reference image and the first image is of the predetermined scene and has a first resolution; determining an estimated second resolution of an enhanced image of the predetermined scene using the reference image and the first image; and in response to the continuous image capture being disabled, determining the enhanced image using the reference image and the first image, wherein the enhanced image has a second resolution that is at least the first resolution and about the estimated second resolution.
    Type: Grant
    Filed: May 5, 2014
    Date of Patent: October 25, 2016
    Assignee: Google Technology Holdings LLC
    Inventor: Michael D McLaughlin
  • Patent number: 9479694
    Abstract: Implementations of the disclosed technology include techniques for autonomously collecting image data, and generating photo summaries based thereon. In some implementations, a plurality of images may be autonomously sampled from an available stream of image data. For example, a camera application of a smartphone or other mobile computing device may present a live preview based on a stream of data from an image capture device. The live stream of image capture data may be sampled and the most interesting photos preserved for further filtering and presentation. The preserved photos may be further winnowed as a photo session continues and an image object generated summarizing the remaining photos. Accordingly, image capture data may be autonomously collected, filtered, and formatted to enable a photographer to see what moments they missed manually capturing during a photo session.
    Type: Grant
    Filed: October 28, 2014
    Date of Patent: October 25, 2016
    Assignee: Google Inc.
    Inventors: Rodrigo Lima Carceroni, Marius Renn, Alan Newberger, Sascha Häberling, Jacob Mintz, Andrew Huibers
  • Patent number: 9432575
    Abstract: An image processing apparatus includes an identification unit configured to identify a main subject region and a background region different from the main subject region with respect to a plurality of images captured by an image capturing unit, a composition unit configured to align the main subject regions of the plurality of images and to generate a composite image in which predetermined blur processing is applied to the background region, a detection unit configured to detect an amount of movement of the background region between the plurality of images based on an image of the background region or information about shaking of the image capturing unit with respect to the plurality of images, and a control unit configured to control an amount of blurring in the blur processing based on the amount of movement of the background region between the plurality of images detected by the detection unit.
    Type: Grant
    Filed: May 23, 2014
    Date of Patent: August 30, 2016
    Assignee: Canon Kabushiki Kaisha
    Inventor: Hiroaki Kuchiki
  • Patent number: 9330446
    Abstract: Provided are a monitoring system and an operating method thereof, and more particularly, an image processing method and apparatus for removing a motion blur of a wide dynamic range (WDR) image by using a machine learning algorithm. The image processing method includes: generating an overlap image by overlapping a first image having a predetermined exposure time and a second image having an exposure time different from that of the first image; detecting a region of interest (ROI) in which a motion blur occurs in the overlap image; and performing a motion blur removing operation of changing an image in the ROI to any one of the first image and the second image by applying a first machine learning algorithm.
    Type: Grant
    Filed: July 30, 2013
    Date of Patent: May 3, 2016
    Assignee: Hanwha Techwin Co., Ltd.
    Inventor: Jong-Seon Park
  • Patent number: 9307212
    Abstract: A technique is provided for generating sharp, well-exposed, color images front low-light images. A series of under-exposed images is acquired. A mean image is computed and a sum image is generated each based on the series of under-exposed images. Chrominance variables of pixels of the mean image are mapped to chrominance variables of pixels of the sum image. Chrominance values of pixels within the series of under-exposed images are replaced with chrominance values of the sum image. A set of sharp, well-exposed, color images is generated based on the series of under-exposed images with replaced chrominance values.
    Type: Grant
    Filed: July 27, 2015
    Date of Patent: April 5, 2016
    Assignee: Fotonation Limited
    Inventor: Ruxandra Vranceanu
  • Patent number: 9124809
    Abstract: There is provided an image processing apparatus including an HDR (High Dynamic Range) processing unit inputting images picked up while exposure control that changes an exposure time is being carried out with a predetermined spatial period and a predetermined temporal period on pixels that compose an image sensor, and carrying out image processing. The HDR processing unit generates a first combined image by combining pixel values of a plurality of images with different sensitivities generated by an interpolation process using a plurality of consecutively picked-up images, generates a second combined image by combining pixel values of a plurality of images with different sensitivities generated by an interpolation process that uses a single picked-up image, and generates an HDR image by executing a pixel value blending process on the first combined image and the second combined image in accordance with a blending ratio calculated in accordance with movement detection information.
    Type: Grant
    Filed: June 30, 2014
    Date of Patent: September 1, 2015
    Assignee: Sony Corporation
    Inventors: Shun Kaizu, Tomoo Mitsunaga, Yasushi Saito, Masanori Kasai
  • Patent number: 9094587
    Abstract: An image processing apparatus includes a calculation unit configured to calculate a first gain for adjusting brightness of a first image and a second image based on the first image captured at a first exposure amount and the second image captured at a second exposure amount, a division unit configured to divide the first gain into a second gain which changes according to a position in an image and a third gain which does not change according to a position in an image, a first gain correction unit configured to perform gain correction on the second image with the second gain, and a second gain correction unit configured to perform gain correction on the second image with the third gain.
    Type: Grant
    Filed: January 12, 2015
    Date of Patent: July 28, 2015
    Assignee: Canon Kabushiki Kaisha
    Inventor: Masaaki Matsuoka
  • Patent number: 9094648
    Abstract: A technique is provided for generating sharp, well-exposed, color images from low-light images. A series of under-exposed images is acquired. A mean image is computed and a sum image is generated each based on the series of under-exposed images. Chrominance variables of pixels of the mean image are mapped to chrominance variables of pixels of the sum image. Chrominance values of pixels within the series of under-exposed images are replaced with chrominance values of the sum image. A set of sharp, well-exposed, color images is generated based on the series of under-exposed images with replaced chrominance values.
    Type: Grant
    Filed: November 6, 2014
    Date of Patent: July 28, 2015
    Assignee: FotoNation Limited
    Inventor: Ruxandra Vranceanu
  • Patent number: 9036045
    Abstract: An image processing apparatus divides image data captured while a flash light is not emitted into a plurality of areas, calculates color evaluation value for each of the divided areas. The apparatus divides image data captured while the flash light is emitted into a plurality of areas and to calculate color evaluation value for each of the divided areas. The apparatus determines a movement for each of the plurality of areas in accordance with the calculated color evaluation value and the calculated color evaluation value. The apparatus performs white balance processing into image data in accordance with the determined movement.
    Type: Grant
    Filed: April 20, 2012
    Date of Patent: May 19, 2015
    Assignee: Canon Kabushiki Kaisha
    Inventor: Takaaki Fukui
  • Patent number: 9025038
    Abstract: This invention provides an image free from camera shake by using a plurality of images captured by a camera array image capturing apparatus. A determination unit determines whether to execute a camera shake correction processing. A memory unit temporarily stores only a group of images determined by a determination unit to be camera shake corrected. A camera shake correcting unit synthesizes images to correct blurs in the images. A matching point searching unit determines matching pixels by checking pixel value similarity between images. A moving amount calculating unit, based on the result acquired by the matching point searching unit, calculates a moving amount of each pixel between images. A position correcting unit, based on the moving amount of each pixel calculated by the moving amount calculating unit, corrects the positions of the images. An image synthesizing unit synthesizes a group of images that are position-corrected by the position correcting unit.
    Type: Grant
    Filed: September 5, 2012
    Date of Patent: May 5, 2015
    Assignee: Canon Kabushiki Kaisha
    Inventor: Shohei Tsutsumi
  • Patent number: 9007471
    Abstract: A method of controlling a digital photographing apparatus successively photographs one short exposure image and a plurality of long exposure images. A global motion of each of the long exposure images is compensated based on the one short exposure image. A Point Spread Function (PSF) of each of the one short exposure image and the long exposure images are extracted. A shake correction image is extracted through a repeat estimation by determining an initial value of the repeat estimation based on the one short exposure image and the long exposure images and using each PSF.
    Type: Grant
    Filed: December 22, 2010
    Date of Patent: April 14, 2015
    Assignee: Samsung Electronics Co., Ltd.
    Inventor: Soon-geun Jang
  • Patent number: 8976254
    Abstract: Embodiments of the invention provide techniques for upsampling a video sequence for coding. According to the method, an estimate of camera motion may be obtained from motion sensor data. Video data may be analyzed to detect motion within frames output from a camera that is not induced by the camera motion. When non-camera motion falls within a predetermined operational limit, video upsampling processes may be engaged. In another embodiment, video upsampling may be performed by twice estimating image content for a hypothetical new a frame using two different sources as inputs. A determination may be made whether the two estimates of the frame match each other sufficiently well. If so, the two estimates may be merged to yield a final estimated frame and the new frame may be integrated into a stream of video data.
    Type: Grant
    Filed: November 30, 2012
    Date of Patent: March 10, 2015
    Assignee: Apple Inc.
    Inventors: Douglas Scott Price, Xiaosong Zhou, Hsi-Jung Wu
  • Patent number: 8964040
    Abstract: Motion sensor data may be used to register a sequence of standard dynamic range images for producing a high dynamic range (HDR) image, reducing use of computational resources over software visual feature mapping techniques. A rotational motion sensor may produce information about orientation changes in the imaging device between images in the sequence of images sufficient to allow registration of the images, instead of using registration based on analysis of visual features of the images. If the imaging device has been moved laterally, then the motion sensor data may not be useful and visual feature mapping techniques may be employed to produce the HDR image.
    Type: Grant
    Filed: June 6, 2012
    Date of Patent: February 24, 2015
    Assignee: Apple Inc.
    Inventors: Marius Tico, Jianping Zhou, Todd S. Sachs
  • Patent number: 8941751
    Abstract: Technologies are generally described herein for generating a higher resolution still frame. Some example technologies may configure a video camera at a first configuration, which the video camera to capture video at a first pixel offset. The technologies may capture a first frame of al field-of-view through the video camera configured at the first configuration. The first flame may contain the field-of-view captured at the first pixel offset. The technologies may adjust the video camera from the first configuration to a second configuration, which adapts the video camera to capture the video at a second pixel offset, the adjustment using a hardware mechanism. The technologies may capture a second frame of the field-of-view through the video camera configured at the second configuration. The second frame may contain the field-of-view captured at the second pixel offset.
    Type: Grant
    Filed: May 6, 2011
    Date of Patent: January 27, 2015
    Assignee: Empire Technology Development LLC
    Inventors: Henry Leung, Xiaoxiang Liu
  • Patent number: 8902319
    Abstract: An image signal processing apparatus includes: a basic movement level finding section for detecting a movement level of a video image; a timing controller for dividing one frame period into a plurality of periods containing a sub-frame A period and a sub-frame B period; a sub-frame A image signal generating section for subjecting, to a smoothing process, image signals which are supplied to pixels, in the sub-frame A period, in accordance with the movement level of the video image; a sub-frame B image signal generating section for subjecting, to an emphasizing process, image signals which are supplied to pixels, in the sub-frame B period, in accordance with the movement level of the video image; and an applied movement level finding section for finding, from a movement level of a video image of a current frame period and a movement level of a video image of a previous frame period, an applied movement level which is applied to the sub-frame A image signal generating section and/or the sub-frame B image signal
    Type: Grant
    Filed: May 28, 2009
    Date of Patent: December 2, 2014
    Assignee: Sharp Kabushiki Kaisha
    Inventor: Tomoyuki Ishihara
  • Patent number: 8878946
    Abstract: An image processing apparatus sets one of a plurality of obtained images as a reference image, and detects a motion vector relative to the reference image for the other images. A scalar amount of the detected motion vector is compared with a threshold value representing an image displacement limit for combining an image with the reference image, and whether a condition for combining with the reference image is satisfied is determined. If the condition is satisfied, a corresponding image is moved to cancel the detected motion vector and combined with the reference image, and an image in which displacement between images has been corrected is output. The threshold value is changed by a control unit depending on an image-capturing setting set when the plurality of images are captured, the condition of an object image in the plurality of captured images, or the like.
    Type: Grant
    Filed: June 30, 2011
    Date of Patent: November 4, 2014
    Assignee: Canon Kabushiki Kaisha
    Inventor: Atsushi Fujita
  • Patent number: 8830395
    Abstract: Systems and methods are provided for upscaling a digital image. A digital image to be upscaled is accessed, where the digital image comprises a plurality of pixel values. A first half pixel value is computed for a first point in the digital image based on a plurality of pixel values of the digital image surrounding the first point and an activity level. A second half pixel value is computed for a second point in the digital image, and an interpolated pixel of an upscaled version of the digital image is determined using a plurality of the pixel values, the first half pixel value, and the second half pixel value.
    Type: Grant
    Filed: December 10, 2013
    Date of Patent: September 9, 2014
    Assignee: Marvell World Trade Ltd.
    Inventors: Yun Gong, Dam Le Quang
  • Patent number: 8830398
    Abstract: It is determined whether or not an input image is an image converted from an image with a relatively low resolution based on one frame of an image. A resolution determination device includes: an edge strength calculator configured to obtain an edge strength of a pixel included in an input image based on luminance of the pixel and luminance of a pixel adjacent to the pixel, for each of a plurality of pixels included in the input image; and a resolution determiner configured to determine whether or not the input image is an image upconverted from an image with a predetermined resolution or less, based on distribution of the edge strengths.
    Type: Grant
    Filed: December 18, 2012
    Date of Patent: September 9, 2014
    Assignee: Panasonic Corporation
    Inventors: Koichi Inoue, Shinichi Tomioka, Atsuhisa Kageyama
  • Patent number: 8824735
    Abstract: A method for determining a shift between two images, determining a first correlation in a first direction, the first correlation being derived from a first image projection characteristics and a second image projection characteristics, and a second correlation in a second direction, the second correlation being derived from the first image projection characteristics and the second image projection characteristics. The method determines a set of hypotheses from a first plurality of local maxima of the first correlation and a second plurality of local maxima of the second correlation. The method then calculates a two-dimensional correlation score between the first image and the second image based on a shift indicated in at least one of the set of hypotheses, and selecting one of the set of hypotheses as the shift between the first image and the second image based on the calculated two-dimensional correlation score.
    Type: Grant
    Filed: July 26, 2012
    Date of Patent: September 2, 2014
    Assignee: Canon Kabushiki Kaisha
    Inventor: Tuan Quang Pham
  • Patent number: 8810665
    Abstract: An imaging device is provided that includes an image blur evaluator, an imaging processor, an edge contour extraction processor, an image divider, and a distance information detector. The image blur evaluator evaluates the amount of blurring in an image. The imaging processor captures secondary images of the same object at different lens positions. The edge contour extraction processor extracts edge contour components of the secondary images and creates an edge-contour extracted image for each of the secondary images. The image divider divides each edge-contour extracted image into a plurality of relatively large blocks when blurring is evaluated to be relatively substantial, and into relatively small blocks when the blurring is evaluated to be moderate. The distance information detector detects distance information of the objects captured in each block based on the contrast.
    Type: Grant
    Filed: August 10, 2012
    Date of Patent: August 19, 2014
    Assignee: Pentax Ricoh Imaging Company, Ltd.
    Inventor: Hiroyasu Uehara
  • Patent number: 8767080
    Abstract: A system, apparatus, or method is provided for imaging and for capturing visuals to provide image manipulation options for increasing resolution of subject images. A system, apparatus or method for increasing resolution of subject images using a camera to deliver unexposed photographic emulsion or a digital image and to generate images of greater resolution by modifying digital images or modifying digital and emulsion images.
    Type: Grant
    Filed: January 29, 2009
    Date of Patent: July 1, 2014
    Assignee: Cedar Crest Partners Inc.
    Inventor: Craig Mowry
  • Patent number: 8750636
    Abstract: Some embodiments allow a video editor to remove unwanted camera motion from a sequence of video images (e.g., video frames). Some embodiments are implemented in a video editing application. Some of these embodiments distinguish unwanted camera motion from the intended underlying motion of a camera (e.g., panning and zooming) and/or motion of objects within the video sequence.
    Type: Grant
    Filed: June 20, 2011
    Date of Patent: June 10, 2014
    Assignee: Apple Inc.
    Inventor: Christophe Souchard
  • Patent number: 8723969
    Abstract: An image processor in an image capture device compensates for the effects of undesirable camera shakes occurring during video capture The image processor receives a pair of source frames representing images of a scene, generates a pair of subsampled frames from the source frames, and computes a coarse displacement of the captured image due to camera shakes by comparing the two subsampled frames. The image processor may then refine the determined coarse displacement by comparing the two source frames and a bound determined by an extent of subsampling, and compensate for the displacement accordingly. Display aberrations such as blank spaces caused due to shifting are also avoided by displaying only a portion of the captured image and shifting the displayed portion to compensate for camera shake. The image processor also recognizes displacements due to intentional camera movement, and does not correct for such displacements.
    Type: Grant
    Filed: March 20, 2007
    Date of Patent: May 13, 2014
    Assignee: Nvidia Corporation
    Inventors: Shashank Garg, Vinayak Pore, Ashish Thakur, Shang-Hung Lin
  • Patent number: 8711231
    Abstract: A digital image processing device and processing method thereof are provided. The device includes a digital image capturing module, an image enlarging module, an image correcting module, and an image blending module. The digital image capturing module captures a plurality of first resolution images. The image enlarging module enlarges the first resolution images and produces a plurality of second resolution images. The image correcting module selects a target image and produces a plurality of corrected images. The image blending module performs direction gradient operation on each of the pixels of the target and corrected images and produces a plurality of gradient differential values. The image blending module performs a weighting sum operation on each of the pixels of the target and corrected images and produces a third resolution images.
    Type: Grant
    Filed: August 5, 2011
    Date of Patent: April 29, 2014
    Assignee: Altek Corporation
    Inventors: Che-Lun Chuang, Chung-Ta Wu, Hong-Long Chou
  • Patent number: 8711230
    Abstract: An image capture apparatus shifts an image sensor at each shooting initiation time in the continuous-shoot mode in the event of a camera-shake in response to a control signal which reflects an offset signal indicating pixel shifting information in a compensating signal indicating shake compensating information. Thereby, a change in the shooting range of the image sensor due to the camera-shake is mostly compensated for and the shooting range is changed by one pixel at each shooting. Thus, even in the event of a camera-shake, the distance between a feature portion of a main subject and a fixed pattern noise can be made different for each of continuously shot images. As a result, even in the event of the camera-shake, a composite image having fixed pattern noises scattered can be obtained by combining continuously shot images with reference to the feature portion.
    Type: Grant
    Filed: April 16, 2009
    Date of Patent: April 29, 2014
    Assignee: Casio Computer Co., Ltd.
    Inventor: Kazuhisa Matsunaga
  • Patent number: 8704900
    Abstract: An imaging apparatus and an imaging method able to obtain images having no blurring with a small number of images under different exposure conditions and capable of shortening a processing time, wherein an imaging apparatus 10 captures a plurality of images including an image having a short exposure time and high resolution, but having much noise and an image having a long exposure time, but having little noise and low resolution by an optical system 11 and an imaging element 12. After the signal processing of a signal processing part 13, a CPU 14 detects positional deviation among the captured images and blurring, changes the ratio of the plurality of images according to the distances from the edge, and combines these to thereby form an image free from blurring and reducing noise.
    Type: Grant
    Filed: October 11, 2006
    Date of Patent: April 22, 2014
    Assignee: Kyocera Corporation
    Inventors: Go Watanabe, Toshiki Hatori, Takeshi Kawano, Ryuuichi Sawada