Abstract: An image processing method is capable of performing a reconstruction for an input image so as to generate a plurality of output images having a plurality of focus positions different from each other, and the image processing method includes the steps of obtaining the input image that is an image containing information of an object space obtained from a plurality of points of view using an image pickup apparatus having an imaging optical system and an image pickup element including a plurality of pixels, and calculating a pixel shift amount of pixels that are to be combined by the reconstruction for a plurality of virtual imaging planes corresponding to the plurality of focus positions.

Abstract: An image generation method is provided for generating an output image from an input image acquired by an image sensor that has an array of multiple pixels, each of which has arranged therein multiple sub-pixels that each receive a light beam that passes through a different pupil sub-area of an imaging optical system. The method includes a step of generating multiple parallax images that respectively correspond to the different pupil sub-areas; a step of generating multiple pixel shifted images by performing different shifting for each of the parallax images according to a virtual image forming plane of the imaging optical system; and a step of generating an output image that has a higher resolution than the resolution of the parallax images from the pixel shifted images through composition processing.

Abstract: An image pickup apparatus includes an imaging optical system, an image pickup element including a plurality of pixels, a lens array configured such that rays from the same position on an object plane are incident on pixels of the image pickup element different from each other depending on a pupil region of the imaging optical system through which the ray passes, an image processing unit configured to perform image processing for the input image acquired by the image pickup element to generate the output images, and a control unit configured to drive the imaging optical system to perform focus control, the image processing unit is configured to acquire information on a refocus control range, and the control unit is configured to perform the focus control based on the information on the refocus control range.

Abstract: An image processing apparatus includes an acquirer which acquires a blurred image and a generator which acquires a first blur estimation area of at least a part of the blurred image to generate an estimated blur, and the generator determines a second blur estimation area based on a signal variation in the first blur estimation area, generates the estimated blur by performing iterative calculation processing that repeats correction processing and estimation processing, and determines the second blur estimation area having a signal variation larger than the signal variation in the first blur estimation area when the signal variation in the first blur estimation area is smaller than a first threshold value.

Abstract: An image processing apparatus includes a determiner which determines a characteristic of a blur to be estimated based on a blurred image, and a generator which acquires a blur estimation area of at least a part of the blurred image to generate an estimated blur based on the blur estimation area, and the generator repeats blur estimation processing and correction processing on information relating to a signal in the blur estimation area using the blur to generate the estimated blur, and changes, depending on the characteristic of the blur, at least one of acquisition processing of the blur estimation area or the blur estimation processing, or a parameter to be used for the acquisition processing or the blur estimation processing.

Abstract: An image processing apparatus includes an acquirer which acquires a blurred image, and a generator which acquires a blur estimation area of at least a part of the blurred image to generate an estimated blur based on the blur estimation area, and the generator generates the estimated blur by performing iterative calculation processing that repeats correction processing and estimation processing, the correction processing correcting a blur included in information relating to a signal in the blur estimation area to generate information relating to a correction signal, and the estimation processing estimating a blur based on the information relating to the signal and the information relating to the correction signal, and generates, as the information relating to the correction signal, information relating to a plurality of correction signals by using a plurality of different calculation expressions in at least one correction processing during the iterative calculation processing.

Abstract: An image processing apparatus generates an output image using an input image including object space information, the image processing apparatus includes a region dividing unit configured to divide the input image into a first region and a second region, and a reconstructed image generating unit configured to reconstruct the input image to generate the output image, and the reconstructed image generating unit applies different shooting conditions from each other to the first region and the second region respectively.

Abstract: An image processing method is capable of performing a reconstruction for an input image so as to generate a plurality of output images having a plurality of focus positions different from each other, and the image processing method includes the steps of obtaining the input image that is an image containing information of an object space obtained from a plurality of points of view using an image pickup apparatus having an imaging optical system and an image pickup element including a plurality of pixels, and calculating a pixel shift amount of pixels that are to be combined by the reconstruction for a plurality of virtual imaging planes corresponding to the plurality of focus positions.

Abstract: An image pickup apparatus includes an imaging optical system, an image pickup element including a plurality of pixels, a lens array configured such that rays from the same position on an object plane are incident on pixels of the image pickup element different from each other depending on a pupil region of the imaging optical system through which the ray passes, an image processing unit configured to perform image processing for the input image acquired by the image pickup element to generate the output images, and a control unit configured to drive the imaging optical system to perform focus control, the image processing unit is configured to acquire information on a refocus control range, and the control unit is configured to perform the focus control based on the information on the refocus control range.

Abstract: An image processing method is capable of performing a reconstruction for an input image so as to generate a plurality of output images having a plurality of focus positions different from each other, and the image processing method includes the steps of obtaining the input image that is an image containing information of an object space obtained from a plurality of points of view using an image pickup apparatus having an imaging optical system and an image pickup element including a plurality of pixels, and calculating a pixel shift amount of pixels that are to be combined by the reconstruction for a plurality of virtual imaging planes corresponding to the plurality of focus positions.

Abstract: The image capturing apparatus includes an image sensor including multiple pixels each including at least one photoelectric convertor and a transmittance distribution provider to provide, to at least part of the multiple pixels, a transmittance distribution depending on an incident angle of light. The multiple pixels includes an adjacent pixel group that includes a first pixel and second pixels adjacent to the first pixel, the adjacent pixel group including pixels to which mutually different four or more kinds of the transmittance distributions are provided. The apparatus produces, by using an input image produced from an output of the image sensor and information on the transmittance distributions, an output image corresponding to an optical observation image observable when an object space is observed from a partial area of a pupil of an imaging optical system.

Abstract: An image generation method is provided for generating an output image from an input image acquired by an image sensor that has an array of mnitple pixels, each of which has arranged therein multiple sub-pixels that each receive a light beam that passes through a different pupil sub-area of an imaging optical system. The method includes a step of generating multiple parallax images that respectively correspond to the different pupil sub-areas; a step of generating multiple pixel shifted images by performing different shifting for each of the parallax images according to a virtual image forming plane of the imaging optical system; and a step of generating an output image that has a higher resolution than the resolution of the parallax images from the pixel shifted images through composition processing.

Abstract: An image generation method is provided for generating an output image from an input image acquired by an image sensor that has an array of multiple pixels, each of which has arranged therein multiple sub-pixels that each receive a light beam that passes through a different pupil sub-area of an imaging optical system. The method includes a step of generating multiple parallax images that respectively correspond to the different pupil sub-areas; a step of generating multiple pixel shifted images by performing different shifting for each of the parallax images according to a virtual image forming plane of the imaging optical system; and a step of generating an output image that has a higher resolution than the resolution of the parallax images from the pixel shifted images through composition processing.

Abstract: The image pickup apparatus includes an imaging optical system to cause light rays from an object plane to image on an image side conjugate plane with respect to the object plane, an image sensor including pixels and to photoelectrically convert the optical image, and an optical element array including optical element cells each of which conjugates the image side conjugate plane with the image sensor and which are arranged so as to cause light rays from a same point on the object plane to enter mutually different pixels depending on positions on a pupil plane of the imaging optical system through which the light rays pass. An arrangement pitch of the optical element cells in the optical element array is 25 times or less of an arrangement pitch of the pixels in the image sensor.

Abstract: An image processing method includes the steps of acquiring a first image shot by using a compound eye image pickup apparatus, acquiring image capturing condition information of the first image, acquiring, depending on the image capturing condition information, optical characteristic information of a plurality of optical systems having a plurality of focal lengths different from each other in the compound eye image pickup apparatus, correcting a deterioration of the first image caused by shooting the first image based on the optical characteristic information, and generating a second image based on information of a position and an angle of a ray obtained from the first image whose deterioration is corrected, and the optical characteristic information contains at least one of information related to aberrations and information related to peripheral illumination of the optical systems.

Abstract: An image pickup apparatus (100) generates a second image based on a first image obtained via an image pickup optical system, and includes an image obtaining unit (101) configured to obtain the first image, a storage unit (106) configured to store a plurality of pieces of correction information calculated for each of groups divided based on information of a deteriorated image generated by applying each of a plurality of deterioration functions to a reference image and on the plurality of deterioration functions, and an image correcting unit (105) configured to generate the second image based on the first image by using correction information selected from among the plurality of pieces of correction information depending on information of the first image and the deterioration function based on shooting condition information.

Abstract: An image pickup apparatus includes an image sensor, and a controller configured to drive a focus unit so that a first position is focused, to acquire first information on a first refocusable range that is available in an image that is captured while the first position is being focused by the focus unit, to acquire second information on a second refocusable range that contains the first position, to acquire information on a second position on which the focus unit is to be focused so as to provide the second refocusable range using the first information and the second information, and make the image sensor capture an image while the second position is being focused by the focus unit.

Abstract: The present invention is directed to an image processing device including: a unit configured to extract, as target data, a set of signal values of pixels included in a target region including a target pixel in an input image; a unit configured to obtain reference data for each of a plurality of reference pixels corresponding to the target pixel; a unit configured to generate transformation reference data by transforming the reference data; a unit configured to determine a weight for each of a plurality of transformation reference pixels based on the target data and the reference data; and a unit configured to generate an output pixel corresponding to the target pixel by using a signal value calculated based on the transformation reference pixels and the weights.

Abstract: An image processing apparatus (105) provides reconstruction processing on an input image including parallax images of an object space at different viewpoints and to produce a reconstructed image, and includes a selector (105c) configured to select, from the input image, pixels that have information of an identical region of the object space, a calculator (105d) configured to calculate an average signal value based on signal values of the pixels thus selected, a substitutor (105b) configured to substitute signal values of at least one of the pixels with the average signal value, a producer (105e) configured to produce the reconstructed image from the input image including a pixel whose signal value is substituted with the average signal value, and a determiner (105h) configured to determine whether a first depth of field is smaller than a second depth of field.

Abstract: The image processing method includes extracting a range block image from an input image produced by image capturing, acquiring a degradation function representing degradation caused in the range block image by the image capturing, acquiring at least one first domain block image from the input image or another image, and applying the degradation function to the first domain block image to produce a second domain block image. The method further includes calculating a correlation between the second domain block image and the range block image, selecting from the at least one first domain block image, depending on the correlation, a corresponding domain block image that corresponds to the range block image, and producing an output image by using the corresponding domain block image.