Abstract: An image processing apparatus generates an output image using an input image including object space information, the image processing apparatus includes a region dividing unit configured to divide the input image into a first region and a second region, and a reconstructed image generating unit configured to reconstruct the input image to generate the output image, and the reconstructed image generating unit applies different shooting conditions from each other to the first region and the second region respectively.

Abstract: An image processing method is capable of performing a reconstruction for an input image so as to generate a plurality of output images having a plurality of focus positions different from each other, and the image processing method includes the steps of obtaining the input image that is an image containing information of an object space obtained from a plurality of points of view using an image pickup apparatus having an imaging optical system and an image pickup element including a plurality of pixels, and calculating a pixel shift amount of pixels that are to be combined by the reconstruction for a plurality of virtual imaging planes corresponding to the plurality of focus positions.

Abstract: An image processing apparatus acquires an input image generated by image pickup via an optical system, and perform unsharp mask processing for the input image using a filter generated based on information of a PSF of the optical system. The filter is generated based on an unsharp mask used for the unsharp mask processing, and includes two-dimensional data having filter coefficients that are arranged rotationally asymmetrically with respect to a filter coefficient corresponding to a target pixel in the input image in convoluting the filter with the input image. A peak position or a center of gravity position of the unsharp mask accords with a position of the filter coefficient corresponding to the target pixel in the filter.

Abstract: An image pickup apparatus includes an imaging optical system, an image pickup element including a plurality of pixels, a lens array configured such that rays from the same position on an object plane are incident on pixels of the image pickup element different from each other depending on a pupil region of the imaging optical system through which the ray passes, an image processing unit configured to perform image processing for the input image acquired by the image pickup element to generate the output images, and a control unit configured to drive the imaging optical system to perform focus control, the image processing unit is configured to acquire information on a refocus control range, and the control unit is configured to perform the focus control based on the information on the refocus control range.

Abstract: An image processing method is capable of performing a reconstruction for an input image so as to generate a plurality of output images having a plurality of focus positions different from each other, and the image processing method includes the steps of obtaining the input image that is an image containing information of an object space obtained from a plurality of points of view using an image pickup apparatus having an imaging optical system and an image pickup element including a plurality of pixels, and calculating a pixel shift amount of pixels that are to be combined by the reconstruction for a plurality of virtual imaging planes corresponding to the plurality of focus positions.

Abstract: An image processing apparatus (100) includes a detection unit (102) configured to detect a first region in which one of a signal level of a color plane corresponding to at least one color filter of an image acquired from an image pickup element including a plurality of color filters and a signal level of a brightness plane generated from the image monotonically increases or monotonically decreases in a first direction in a predetermined section, a determination unit (103) configured to determine a second region in which color bleeding occurs, based on optical information on the color bleeding of an image pickup optical system and information on the first region, and a correction unit (104) configured to correct the image so as to reduce the color bleeding.

Abstract: An image pickup apparatus includes an image pickup element which obtains an original image, a storage unit which stores a first optical transfer function different depending on an image pickup condition and on a position in the original image and an image restoration filter or a function designed to generate a second optical transfer function, a data obtaining unit which obtains the first optical transfer function when the aperture value is less than a predetermined value, and the image restoration filter or the second optical transfer function generated based on the function designed to generate the second optical transfer function when the aperture value is not less than the predetermined value, a filter generating unit which generates an image restoration filter based on the first or second optical transfer function, and an image processing unit which generates the restored image by using the image restoration filter.

Abstract: An image generation method is provided for generating an output image from an input image acquired by an image sensor that has an array of mnitple pixels, each of which has arranged therein multiple sub-pixels that each receive a light beam that passes through a different pupil sub-area of an imaging optical system. The method includes a step of generating multiple parallax images that respectively correspond to the different pupil sub-areas; a step of generating multiple pixel shifted images by performing different shifting for each of the parallax images according to a virtual image forming plane of the imaging optical system; and a step of generating an output image that has a higher resolution than the resolution of the parallax images from the pixel shifted images through composition processing.

Abstract: An image generation method is provided for generating an output image from an input image acquired by an image sensor that has an array of multiple pixels, each of which has arranged therein multiple sub-pixels that each receive a light beam that passes through a different pupil sub-area of an imaging optical system. The method includes a step of generating multiple parallax images that respectively correspond to the different pupil sub-areas; a step of generating multiple pixel shifted images by performing different shifting for each of the parallax images according to a virtual image forming plane of the imaging optical system; and a step of generating an output image that has a higher resolution than the resolution of the parallax images from the pixel shifted images through composition processing.

Abstract: The image pickup apparatus includes an imaging optical system to cause light rays from an object plane to image on an image side conjugate plane with respect to the object plane, an image sensor including pixels and to photoelectrically convert the optical image, and an optical element array including optical element cells each of which conjugates the image side conjugate plane with the image sensor and which are arranged so as to cause light rays from a same point on the object plane to enter mutually different pixels depending on positions on a pupil plane of the imaging optical system through which the light rays pass. An arrangement pitch of the optical element cells in the optical element array is 25 times or less of an arrangement pitch of the pixels in the image sensor.

Abstract: The image processing apparatus performs an image process on a first image produced by image capturing. The apparatus includes: a memory storing coefficient data containing coefficients to be used in an approximation function to approximate a PSF of an image capturing optical system, and a processor performs the image process. The processor produces an approximated point spread function by using the coefficient of the coefficient data corresponding to an optical state of the optical system in the image capturing and two of a pixel pitch of an image sensor, a kernel size of a filter to be used for a sharpening process on the first image and number of taps of the filter; produce the filter by using the approximated PSF; and produce a second image that is a sharpened image by applying the filter to the first image.

Abstract: A non-transitory computer-readable storage medium storing an image processing program that causes a computer to execute an image processing method comprising acquiring step of acquiring an input image generated by image pickup through an optical system, generating step of generating a point spread function by using coefficient data of a function approximating a point spread function corresponding to an image-pickup condition of the optical system, and providing step of providing unsharp mask processing to the input image using a filter generated based on information of the point spread function generated by using the coefficient data. The filter is a filter having two-dimensional data.

Abstract: An image processing method includes the steps of acquiring a first image shot by using a compound eye image pickup apparatus, acquiring image capturing condition information of the first image, acquiring, depending on the image capturing condition information, optical characteristic information of a plurality of optical systems having a plurality of focal lengths different from each other in the compound eye image pickup apparatus, correcting a deterioration of the first image caused by shooting the first image based on the optical characteristic information, and generating a second image based on information of a position and an angle of a ray obtained from the first image whose deterioration is corrected, and the optical characteristic information contains at least one of information related to aberrations and information related to peripheral illumination of the optical systems.

Abstract: An image pickup apparatus includes an image pickup element which obtains an original image, a storage unit which stores a first optical transfer function different depending on an image pickup condition and on a position in the original image and an image restoration filter or a function designed to generate a second optical transfer function, a data obtaining unit which obtains the first optical transfer function when the aperture value is less than a predetermined value, and the image restoration filter or the second optical transfer function generated based on the function designed to generate the second optical transfer function when the aperture value is not less than the predetermined value, a filter generating unit which generates an image restoration filter based on the first or second optical transfer function, and an image processing unit which generates the restored image by using the image restoration filter.

Abstract: The image processing method of producing, based on information on positions and angles of light rays obtained from a first image, a second image, the first image being produced by using an image pickup apparatus that introduces the light rays from an image capturing optical system (101) to an image sensor (102) through an optical element array (101c). The method includes a step of acquiring the first image and information on an image capturing condition at a time of producing the first image, a step of acquiring an image restoration filter corresponding to the information on the image capturing condition and corresponding to respective optical transfer functions of the image capturing optical system and the optical element array, a step of performing an image restoration process on the first image by using the image restoration filter, and a step of producing the second image by using the restored first image.

Abstract: An image pickup apparatus includes an image sensor, and a controller configured to drive a focus unit so that a first position is focused, to acquire first information on a first refocusable range that is available in an image that is captured while the first position is being focused by the focus unit, to acquire second information on a second refocusable range that contains the first position, to acquire information on a second position on which the focus unit is to be focused so as to provide the second refocusable range using the first information and the second information, and make the image sensor capture an image while the second position is being focused by the focus unit.

Abstract: An image pickup apparatus includes an image pickup element which obtains an original image, a storage unit which stores a first optical transfer function different depending on an image pickup condition and on a position in the original image and an image restoration filter or a function designed to generate a second optical transfer function, a data obtaining unit which obtains the first optical transfer function when the aperture value is less than a predetermined value, and the image restoration filter or the second optical transfer function generated based on the function designed to generate the second optical transfer function when the aperture value is not less than the predetermined value, a filter generating unit which generates an image restoration filter based on the first or second optical transfer function, and an image processing unit which generates the restored image by using the image restoration filter.

Abstract: An image processing apparatus (105) provides reconstruction processing on an input image including parallax images of an object space at different viewpoints and to produce a reconstructed image, and includes a selector (105c) configured to select, from the input image, pixels that have information of an identical region of the object space, a calculator (105d) configured to calculate an average signal value based on signal values of the pixels thus selected, a substitutor (105b) configured to substitute signal values of at least one of the pixels with the average signal value, a producer (105e) configured to produce the reconstructed image from the input image including a pixel whose signal value is substituted with the average signal value, and a determiner (105h) configured to determine whether a first depth of field is smaller than a second depth of field.

Abstract: The image processing apparatus includes a memory configured to store information on a specific image pickup condition under which a specific blur component is generated in an input image produced by image capturing using an optical system, the specific blur component being caused by the optical system; an image pickup condition acquirer configured to acquire information on an actual image pickup condition when capturing the input image; and a processor configured to perform, when the actual image pickup condition coincides with the specific image pickup condition, a blur detection process to detect the specific blur component included in the input image.

Abstract: An image processing apparatus includes an image obtaining portion 200 that obtains a first and second images having exposures different from each other, a region selecting portion 203 that selects a first region different from a region containing a luminance saturation part from the first image, and that selects a second region corresponding to the region containing the luminance saturation part of the first image from the second image, an image restoration processing portion 201 that performs an image restoration processing using an image restoration filter generated based on an optical transfer function for the first region of the first image and the second region of the second image, and an image replacing portion 202 that replaces the region containing the luminance saturation part of the first image with the second region of the second image so as to synthesize the first and second regions to generate one image.