Abstract: To produce both 2D image data and color parallax image data from image data output from a single-plate image sensor, the resolutions of the 2D image data and parallax image data may be both degraded. The image sensor has a primitive lattice that is a group of pixels including (i) at least four types of parallax pixels formed by photoelectric converter elements each of which is associated with one of combinations of two different types of aperture masks and two different types of color filters and (ii) no-parallax pixels configured to guide an incident luminous flux to photoelectric converter elements without limitation. In the group of pixels, the no-parallax pixels are more than the parallax pixels.

Abstract: Reception of the invisible light wavelength band in an incident luminous flux has not been considered. In view of this, an image sensor is provided, the image sensor including: a visible parallax pixel that is associated with a visible range light-receiving photoelectric conversion pixel having any of a plurality of types of aperture mask each including an aperture that is positioned to allow passage, to a photoelectric converting element, of a mutually different partial luminous flux of an incident luminous flux in a visible light wavelength band; and an invisible parallax pixel that is associated with an invisible range light-receiving photoelectric conversion pixel having any of a plurality of types of aperture mask each including an aperture positioned to allow passage, to the photoelectric converting element, of a mutually different partial luminous flux in an incident luminous flux in an invisible light wavelength band.

Abstract: A technique is disclosed to obtain first image data including a first pixel value of a first pixel having a no-parallax opening corresponding to a reference direction, a second pixel value of a second pixel having a parallax opening that causes parallax in one direction and a third pixel value of a third pixel having a parallax opening that causes parallax in the opposite direction, to calculate, for a pixel position of the second pixel, a fourth pixel value by averaging the second pixel value at the pixel position and a hypothetical third pixel value and to calculate, for a pixel position of the third pixel, a fifth pixel value by averaging the third pixel value at the pixel position and a hypothetical second pixel value, and to use the first pixel value, the fourth pixel value and the fifth pixel value to produce 2D second image data.

Abstract: To produce both 2D image data and color parallax image data from image data output from a single-plate image sensor, the resolutions of the 2D image data and parallax image data may be both degraded. The image sensor has a primitive lattice that is a group of pixels including (i) at least four types of parallax pixels formed by photoelectric converter elements each of which is associated with one of combinations of two different types of aperture masks and two different types of color filters and (ii) no-parallax pixels configured to guide an incident luminous flux to photoelectric converter elements without limitation. In the group of pixels, the no-parallax pixels are more than the parallax pixels.

Abstract: An image processing device including means for: inputting two parallax images composed of images of first and second viewpoint directions; generating first and second smoothed images by applying, respectively to the images of the first and second viewpoint directions, a smoothing filter in a direction joining the first and second viewpoints; generating a reference image formed by a value based on a product of values of the first and second smoothed images; generating gain distribution data by dividing a value of the reference image by the value of the first smoothed image; performing gain correction of each pixel on the image of the first viewpoint direction, based on the gain distribution data; and outputting the gain corrected image of the first viewpoint direction as a parallax image, the smoothing filter having a filter diameter approximately equal to a parallax amount between the images of the first and second viewpoint directions.

Abstract: To produce both 2D image data and color parallax image data from image data output from a single-plate image sensor, the resolutions of the 2D image data and parallax image data may be both degraded. The image sensor has a primitive lattice that is a group of pixels including (i) at least four types of parallax pixels formed by photoelectric converter elements each of which is associated with one of combinations of two different types of aperture masks and two different types of color filters and (ii) no-parallax pixels configured to guide an incident luminous flux to photoelectric converter elements without limitation. In the group of pixels, the no-parallax pixels are more than the parallax pixels.

Abstract: An image processing apparatus, comprising: a receiving unit receiving a first image and image-capturing information associated with the first image; a determining unit determining, based on the image-capturing information, whether or not the first image is an image of a subject image in a view point direction corresponding to the aperture mask, using an output signal of an image sensor that includes a pixel array including first parallax pixels having the aperture mask with asymmetric directional properties in the long axis and short axis with respect to incident light flux of a single optical system and positioned to permit a portion of the light flux to pass; and an outputting unit outputting a corrected first image by carrying out a correction process having different frequency properties between the long axis direction and the short axis direction on the subject image in an unfocused region of the first image.

Abstract: An image processing apparatus is provided, that is configured to: extract a first pixel value corresponding to a first viewpoint that is one of a plurality of viewpoints to capture a subject image, at a target pixel position from image data having the first pixel value; extract second and third luminance values corresponding to second and third viewpoints that are different from the first viewpoint, at the target pixel position from luminance image data having the second and third luminance values; and calculate at least any of second and third pixel values of the second and third viewpoints such that a relational expression between the second or third pixel value and the first pixel value extracted by the pixel value extracting unit remains correlated with a relational expression defined by the second and third luminance values.

Abstract: An image sensor captures a subject image of a partial luminous flux passed through different regions from among a total luminous flux through one optical system. The image sensor is configured from a pixel arrangement in which a plurality of each of at least three types of pixels are arranged, including non-parallax pixels which comprise an aperture mask that produces a viewpoint in a reference direction, first parallax pixels which comprise an aperture mask that produces a viewpoint in a first direction different from the reference direction and second parallax pixels which comprise an aperture mask that produces a viewpoint in a second direction different from the reference direction, wherein each of the aperture mask of the first parallax pixels and the aperture mask of the second parallax pixels has a width of a greater-than-half-aperture aperture area in the direction of change of the viewpoint.

Abstract: A technique is disclosed to calculate a new pixel value of a particular color of second-viewpoint color image data in such a manner that a difference between the new pixel value of the particular color of the second viewpoint and the pixel value of the particular color of the first-viewpoint color image data maintains correlation with a sum of a difference between the pixel value of a first color component of the second-viewpoint color image data and the pixel value of the first color component of third-viewpoint color image data and a difference between the pixel value of a second color component of the second-viewpoint color image data and the pixel value of the second color component of the third-viewpoint color image data.

Abstract: An image processing apparatus includes a receiving unit configured to receive at least two parallax images that are obtained from a subject image captured via a single optical system, where the at least two parallax images include an image in a first viewpoint direction and an image in a second viewpoint direction, an average calculating unit configured to calculate, for each pixel, an arithmetic average and a geometric average between the image in the first viewpoint direction and the image in the second viewpoint direction, a ratio calculating unit configured to calculate, for each pixel, a ratio of the arithmetic average to the geometric average, and a disparity calculating unit configured to calculate, on a pixel-by-pixel basis, a disparity between the image in the first viewpoint direction and the image in the second viewpoint direction based on the ratio.

Abstract: An imaging apparatus including an image generating section that generates first parallax image data based on the output of the first pixel and second parallax image data based on the output of the second pixel. When the imaging element captures an image of an object point located in an unfocused region on the optical axis, a pixel value of a center pixel corresponding to the optical axis in the first parallax image data is greater than or equal to 50% of a pixel value of a peak pixel in the first parallax image data, and a pixel value of a center pixel corresponding to the optical axis in the second parallax image data is greater than or equal to 50% of a pixel value of a peak pixel in the second parallax image data.

Abstract: Reception of the invisible light wavelength band in an incident luminous flux has not been considered. In view of this, an image sensor is provided, the image sensor including: a visible parallax pixel that is associated with a visible range light-receiving photoelectric conversion pixel having any of a plurality of types of aperture mask each including an aperture that is positioned to allow passage, to a photoelectric converting element, of a mutually different partial luminous flux of an incident luminous flux in a visible light wavelength band; and an invisible parallax pixel that is associated with an invisible range light-receiving photoelectric conversion pixel having any of a plurality of types of aperture mask each including an aperture positioned to allow passage, to the photoelectric converting element, of a mutually different partial luminous flux in an incident luminous flux in an invisible light wavelength band.

Abstract: The present invention includes the steps of: generating a temporary left parallax image using the pixel value of the left parallax pixels of the first image for each pixel; generating a temporary right parallax image using the pixel value of the right parallax pixels of the first image for each pixel; generating a standard view point image using the pixel values of at least the non-parallax pixels of the first image for each pixel; generating an edge enhanced standard view point image, by carrying out an edge enhancement process on the standard view point image; and generating a left direction view point image and a right direction view point image based on the edge enhanced standard view point image and the temporary left parallax image and the temporary right parallax image for each pixel.

Abstract: An image sensor captures a subject image of a partial luminous flux passed through different regions from among a total luminous flux through one optical system. The image sensor is configured from a pixel arrangement in which a plurality of each of at least three types of pixels are arranged, including non-parallax pixels which comprise an aperture mask that produces a viewpoint in a reference direction, first parallax pixels which comprise an aperture mask that produces a viewpoint in a first direction different from the reference direction and second parallax pixels which comprise an aperture mask that produces a viewpoint in a second direction different from the reference direction, wherein each of the aperture mask of the first parallax pixels and the aperture mask of the second parallax pixels has a width of a greater-than-half-aperture aperture area in the direction of change of the viewpoint.

Abstract: An imaging apparatus including an image generating section that generates first parallax image data based on the output of the first pixel and second parallax image data based on the output of the second pixel. When the imaging element captures an image of an object point located in an unfocused region on the optical axis, a pixel value of a center pixel corresponding to the optical axis in the first parallax image data is greater than or equal to 50% of a pixel value of a peak pixel in the first parallax image data, and a pixel value of a center pixel corresponding to the optical axis in the second parallax image data is greater than or equal to 50% of a pixel value of a peak pixel in the second parallax image data.

Abstract: An image processing device including means for: inputting two parallax images composed of images of first and second viewpoint directions; generating first and second smoothed images by applying, respectively to the images of the first and second viewpoint directions, a smoothing filter in a direction joining the first and second viewpoints; generating a reference image formed by a value based on a product of values of the first and second smoothed images; generating gain distribution data by dividing a value of the reference image by the value of the first smoothed image; performing gain correction of each pixel on the image of the first viewpoint direction, based on the gain distribution data; and outputting the gain corrected image of the first viewpoint direction as a parallax image, the smoothing filter having a filter diameter approximately equal to a parallax amount between the images of the first and second viewpoint directions.

Abstract: The present invention includes the steps of: generating a temporary left parallax image using the pixel value of the left parallax pixels of the first image for each pixel; generating a temporary right parallax image using the pixel value of the right parallax pixels of the first image for each pixel; generating a standard view point image using the pixel values of at least the non-parallax pixels of the first image for each pixel; generating an edge enhanced standard view point image, by carrying out an edge enhancement process on the standard view point image; and generating a left direction view point image and a right direction view point image based on the edge enhanced standard view point image and the temporary left parallax image and the temporary right parallax image for each pixel.

Abstract: An image processing apparatus, comprising: a receiving unit receiving a first image and image-capturing information associated with the first image; a determining unit determining, based on the image-capturing information, whether or not the first image is an image of a subject image in a view point direction corresponding to the aperture mask, using an output signal of an image sensor that includes a pixel array including first parallax pixels having the aperture mask with asymmetric directional properties in the long axis and short axis with respect to incident light flux of a single optical system and positioned to permit a portion of the light flux to pass; and an outputting unit outputting a corrected first image by carrying out a correction process having different frequency properties between the long axis direction and the short axis direction on the subject image in an unfocused region of the first image.

Abstract: An image processing apparatus is provided, that is configured to: extract a first pixel value corresponding to a first viewpoint that is one of a plurality of viewpoints to capture a subject image, at a target pixel position from image data having the first pixel value; extract second and third luminance values corresponding to second and third viewpoints that are different from the first viewpoint, at the target pixel position from luminance image data having the second and third luminance values; and calculate at least any of second and third pixel values of the second and third viewpoints such that a relational expression between the second or third pixel value and the first pixel value extracted by the pixel value extracting unit remains correlated with a relational expression defined by the second and third luminance values.