Abstract: A method for encoding a plurality of input images is provided. The method includes the steps of generating a first residual image from a difference between a first input image and a corresponding first predicted image, converting, into a remainder, a pixel value included in a region of the first residual image which should be defined by a remainder, generating a second residual image from a difference between a second input image and a corresponding second predicted image, converting, into a remainder, the pixel value included in a region of the second residual image which should be defined by a remainder, and coding the first residual image after the conversion, the second residual image after the conversion, and additional information specifying the region which should be defined by a remainder in each of the first residual image and the second residual image.

Abstract: A method for encoding a plurality of input images containing mutually related information is provided. The method includes the steps of estimating a motion image indicating a change component from one or more previous input images, contained in a subsequent input image, generating a residual image from a difference between the subsequent input image and the estimated motion image, specifying a region whose pixel value should be defined by a remainder among pixels constituting the residual image, based on the pixel value of the residual image, converting, into a remainder, the pixel value for the specified region which should be defined by a remainder, and encoding the residual image after the conversion and additional information specifying the region which should be defined by a remainder.

Abstract: Provided is a method for reducing data size of images containing mutually similar information. The present method includes the steps of: generating a synthesized image corresponding to a target image based on a first reference image and a second reference image; generating a mask image indicating a potential error of the synthesized image with respect to the target image from the first reference image and the second reference image; generating a residual image in accordance with a difference between the target image and the synthesized image; generating a converted image corresponding to the target image by assigning, based on the mask image, a region where the potential error is relatively large, information on a corresponding region of the residual image; and outputting the first reference image, the second reference image and the converted image as information representing the target image, the first reference image and the second reference image.

Abstract: Provided is a method including: generating a synthesized image corresponding to a target image based on a first reference image and a second reference image; generating side information which is information on a virtual view at a location of the target image, based on at least one of the target image and the synthesized image; generating a gradient image based on the side information; determining a factor in accordance with a gradient for each pixel location of the gradient image, and performing a modulo operation using, as a modulus, a factor corresponding to an intensity value at each pixel location of the target image, to generate a remainder image composed of remainders of respective pixel locations calculated by the modulo operation; and outputting the first reference image, the second reference image and the remainder image as information representing the target image, the first reference image and the second reference image.

Abstract: Provided is a virtual viewpoint image synthesizing method in which a virtual viewpoint image viewed from a virtual viewpoint is synthesized based on image information obtained from a plurality of viewpoints. The virtual viewpoint image is synthesized through a reference images obtaining step, a depth maps generating step, an up-sampling step, a virtual viewpoint information obtaining step, and a virtual viewpoint image synthesizing step.

Abstract: The present invention relate to an image generating apparatus for generating an image from a viewpoint specified by a user. According to the invention, the apparatus has a storage unit that stores data of a plurality of images, and a disparity map generating unit that generates a disparity map. The disparity map is associated with a pair of images including a first image and a second image, and indicates corresponding pixels in the first and second images.

Abstract: The present invention relates to an image generating apparatus for generating an image from a viewpoint specified by a user. According to the invention, the apparatus has a storage unit that stores a plurality of pre-images from viewpoints on predetermined lines and depth maps indicating a depth value of each pixel of a corresponding pre-image, and an image generating unit that generates a first image from a first viewpoint based on the pre-images and depth maps.

Abstract: The present invention relates to an image processing apparatus for compressing image data used in an image generating apparatus for generating a free-viewpoint image. According to the invention, the apparatus has a selecting unit that selects one image as a first image, and defines other images as second images, a projective transformation unit that generates a projected depth map of a second image from a depth map of the first image, a subtracting unit that creates a difference map of the second image, and a storage unit that stores the depth map of the first image and the difference map of the second image. Here, the difference map is a difference between a depth map of the second image and the projected depth map of the second image, and the depth map indicates a depth value of each pixel of a corresponding image.

Abstract: The present invention relates to color correction of video signals from a plurality of cameras. The apparatus has a correction order that defines a plurality of camera pairs and an order of the camera pairs, where each camera pair specifies a reference camera and a target camera. In the correction order, each camera is treated as a target camera at least one time, and the reference camera is treated as the target camera in a previous camera pair except for the first camera pair. The apparatus corrects a value of each pixel in a target image captured by the target camera based on a reference image captured by the reference camera, and replaces the target image by the corrected target image, for each camera pair in accordance with the order of the camera pairs.

Abstract: A present invention provides a method and a program for extracting the high accuracy silhouette by relatively simple process not using manual labor or special photography environment. A method for extracting the high accuracy silhouette comprises: extracting a number of first silhouettes from a number of object images and a number of background images by a background subtraction; constructing first visual hull from a number of the first silhouettes by a shape from silhouette method; constructing second visual hull by process to repair missed parts and/or to remove unwanted regions in first visual hull; and extracting a number of second silhouettes from the second visual hull.

Abstract: The present invention relates to an apparatus and a method for classifying pixels in each frame of a motion picture as foreground or background. According to the invention, the apparatus has a decision unit and adjustment unit. The decision unit classifies a pixel as a foreground pixel, a background pixel or an undefined pixel based on a first threshold, a second threshold and a distribution value of the pixel representing an occurrence probability of a pixel value of the pixel. The decision unit sorts each pixel into a foreground pixel or a background pixel based on the classification result of the decision unit.

Abstract: Provided is a virtual viewpoint image synthesizing method in which a virtual viewpoint image viewed from a virtual viewpoint is synthesized based on image information obtained from a plurality of viewpoints. The virtual viewpoint image is synthesized through a reference images obtaining step, a depth maps generating step, an up-sampling step, a virtual viewpoint information obtaining step, and a virtual viewpoint image synthesizing step.

Abstract: The present invention relates to calibration of camera parameters for converting a world coordinate system, which indicates a position in the real space, to a coordinate used in an image and vice versa. The apparatus according to the invention has a detection unit, which determines corresponding pixel pairs from the captured image and the model image and outputs corresponding data indicating determined pixel pairs, and a selection unit, which selects pixel pairs to be left in the corresponding data and removes data related to an unselected pixel pair from the corresponding data for generating selected corresponding data. The apparatus further has a calculation unit, which calculates camera parameters based on the selected corresponding data.

Abstract: The present invention relates to an image processing apparatus for compressing image data used in an image generating apparatus for generating a free-viewpoint image. According to the invention, the apparatus has a selecting unit that selects one image as a first image, and defines other images as second images, a projective transformation unit that generates a projected depth map of a second image from a depth map of the first image, a subtracting unit that creates a difference map of the second image, and a storage unit that stores the depth map of the first image and the difference map of the second image. Here, the difference map is a difference between a depth map of the second image and the projected depth map of the second image, and the depth map indicates a depth value of each pixel of a corresponding image.

Abstract: The present invention relates to an image generating apparatus for generating an image from a viewpoint specified by a user. According to the invention, the apparatus has a storage unit that stores a plurality of pre-images from viewpoints on predetermined lines and depth maps indicating a depth value of each pixel of a corresponding pre-image, and an image generating unit that generates a first image from a first viewpoint based on the pre-images and depth maps.

Abstract: A present invention provides a method and a program for extracting the high accuracy silhouette by relatively simple process not using manual labor or special photography environment. A method for extracting the high accuracy silhouette comprises: extracting a number of first silhouettes from a number of object images and a number of background images by a background subtraction; constructing first visual hull from a number of the first silhouettes by a shape from silhouette method; constructing second visual hull by process to repair missed parts and/or to remove unwanted regions in first visual hull; and extracting a number of second silhouettes from the second visual hull.

Abstract: The present invention relate to an image generating apparatus for generating an image from a viewpoint specified by a user. According to the invention, the apparatus has a storage unit that stores data of a plurality of images, and a disparity map generating unit that generates a disparity map. The disparity map is associated with a pair of images including a first image and a second image, and indicates corresponding pixels in the first and second images.

Abstract: The present invention relates to calibration of camera parameters for converting a world coordinate system, which indicates a position in the real space, to a coordinate used in an image and vice versa. The apparatus according to the invention has a detection unit, which determines corresponding pixel pairs from the captured image and the model image and outputs corresponding data indicating determined pixel pairs, and a selection unit, which selects pixel pairs to be left in the corresponding data and removes data related to an unselected pixel pair from the corresponding data for generating selected corresponding data. The apparatus further has a calculation unit, which calculates camera parameters based on the selected corresponding data.

Abstract: The present invention relates to an apparatus and a method for classifying pixels in each frame of a motion picture as foreground or background. According to the invention, the apparatus has a decision unit and adjustment unit. The decision unit classifies a pixel as a foreground pixel, a background pixel or an undefined pixel based on a first threshold, a second threshold and a distribution value of the pixel representing an occurrence probability of a pixel value of the pixel. The decision unit sorts each pixel into a foreground pixel or a background pixel based on the classification result of the decision unit.

Abstract: The present invention relates to color correction of video signals from a plurality of cameras. The apparatus has a correction order that defines a plurality of camera pairs and an order of the camera pairs, where each camera pair specifies a reference camera and a target camera. In the correction order, each camera is treated as a target camera at least one time, and the reference camera is treated as the target camera in a previous camera pair except for the first camera pair. The apparatus corrects a value of each pixel in a target image captured by the target camera based on a reference image captured by the reference camera, and replaces the target image by the corrected target image, for each camera pair in accordance with the order of the camera pairs.