Abstract: The present invention relates to camera arrangements with backlighting detection. The camera arrangements are capable of simultaneously capturing real scene data from various viewpoints. This data may include illumination data impinging the scene. The illumination data may then be utilized to alter the apparent illumination of a second image, either real or virtual, which is to be superimposed over the real scene so that the illumination across the entire superimposed scene is consistent. The camera arrangements may utilize combinations of umbilical cables and light tubes to expand or contract the field of capture. The camera arrangements may also include in-line signal processing of the data output.

Abstract: In a telepresence system, a scene is captured by recording pixel data elements, each associated with a pixel ray vector having a direction and an intercept on an known locus in the frame of reference of the scene. Each pixel data element includes data representing the illumination along the pixel ray vector. For example, the pixel data elements may be captured by operating numerous video cameras pointing in different directions on a spherical locus. A virtual viewpoint image representing the image which would be seen from an arbitrary viewpoint, looking in an arbitrary direction, can be synthesized by determining the directions of synthetic pixel ray vectors from each pixel of the virtual viewpoint image through the virtual viewpoint and the intercepts of these vectors on the locus. Recorded pixel data elements having pixel ray vector directions and intercepts close to those of the synthetic pixel ray vector can be copied or interpolated to provide data representing illumination in the synthetic pixel.