Abstract: Shakiness of a panoramic video can be reduced by an image processing method comprising: setting a common coordinate system which can be transformed from individual coordinate systems of a plurality of image sensing devices; estimating postures of at least one of the plurality of image sensing devices; calculating an estimated posture of the common coordinate system using at least one of the estimated posture of the plurality of image sensing devices; calculating a correction transform for reducing a shakiness of the common coordinate system using the estimated posture of the common coordinate system; calculating a correction transform for reducing a shakiness of each of the plurality of image sensing devices using the correction transform; applying the corresponding correction transform to a sensed image which is sensed by each of the plurality of image sensing devices; and composing a panoramic image by joining a plurality of transformed sensed images.

Abstract: A view image rendering module (102) generates an image of a virtual space on the basis of a first viewpoint indicating the viewpoint of a user on the virtual space, and displays the generated image on a view image presentation module (101). A map image rendering module (104) generates a map image indicating a region around the position of the first viewpoint on the virtual space, and displays the generated map image on a map image presentation module (103). The position of the first viewpoint and the visual axis direction are controlled using a viewpoint position•visual axis direction input module (106) which is provided on the display surface of the map image presentation module (103). Furthermore, a virtual object set on the virtual space undergoes manipulation control using a virtual object manipulation input module (107) which is provided on the display surface of the map image presentation module (103).

Abstract: There are disclosed an image processing method and apparatus for generating a panoramic image in which the hue or brightness discontinuities generated at the seams of source images are reduced. According to the image processing method of this invention, upon compositing source images (221, 222) having an overlap portion, the hue or brightness of each source image is converted (H211, H221) using a reference image which has overlap portions with both the source images to be composited, and the converted images (241, 242) are composited, thereby reducing the hue or brightness discontinuities generated at the seam of images.

Abstract: An operation panel display device (304) presents the state of each virtual object to an observer on the basis of information about the state of each virtual object. An input device (303) or 3D pointing device (302) inputs an operation to change a desired one of the presented states. A computer (400) updates the information in accordance with the input operation.

Abstract: A view image rendering module (102) generates an image of a virtual space on the basis of a first viewpoint indicating the viewpoint of a user on the virtual space, and displays the generated image on a view image presentation module (101). A map image rendering module (104) generates a map image indicating a region around the position of the first viewpoint on the virtual space, and displays the generated map image on a map image presentation module (103). The position of the first viewpoint and the visual axis direction are controlled using a viewpoint position•visual axis direction input module (106) which is provided on the display surface of the map image presentation module (103). Furthermore, a virtual object set on the virtual space undergoes manipulation control using a virtual object manipulation input module (107) which is provided on the display surface of the map image presentation module (103).

Abstract: A shadow body generation unit 1711 generates a shadow model defined by a limited numbers of lines extending from a position of a light source to an outline of a hand. If the virtual object is partially or fully included in the shadow model and a position of the virtual object is further away from the position of the light source, the shadow body generation unit 1711 generates a shadow for partially obscured area included in the shadow model.

Abstract: There is provided an image processing apparatus for describing a real image as a virtual object using space data formed based on a photo image of the real object, and generating a virtual image of the virtual object using the space data. The image processing apparatus captures the photo image of the real object from a real camera viewpoint position while illuminating the real object by an illumination light source placed at a real illumination position. The captured photo image is converted into space data in the form of an object for each pixel, and the space data is stored in a memory together with various illumination conditions at the real illumination position to allow a search at a later time.

Abstract: There is provided an image processing apparatus for describing a real image as a virtual object using space data formed based on a photo image of the real object, and generating a virtual image of the virtual object using the space data. The image processing apparatus captures the photo image of the real object from a real camera viewpoint position while illuminating the real object by an illumination light source placed at a real illumination position. The captured photo image is converted into space data in the form of an object for each pixel, and the space data is stored in a memory together with various illumination conditions at the real illumination position to allow a search at a later time.

Abstract: There is provided an image processing apparatus for describing a real image as a virtual object using space data formed based on a photo image of the real object, and generating a virtual image of the virtual object using the space data. The image processing apparatus captures the photo image of the real object from a real camera viewpoint position while illuminating the real object by an illumination light source placed at a real illumination position. The captured photo image is converted into space data in the form of an object for each pixel, and the space data is stored in a memory together with various illumination conditions at the real illumination position to allow a search at a later time.

Abstract: In an image reproducing method capable of displaying annotations in virtual space constituted by a group of real panoramic images. If an object for which an annotation is to be displayed falls within the field of view at the position of the observer, the object is adopted as the display position of the annotation. If the object is not within the field of view, then the direction of forward travel of the observer is adopted as the annotation display position. If the object is in a state of transition, then the annotation display position is decided by interpolation.

Abstract: In a space rendering method for rendering a virtual space, ray space data of virtual objects in the virtual space are stored in an external database (29). When space data of a virtual object located within a predetermined range from the indicated position is not present in an internal memory (27), the space data is downloaded from the database (29) to the internal memory (27), and the virtual space is rendered on the basis of the downloaded space data.

Abstract: There are disclosed an image processing method and apparatus for generating a panoramic image in which the hue or brightness discontinuities generated at the seams of source images are reduced. According to the image processing method of this invention, upon compositing source images (221, 222) having an overlap portion, the hue or brightness of each source image is converted (H211, H221) using a reference image which has overlap portions with both the source images to be composited, and the converted images (241, 242) are composited, thereby reducing the hue or brightness discontinuities generated at the seam of images.

Abstract: Shakiness of a panoramic video can be reduced by an image processing method comprising: setting a common coordinate system which can be transformed from individual coordinate systems of a plurality of image sensing devices; estimating postures of at least one of the plurality of image sensing devices; calculating an estimated posture of the common coordinate system using at least one of the estimated posture of the plurality of image sensing devices; calculating a correction transform for reducing a shakiness of the common coordinate system using the estimated posture of the common coordinate system; calculating a correction transform for reducing a shakiness of each of the plurality of image sensing devices using the correction transform; applying the corresponding correction transform to a sensed image which is sensed by each of the plurality of image sensing devices; and composing a panoramic image by joining a plurality of transformed sensed images.

Abstract: By simplifying determination of annotation display positions, annotations can be synthesized to a large number of images and the annotation-synthesized images can be then displayed with less work and time. To achieve this, a viewpoint position and a sight line direction on a map are determined, and the annotation display position of an object is determined from the position of the object on the map determined based on observation directions of the object in plural panoramic images, the viewpoint position, and the sight line direction. Then, an annotation image is synthesized to the annotation display position on an actually taken image corresponding to the viewpoint position.

Abstract: There is disclosed a progressive display method upon rendering and displaying a virtual object expressed by RS data. When RS data is generated and recorded on the basis of real image data, map data that represents correspondence between a real image space and ray space is divided into a plurality of layers, and the resolutions of the individual layers are hierarchically changed, thus storing hierarchical map data. Upon rendering a virtual image from an arbitrary viewpoint position, rendering is done by looking up the hierarchical map data in turn in ascending order of resolution. Also, there are disclosed a downloading method of selectively downloading RS data and VRML data, and a rendering method of downloaded data. RS data with high resolution is stored in an external memory, and VRML data having a required memory size is stored in an internal memory. In principle, a virtual image is generated by rendering RS data to allow the user to experience the virtual space.

Abstract: There is disclosed an image data encoding/decoding method and apparatus which can efficiently encode multi-viewpoint image data obtained by sensing a single object from many viewpoints, and can obtain decoded data in units of pixels at high speed. Reference images which are not encoded are set every predetermined number of images (S701), and encoding is done using only the relationship with this reference image. An orthogonal transform is used in encoding, and an inverse transform formula for each pixel in a block is expanded and stored upon computing the orthogonal transform (S704). The coefficients after the orthogonal transform are fixed-length encoded in units of pixels (S705). Upon decoding, fixed-length encoded data corresponding to desired pixels are decoded, and pixel data are decoded by substituting coefficients obtained by decoding in the expansions generated upon encoding.

Abstract: An apparatus includes a camera (201) for sensing a first direction, a camera (202) for sensing a second direction, a mirror (221) for controlling the view of the camera (201) to a first view, and a mirror (222) for controlling the view of the camera (202) to a second view. The mirrors (221, 222) do not share ridge lines with each other, and the lens center of a virtual camera having the first view approximately matches that of a virtual camera having the second view.

Abstract: An image processing apparatus includes recording unit which has recorded image information appended with quantized space information in units of pixels, setting unit for setting a resolution higher than that of the recorded image information, reconstruction unit for reconstructing an image by reading out image information from the recording unit at the recorded resolution irrespective of the resolution set by the setting unit, and interpolation unit for making pixel interpolation of the reconstructed image with the resolution of the reconstructed image matching the resolution set by the setting unit.

Abstract: An image information processing apparatus is provided to generate, from previously held image data, a display image according to a viewpoint designated by a user, and to communicate with a control apparatus which can communicate with plural image information processing apparatuses respectively generating display images and has a controller for controlling viewpoint information shared by the plural image information processing apparatuses on the basis of viewpoint information sent from the image information processing apparatus. This apparatus comprises a transmitter to calculate the viewpoint information based on a user’s instruction and transmit it to the controller, a receiver to receive from the controller the viewpoint information for generating the display image, and an image generator to read the image data corresponding to the viewpoint information from the previously held image data for generating the display image, and generate and output the display image to a display.