Abstract: An orientation sensor measures the orientation of the orientation sensor itself on a sensor coordinate system, and outputs an orientation measurement value. A first alignment data setting unit reads out first alignment data, which is recorded in an external storage device and represents the gravitational direction on a reference coordinate system, and sets that data in an orientation calculation unit. A second alignment data setting unit sets second alignment data, which represents a difference angle in the azimuth direction between the sensor coordinate system and reference coordinate system in the orientation calculation unit in accordance with a command input by an operator from a command input unit. The orientation calculation unit calculates the orientation of an object on the reference coordinate system using the first and second alignment data on the basis of the orientation measurement value input from the orientation sensor, and outputs it.

Abstract: A display apparatus for displaying a virtual image superposed upon a real world image, which can automatically set the focus of a lens to the virtual image, a virtual world image object having a high importance degree or a real world image object having a high importance degree. The display apparatus has a lens for capturing light from a real world, a lens state control unit for controlling a state of the lens including a focus, a storage unit for storing computer image data, a computer image generation unit for generating a computer image by using the computer image data stored in the storage unit in accordance with lens state information of the lens state and the like; and a synthesis display unit for synthesizing an image captured via the lens and the generated computer image and displaying a synthetic image.

Abstract: A mixed reality presenting apparatus for merging an incident real-space image and a generated virtual-space image comprises a measuring section which measures the orientation of a unit for entering the real-space image and the speed or acceleration thereof; a virtual-space image generating section which generates virtual-space image data on the basis of the measurement result; a correcting optical system which corrects the path of light passing therethrough; and a correcting optical system control section which controls the correcting optical system on the basis of the measured speed or acceleration.

Abstract: An image display apparatus calculating correction information to correct measured values of an orientation sensor includes an orientation sensor that senses orientation of a viewpoint and outputs measured values, a correction unit for correcting the measured values by using correction information, and a generation unit for generating a virtual image based on the corrected measured values. A display unit displays the generated virtual image, and a calculation unit calculates the corrected information. The calculation unit obtains the measured values of the sensor at the point of when the virtual image is displayed at an appropriate position, generates the correction information from the measured values and predetermined orientation information, and generates the virtual image based on orientation information relating to the viewpoint obtained using the measured values of the orientation sensor and the correction information.

Abstract: A first error coordinate between the image coordinate of a first indicator, which is arranged on the real space and detected on a first image captured by a first image sensing unit, and the estimated image coordinate of the first indicator, which is estimated to be located on the first image in accordance with the position/orientation relationship between the first image sensing unit (with the position and orientation according to a previously calculated position/orientation parameter) and the first indicator, is calculated.

Abstract: An information processing method includes steps of: acquiring arrangement information indicating the position or the position and orientation of an index arranged in a physical space; capturing an image in the physical space; detecting the index from the captured image; and calculating the position and/or orientation of a viewpoint of the captured image based on the arrangement information and the detected index.

Abstract: In a case where a position and/or orientation of a shooting viewpoint is calculated by using information about image coordinates of markers placed in a scene, the present invention enables a user to easily determine positions of the markers so that the position and orientation can be calculated more accurately. Information about markers placed in a physical space is obtained and area information about mixing accuracy between a physical space image and a virtual space image is obtained based on the information about the markers, so that a virtual space image is generated in accordance with the obtained area information.

Abstract: To correct the field angle of an image sensing device which senses an image of a real space that includes a feature point whose position on the real space is known, the position of the feature point (real marker position) in the image of the real space sensed by the device is detected. Then, the feature point position (virtual marker position), which is seen when the real space is viewed from a virtual viewpoint specified by the measured position, orientation, and field angle of the device, in that image is calculated. A correction amount of the position and/or a correction amount of the orientation of the device are/is calculated so as to minimize a deviation between the real and virtual marker positions in the image. The correction amount of the field angle of the device is calculated in accordance with the minimized deviation or the correction amount of the position.

Abstract: A display apparatus for displaying a virtual image superposed upon a real world image, which can automatically set the focus of a lens to the virtual image, a virtual world image object having a high importance degree or a real world image object having a high importance degree. The display apparatus has a lens for capturing light from a real world, a lens state control unit for controlling a state of the lens including a focus, a storage unit for storing computer image data, a computer image generation unit for generating a computer image by using the computer image data stored in the storage unit in accordance with lens state information of the lens state and the like; and a synthesis display unit for synthesizing an image captured via the lens and the generated computer image and displaying a synthetic image.

Abstract: An information-processing system for calibrating an orientation sensor mounted on an image-pickup device calculates a relative orientation between the orientation sensor and the image-pickup device by using information about image coordinates of indices detected from a photographed image and an orientation-measurement value. At the same time, the information-processing system obtains parameters for minimizing an error between actual measured values of the image coordinates of the indices and theoretical values of the image coordinates of the indices, where the theoretical values are calculated from estimates of unknown parameters, by using a position of the image-pickup device and the relative orientation between the orientation sensor and the image-pickup device, as the unknown parameters.

Abstract: An information processing device for obtaining placement information regarding an imaging device of markers to be calibrated that are provided upon the imaging device, includes: a first image obtaining unit adapted to obtain a first image photographed with the imaging device; a second image obtaining unit adapted to obtain a second image obtained by photographing the imaging device from a bird's-eye view position; a first detecting unit adapted to detect information relating to image coordinates of reference markers placed in a scene, from the first image; a second detecting unit adapted to detect information relating to image coordinates of markers to be calibrated from the second image; and a calibration information calculating unit adapted to obtain the calibration information using information relating to image coordinates of the reference markers detected by the first detecting unit and the second detecting unit.

Abstract: A display apparatus for displaying a virtual image superposed upon a real world image, which can automatically set the focus of a lens to the virtual image, a virtual world image object having a high importance degree or a real world image object having a high importance degree. The display apparatus has a lens for capturing light from a real world, a lens state control unit for controlling a state of the lens including a focus, a storage unit for storing computer image data, a computer image generation unit for generating a computer image by using the computer image data stored in the storage unit in accordance with lens state information of the lens state and the like; and a synthesis display unit for synthesizing an image captured via the lens and the generated computer image and displaying a synthetic image.

Abstract: In order to correct the field angle of an image sensing device which senses an image of a real space that includes a feature point whose position on the real space is known, the position of the feature point (real marker position) in the image of the real space sensed by the image sensing device is detected. Then, the position of the feature point (virtual marker position), which is seen when the real space is viewed from a virtual viewpoint specified by the measured position, orientation, and field angle of the image sensing device, in that image is calculated. A correction amount of the position and/or a correction amount of the orientation of the image sensing device are/is calculated so as to minimize a deviation between the real and virtual marker positions in the image. The correction amount of the field angle of the image sensing device is calculated in accordance with the minimized deviation or the correction amount of the position.

Abstract: An orientation sensor (130) measures the orientation of the orientation sensor (130) itself on a sensor coordinate system, and outputs an orientation measurement value. A first alignment data setting unit (410) reads out first alignment data, which is recorded in an external storage device (not shown) and represents the gravitational direction on a reference coordinate system, and sets that data in an orientation calculation unit (940). A second alignment data setting unit (420) sets second alignment data, which represents a difference angle in the azimuth direction between the sensor coordinate system and reference coordinate system in the orientation calculation unit (940) in accordance with a command input by an operator from a command input unit (not shown).

Abstract: A view transformation matrix that represents the position/attitude of an HMD is generated based on a signal that represents the position/attitude of the HMD (S602). On the other hand, landmarks and their locations are detected based on a captured picture (S604) and a calibration matrix ?Mc is generated using the detected locations of the landmarks (S605). The position/attitude of the HMD is calibrated using the view transformation matrix and calibration matrix ?Mc generated by the above processes (S606), a picture of a virtual object is generated based on external parameters that represent the position/attitude of the calibrated HMD, and a mixed reality picture is generated (S607). The generated mixed reality picture is displayed in the display section (S609).

Abstract: A first error coordinate between the image coordinate of a first indicator, which is arranged on the real space and detected on a first image captured by a first image sensing unit, and the estimated image coordinate of the first indicator, which is estimated to be located on the first image in accordance with the position/orientation relationship between the first image sensing unit (with the position and orientation according to a previously calculated position/orientation parameter) and the first indicator, is calculated.

Abstract: There is disclosed a protection apparatus which can prevent illicit use of contents without any real objects required for the contents, which are included in those for a mixed reality system. A protection apparatus has ID transmission units which are provided in correspondence with real objects required for contents, and transmit unique IDs, an ID reception unit which receives IDs from the ID transmission units, and an ID collation unit for determining whether an execution of the contents is to be authorized or not on the basis of whether or not the received IDs correspond to real objects required to execute the contents. When a main system that provides mixed reality to the user is to execute contents, the protection apparatus determines that execution of the contents is to be authorized or not and sends the determination result to the main system.

Abstract: A mixed reality presentation apparatus and control method thereof which can allow for suitable display of a virtual object depending on tone, brightness, and lighting conditions of a physical space on which the virtual object is superimposed and displayed is disclosed. The mixed reality presentation apparatus according to the present invention includes: a display setting adjustor 106 which adjusts the display colors and position of virtual information and the lighting settings based on the tone of the physical space which the observer sees and it determines the display colors such that a large tone difference from the physical space image is achieved for virtual text information and virtual wire frame information as well as the lighting settings used for drawing a three-dimensional virtual object with a tone such that it is merged with an object existing in the physical space.

Abstract: A mixed reality presenting apparatus for merging an incident real-space image and a generated virtual-space image comprises a measuring section which measures the orientation of a unit for entering the real-space image and the speed or acceleration thereof; a virtual-space image generating section which generates virtual-space image data on the basis of the measurement result; a correcting optical system which corrects the path of light passing therethrough; and a correcting optical system control section which controls the correcting optical system on the basis of the measured speed or acceleration.

Abstract: A system displays an agent character, which does not give any anxious feeling to the user and is easy to use, by taking into consideration the position/posture of the user. Upon displaying the agent character, the distance from the user and the line-of-sight direction of the user are taken into consideration, and the agent character is inhibited from being located at the closest distance position in front of the user. When the user does not communicate with the agent character, the agent character is located in a far zone in front of the user. When the user communicates with the agent character, the agent character is located in a middle-distance zone in front of the user.