Method and system for controlling space magnification for stereoscopic images
The invention relates to a method and system of displaying stereoscopic images. The method comprises displaying at least one stereoscopic image on a set of display device, the stereoscopic image comprising a pair of two-dimensional plane images, and providing adjustment data for spatial magnification, the spatial magnification relating to a size of a space that is imaged. The method also comprises transmitting the magnification adjustment data to a set of remotely located stereoscopic cameras, and adjusting the distance between the stereoscopic cameras based on the magnification adjustment data.
This application is a continuation application, and claims the benefit under 35 U.S.C. §120 of application Ser. No. 10/280,344 filed on Oct. 24, 2002, which is hereby incorporated by reference herein. U.S. application Ser. No. 10/280,344 is a continuation application of PCT application No. PCT/KR01/01398 filed on Aug. 17, 2001 and published on Feb. 21, 2002, which is hereby incorporated by reference herein. This application is related to, and hereby incorporates by reference, the following patent applications:
U.S. patent application entitled “METHOD AND SYSTEM FOR CALCULATING A PHOTOGRAPHING RATIO OF A CAMERA”, filed on even date herewith and having application Ser. No. 10/280,239 (Attorney Docket No. GRANP2.001C1 (GEORA.010C1));
U.S. patent application entitled “METHOD AND SYSTEM FOR CONTROLLING A SCREEN RATIO BASED ON A PHOTOGRAPHING RATIO”, filed on even date herewith and having application Ser. No. 10/280,246 (Attorney Docket No. GRANP2.001C2 (GEORA.010C2));
U.S. patent application entitled “METHOD AND SYSTEM FOR CONTROLLING THE DISPLAY LOCATION OF STEREOSCOPIC IMAGES”, filed on even date herewith and having application Ser. No. 10/280,241 (Attorney Docket No. GRANP2.001C3 (GEORA.010C3));
U.S. patent application entitled “METHOD AND SYSTEM FOR PROVIDING THE MOTION INFORMATION OF STEREOSCOPIC CAMERAS”, filed on even date herewith and having application Ser. No. 10/280,436 (Attorney Docket No. GRANP2.001C4 (GEORA.010C4));
U.S. patent application entitled “METHOD AND SYSTEM FOR CONTROLLING THE MOTION OF STEREOSCOPIC CAMERAS BASED ON A VIEWER'S EYE MOTION”, filed on even date herewith and having application Ser. No. 10,280,251 (Attorney Docket No. GRANP2.001C5 (GEORA.010C5));
U.S. patent application entitled “METHOD AND SYSTEM OF STEREOSCOPIC IMAGE DISPLAY FOR GUIDING A VIEWER'S EYE MOTION USING A THREE-DIMENSIONAL MOUSE”, filed on even date herewith and having application Ser. No. 10/280,465 (Attorney Docket No. GRANP2.001C6(GEORA.010C6));
U.S. patent application entitled “METHOD AND SYSTEM FOR CONTROLLING THE MOTION OF STEREOSCOPIC CAMERAS USING A THREE-DIMENSIONAL MOUSE”, filed on even date herewith and having application Ser. No. 10/280,419 (Attorney Docket No. GRANP2.001C7 (GEORA.010C7));
U.S. patent application entitled “METHOD AND SYSTEM FOR ADJUSTING DISPLAY ANGLES OF A STEREOSCOPIC IMAGE BASED ON A CAMERA LOCATION ”, filed on even date herewith and having application Ser. No. 10/280,248 (Attorney Docket No. GRANP2.001C9 (GEORA.010C9));
U.S. patent application entitled “METHOD AND SYSTEM FOR TRANSMITTING OR STORING STEREOSCOPIC IMAGES AND PHOTOGRAPHING RATIOS FOR THE IMAGES”, filed on even date herewith and having application Ser. No. 10/280,464 (Attorney Docket No. GRANP2.001C10 (GEORA.010C10)); and
U.S. patent application entitled “PORTABLE COMMUNICATION DEVICE FOR STEREOSCOPIC IMAGE DISPLAY AND TRANSMISSION”, filed on even date herewith and having application Ser. No. 10/280,179 (Attorney Docket No. GRANP2.001C11 (GEORA.010C11)).
BACKGROUND OF THE INVENTION1. Field of the Invention
The present invention relates to a method and system for generating and/or displaying a more realistic stereoscopic image. Specifically, the present invention relates to a method and system for adjusting a size of a space, which a viewer perceives, that is imaged by a set of stereoscopic cameras, by adjusting the distance between a set of stereoscopic cameras.
2. Description of the Related Technology
In general, a human being can recognize an object by sensing the environment through eyes. Also, as the two eyes are spaced apart a predetermined distance from each other, the object perceived by the two eyes is initially sensed as two images, each image being formed by one of the left or right eyes. The object is recognized by the human brain as the two images are partially overlapped. Here, in the portion where the images perceived by a human being overlap, as the two different images transmitted from the left and right eyes are synthesized in the brain, there is a perception of 3-dimensions.
By using the above principle, various conventional 3-D image generating and reproducing systems using cameras and displays have been developed.
As one example of the systems, U.S. Pat. No. 4,729,017 discloses “Stereoscopic display method and apparatus therefor.” With a relatively simple construction, the apparatus allows a viewer to view a stereoscopic image via the naked eye.
As another example of the systems, U.S. Pat. No. 5,978,143 discloses “Stereoscopic recording and display system.” The patent discloses that the stereoscopically shown image content is easily controllable by the observer within the scene, which is recorded by the stereo camera.
As another example of the systems, U.S. Pat. No. 6,005,607 discloses “Stereoscopic computer graphics image generating apparatus and stereoscopic TV apparatus.” This apparatus stereoscopically displays two-dimensional images generated from three-dimensional structural information.
SUMMARY OF CERTAIN INVENTIVE ASPECTS OF THE INVENTIONOne aspect of the invention provides a method of displaying stereoscopic images. The method comprises displaying at least one stereoscopic image on a set of display device, the stereoscopic image comprising a pair of two-dimensional plane images, and providing adjustment data for spatial magnification, the spatial magnification relating to a size of a space that is imaged. The method also comprises transmitting the magnification adjustment data to a set of remotely located stereoscopic cameras, and adjusting the distance between the stereoscopic cameras based on the magnification adjustment data.
Another aspect of the invention provides a system for displaying stereoscopic images. The system comprises a set of display device, an input device, a transmitter, a transmitter, a receiver, a set of stereoscopic cameras and a camera controller. The set of display device displays at least one stereoscopic image, the stereoscopic image comprising a pair of two-dimensional plane images. The input device provides adjustment data for spatial magnification, the spatial magnification relating to a size of a space that is imaged. The transmitter transmits the magnification adjustment data. The receiver receives the magnification adjustment data. The camera controller adjusts the distance between the stereoscopic cameras based on the magnification adjustment data.
Another aspect of the invention provides a camera system for communicating data with a set of display device. The system comprises a set of stereoscopic cameras, a receiver and a camera controller. The set of stereoscopic cameras produce at least one stereoscopic image. The receiver receives adjustment data for spatial magnification from the set of display device, the space magnification relating to a size of the space that is imaged by the set of stereoscopic cameras. The camera controller adjusts the distance between the set of stereoscopic cameras based on the spatial magnification adjustment data.
Still another aspect of the invention provides a method of controlling a set of stereoscopic cameras. The method comprises providing a set of stereoscopic cameras, and generating at least one stereoscopic image, the stereoscopic image comprising a pair of two-dimensional plane images generated by the set of stereoscopic cameras, respectively. The method also comprises receiving adjustment data for spatial magnification, the spatial magnification relating to a size of the space that is imaged by the set of stereoscopic cameras, and adjusting the distance between the stereoscopic cameras based on the spatial magnification adjustment data.
Yet another aspect of the invention provides a method of displaying stereoscopic images. The method comprises providing a pair of projection portions spaced at a predetermined distance apart from each other and configured to project three-dimensional structural data of a scene into a pair of two-dimensional planes. The method also comprises producing at least one stereoscopic image from the three-dimensional structural data by the pair of projection portions spaced at a predetermined distance apart from each other, the stereoscopic image comprising a pair of two-dimensional plane images. The method also comprises displaying the pair of two-dimensional plane images in a pair of display devices, respectively, and providing adjustment data for space magnification to the display devices, the space magnification relating to a size of the scene. The method comprises adjusting the distance between the projection portions based on the magnification adjustment data.
Yet another aspect of the invention provides a system for displaying stereoscopic images. The system comprises a pair of projection portions, a pair of display devices, an input device, and a computing device. The pair of projection portions are spaced at a predetermined distance apart from each other and produce at least one stereoscopic image from three-dimensional structural data of a scene, the stereoscopic image comprising a pair of two-dimensional plane images. The pair of display devices display the pair of two-dimensional plane images, respectively. The input device provides adjustment data for spatial magnification, the spatial magnification relating to a size of the scene. The computing device adjusts the distance between the projection portions based on the magnification adjustment data.
BRIEF DESCRIPTION OF THE DRAWINGS
FIGS. 70 illustrates a 3D display system for conforming the resolution between the stereoscopic cameras and display devices.
DETAILED DESCRIPTION OF THE CERTAIN EMBODIMENTS OF THE INVENTION
Here, according to the distance between the two cameras and the object to be photographed by the cameras, and the size of the photographed object, the size of the displayed image is determined. Also, as the distance between the left and right images displayed on the display device has the same ratio as the distance between a viewer's two eyes, the viewer feels a sense of viewing the actual object in 3-dimensions.
In the above technology, an object may be photographed by a camera while the object moves, the camera moves, or a magnifying (zoom-in) or reducing (zoom-out) imaging function is performed with respect to the object, not being in a state in which a fixed object is photographed by a fixed camera. In those situations, the distance between the camera and the photographed object, or the size of the photographed object changes. Thus, a viewer may perceive the image having a sense of distance different than is the actual distance from the camera to the object.
Also, even when the distance between the object and the stereoscopic cameras is fixed during photographing, each viewer has their own unique eye distance, a biometric which is measured as the distance between the center points of the viewer's eyes. For example, the distance between an adult's eyes is quite different from that of a child's eyes. Also the eye distance varies between viewers of the same age. In the meantime, in current 3D display systems, the distance between the center points of each stereoscopic image is fixed at the distance value of the average adult (i.e., 70 mm) as exemplified in
In order to display a realistic 3D image, one aspect of the invention is to adjust display images or display devices such that a screen ratio (D:E:F) in the display device is substantially the same as a photographing ratio (A:B:C) in the camera. Hereinafter, the term 3D images and stereoscopic images will be used to convey the same meaning. Also, a stereoscopic image comprises a pair of two-dimensional plane images produced by a pair of stereoscopic cameras. Stereoscopic images comprise a plurality of stereoscopic images.
Photographing Ratio (A:B:C) and Screen Ratio (D:E:F)
For the purpose of the explanation, assume that the object 22 is located “10 m” away from the camera 20 and is photographed such that the object 22 is included in a single film or an image frame as shown in
As shown in
Since the location of the object lens is normally fixed, in order to change the distance from d1 to d4, the location of the film ranges from Ps to Pl as much as “d” according to the focal length. The focus adjusting portion of the camera 20 adjusts the location of the film from Ps to Pe. The focus adjusting of the camera 20 may be manually performed or may be automatically made.
The focus adjusting portion 52 adjusts the focus of the object lens of the camera 20. The focus adjusting portion 52 may perform its function either manually or automatically. In one embodiment of the invention, the focus adjusting portion 52 may comprise 10 scales between “0.3 m and infinity,” and is located in one of the scales while the camera 20 is photographing an object. In one embodiment of the invention, the focus adjusting portion 52 may use a known focus adjusting portion that is used in a typical camera.
The FAP location detection portion 54 detects the current scale location of the focus adjusting portion 52 among the scales. In one embodiment of the invention, the FAP location detection portion 54 may comprise a known position detection sensor that detects the scale value in which the focus adjusting portion 52 is located. In another embodiment of the invention, since the variation of the scale location is proportional to the distance between the object lens and film as shown in
The memory 56 stores data representing maximum and minimum photographing ratios of the camera 20. In one embodiment of the invention, the memory 56 comprise a ROM, a flash memory or a programmable ROM. This may apply to all of the other memories described throughout the specification.
The photographing ratio calculation portion 58 calculates a photographing ratio (A:B:C) based on the detected scale location and the maximum and minimum photographing ratios. In one embodiment of the invention, the photographing ratio calculation portion 58 comprises a digital signal processor (DSP) calculating the ratio (A:B:C) using the following Equations I and II.
In Equations I and II, parameters Amax and Bmax represent horizontal and vertical length values (A and B) of the maximum photographing ratio, respectively, exemplified as “3” and “2” in
In another embodiment of the invention, the photographing ratio calculation portion 58 calculates a photographing ratio (A:B:C) such that the ratio falls between the maximum and minimum photographing ratios and at the same time is proportional to the value of the detected scale location. Thus, as long as the ratio falls between the maximum and minimum photographing ratios and is proportional to the value of the detected scale location, any other equation may be used for calculating the photographing ratio.
Referring to
Maximum and minimum photographing ratios are provided from the memory 56 to the photographing ratio calculation portion 58 (604). In one embodiment of the invention, the photographing ratio calculation portion 58 may store the maximum and minimum photographing ratios therein. In this situation, the memory 56 may be omitted from the apparatus.
The FAP location detection portion 54 detects the current location of the focus adjusting portion 52 while the camera 20 is photographing the object (606). While the camera is photographing the object, the focal length may be changed. The detected current location of the focus adjusting portion 52 is provided to the photographing ratio calculation portion 58.
The photographing ratio calculation portion 58 calculates a horizontal value (A) of a current photographing ratio from Equation I (608). It is assumed that the detected current location value is “5” among the total scale values “10.” Using Equation I and the table of
The photographing ratio calculation portion 58 calculates a vertical value (B) of a current photographing ratio from Equation II (610). In the above example, using Equation II and the table of
The photographing ratio calculation portion 58 retrieves parameter C from the maximum and minimum ratios that have been used for calculating parameters A and B (612). Referring to the table of
The iris 62 is a device that adjusts an amount of light coming into the camera 20 according to the degree of its opening. When the degree of the opening of the iris 62 is largest, the maximum amount of light shines on the film of the camera 20. This largest opening corresponds to the longest focal length and the maximum photographing ratio. In contrast, when the degree of the opening of the iris 62 is smallest, the least amount of light comes into the camera 20. This smallest opening corresponds to the shortest focal length and the minimum photographing ratio. In one embodiment of the invention, the iris 62 may be a known iris that is used in a typical camera.
The iris opening detection portion 64 detects the degree of the opening of the iris 62. The degree of the opening of the iris 62 may be quantitized to a range of, for example, 1-10. Degree “10” may mean the largest opening of the iris 62 and degree “1” may mean the smallest opening of the iris 62. The memory 66 stores data representing maximum and minimum photographing ratios of the camera 20.
The photographing ratio calculation portion 68 calculates a photographing ratio (A:B:C) based on the detected degree of the opening and the maximum and minimum photographing ratios. In one embodiment of the invention, the photographing ratio calculation portion 68 comprises a digital signal processor (DSP) calculating the ratio (A:B:C) using the following Equations III and IV.
In Equations III and IV, parameters Amax and Bmax, Amin and Bmin, and “c” are the same as the parameters used in Equations I and II. Parameters Icur and Ilargest represent the detected current degree of the opening and the largest degree of the opening, respectively.
Referring to
The iris opening detection portion 64 detects the current degree of the opening of the iris 62 while the camera 20 is photographing the object (706). The detected degree of the opening of the iris 62 is provided to the photographing ratio calculation portion 68.
The photographing ratio calculation portion 68 calculates a horizontal value (A) of a current photographing ratio from Equation III (708). It is assumed that the detected current opening degree is 2 among the total degree values 10. Using Equation III and
The photographing ratio calculation portion 68 calculates a vertical value (B) of a current photographing ratio from Equation IV (710). In the above example, using equation IV and
The photographing ratio calculation portion 68 retrieves parameter C from the maximum and minimum ratios that have been used for calculating parameters A and B (712). Referring to
In one embodiment of the invention, the camera 20 comprises an analog camera and a digital camera. When the camera 20 photographs an object, the image data processing apparatus 70 performs a typical image processing of the photographed image according to the control of the microcomputer 72. In one embodiment of the invention, the image data processing apparatus 70 may comprise a digitizer that digitizes the photographed analog image into digital values, a memory that stores the digitized data, and a digital signal processor (DSP) that performs an image data processing of the digitized image data (all not shown). The image data processing apparatus 70 provides the processed data to a data combiner 76.
In one embodiment, the photographing ratio calculation apparatus 74 comprises the apparatus shown in
The microcomputer 72 controls the image data processing apparatus 70, the photographing ratio calculation apparatus 74, and the data combiner 76 such that the camera 20 outputs the combined data 78. In one embodiment of the invention, the microcomputer 72 controls the image data processing apparatus 70 such that the apparatus properly processes the digital image data. In this embodiment of the invention, the microcomputer 72 controls the photographing ratio calculation apparatus 74 to calculate a photographing ratio for the image being photographed. In this embodiment of the invention, the microcomputer 72 controls the data combiner 76 to combine the processed data and the photographing ratio data corresponding to the processed data. In one embodiment of the invention, the microcomputer 72 may provide a synchronization signal to the data combiner 76 so as to synchronize the image data and the ratio data. As discussed above, as long as the current scale location of the focus adjusting portion or the opening degree of the iris is not changed, the photographing ratio is not changed. The microcomputer 72 may detect the change of the scale location or the opening degree, and control the data combiner 76 such that the image data and the corresponding ratio data are properly combined.
In one embodiment of the invention, the microcomputer 72 is programmed to perform the above function using typical microcomputer products, available from the Intel, IBM and Motorola companies, etc. This product may also apply to the other microcomputers described throughout this specification.
The data combiner 76 combines the image data from the image data processing apparatus 70 and the calculated photographing ratio (A:B:C) data according to the control of the microcomputer 72. The combiner 76 outputs the combined data 78 in which the image data and the ratio data may be synchronized with each other. In one embodiment of the invention, the combiner 76 comprises a known multiplexer.
Method and System for Controlling a Screen Ratio Based on a Photographing Ratio
The embodiment of camera site 80 shown in
In one embodiment of the invention, the photographing ratios “A1:B1:C” and “A2:B2:C2” are substantially the same. In one embodiment of the invention, the data 802 and 804 may be combined and transmitted to the display site 82. In one embodiment of the invention, the photographing ratio may have a standard data format in each of the camera and display sites so that the display site can identify the photographing ratio easily.
The display site 82 comprises a set of receivers 820, 832, a set of display devices 86, 88. Each receiver 820, 832 receives the combined data transmitted from the camera site 80 and provides each data set to the display devices 86, 88, respectively. In one embodiment of the invention, each of the receivers 820, 832 is separate from the display devices 86, 88. In another embodiment of the invention, receivers 820, 832 may be embedded into each display device 86, 88.
The display devices 86 and 88 comprise data separators 822 and 834, image size adjusting portions 828 and 840, and display screens 830 and 842. The data separators 822 and 834 separate the photographing ratio data (24, 838) and the image data (826, 836) from the received data. In one embodiment of the invention, each of the data separators 822 and 834 comprises a typical demultiplexer.
The image size adjusting portion 828 adjusts the size of the image to be displayed in the display screen 830 based on the photographing ratio (A1:B1:C1), and screen-viewer distance (F1) and display screen size values (G1, H1). The screen-viewer distance (F1) represents the distance between the display screen 830 and one of a viewer's eyes, e.g., a left eye, that is directed to the screen 830. In one embodiment of the invention, the distance F1 may be fixed. In this situation, a viewer's eyes may be located in a eye fixing structure, which will be described in more detail later. Also, the image size adjusting portion 828 may store the fixed value F1 therein. The screen size values GI and HI represent the horizontal and vertical dimensions of the screen 830, respectively. In one embodiment of the invention, the size values G1 and H1 may be stored in the image size adjusting portion 828.
The image size adjusting portion 840 adjusts the size of the image to be displayed in the display screen 842 based on the photographing ratio (A2:B2:C2), and screen-viewer distance (F2) and display screen size values (G2, H2). The screen-viewer distance (F2) represents the distance between the display screen 842 and one of a viewer's eyes, e.g., a right eye, that is directed to the screen 842. In one embodiment of the invention, the distance F2 may be fixed. In one embodiment of the invention, the screen-viewer distance (F2) is substantially the same as the screen-viewer distance (F1). The screen size values G2 and H2 represent the horizontal and vertical dimensions of the screen 842, respectively. In one embodiment of the invention, the display screen size values G2 and H2 are substantially the same as the display screen size values G1 and H1.
The operation of the image size adjusting portions 828 and 840 will be described in more detail by referring to
The image data 826, the photographing ratio data (A1:B1:C1) and the screen-viewer distance (F1) are provided to the image size adjusting portion 828 (902). A screen ratio (D1:E1:F1) is calculated based on the photographing ratio (A1:B1:C1) and the screen-viewer distance (F1) using the following Equation V (904). Since the value F1 is already provided, the parameters D1 and E1 of the screen ratio are obtained from Equation V.
The horizontal and vertical screen size values (G1, H1) of the display screen 830 are provided to the image size adjusting portion 828 (906). In one embodiment of the invention, the screen size values G1 and H1, and the distance value F1 are fixed and stored in the image size adjusting portion 828. In another embodiment of the invention, the screen size values G1 and H1, and the distance value F1 are manually provided to the image size adjusting portion 828.
Image magnification (reduction) ratios d and e are calculated from the following Equation VI (908). The ratios d and e represent horizontal and vertical magnification (reduction) ratios for the display screens 830 and 842, respectively.
This is to perform magnification or reduction of the provided image 826 with regard to the screen sizes (G1, H1). If the calculated value “D1” is greater than the horizontal screen size value (G1), the provided image needs to be magnified as much as “d.” If the calculated value “D1” is less than the horizontal screen size value (G1), the provided image needs to be reduced as much as “d.” The same applies to the calculated value “E1.” This magnification or reduction enables a viewer to recognize the image at the same ratio that the camera 110 photographed the object. The combination of the display devices 86 and 88 provides a viewer with a more realistic three-dimensional image.
It is determined whether the magnification (reduction) ratios (d, e) are greater than “1” (910). If both of the ratios (d, e) are greater than 1, the image data 826 are magnified as much as “d” and “e,” respectively, as shown in
If both of the ratios “d” and “e” are not greater than 1, it is determined whether the magnification (reduction) ratios (d, e) are less than “1” (916). If both of the ratios d and e are less than 1, the image data 826 are reduced as much as “d” and “e,” respectively, as shown in
If both of the ratios d and e are equal to 1, no adjustment of the image size is made (922). In this situation, since the magnification (reduction) ratio is 1, no magnification or reduction of the image is made as shown in
Now referring to
The image data and the photographing ratio that are calculated for the image are combined for each of the stereoscopic cameras 110 and 120 (1160). The combined data are illustrated as reference numerals 802 and 804 in
Each of the receivers 820 and 832 receives the transmitted data from the camera site 80 (1200). The photographing ratio and image data are separated from the combined data (1220). Alternatively to 1200 and 1220, the image data and photographing ratio are separately received as they are not combined in transmission. In one embodiment of the invention, the combined data may not include a photographing ratio. In that circumstance, the photographing ratio that has been received most recently is used for calculating the screen ratio. In one embodiment of the invention, the screen ratio may remain unchanged until the new photographing ratio is received.
The screen ratios (D1:E1:F1) and (D2:E2:F2) for each of the display devices 86 and 88 are calculated using the method described with regard to
Traditional 3D display systems display images without considering the value Wa. This means that the distance value (Wd) is the same for all viewers regardless of the fact that they have a different Wa value. These traditional systems caused several undesirable problems such as headache or dizziness of the viewer, and deterioration of a sense of three dimension. In order to produce a more realistic three-dimensional image and to reduce headaches or dizziness of a viewer, the distance Wd needs to be determined by considering the distance Wa. The consideration of the Wa value may provide a viewer with better and more realistic three-dimensional images. In one embodiment of the invention, the distance Wd is adjusted such that the distance Wd is substantially the same as Wa.
The system shown in
The display devices 1260 and 1280 comprise interfaces 1510 and 1550, microcomputers 1520 and 1560, display drivers 1530 and 1570, and display screens 1540 and 1580, respectively. In one embodiment of the invention, each of the display screens 1540 and 1580 comprises a LCD screen, a CRT screen, or a PDP screen. The interfaces 1510 and 1550 provide the interface between the input devices 1400 and 1500 and the microcomputers 1520 and 1560, respectively. In one embodiment of the invention, each of the interfaces 1510 and 1550 comprises a typical input device controller and/or a typical interface module (not shown).
There may be several methods to measure and provide the distance (Wa). As one example, an optometrist may measure the Wa value of a viewer with eye examination equipment. In this situation, the viewer may input the value (Wa) via the input devices 1400, 1500. As another example, an eye lens motion detector may be used in measuring the Wa value. In this situation, the Wa value may be provided from the detector to either the input devices 1400, 1500 or the interfaces 1510, 1550 in
As another example, as shown in
Each of the microcomputers 1520 and 1560 determines an amount of movement for the displayed images based on the provided Wa value such that the Wd value is substantially the same as the Wa value. In one embodiment of the invention, each microcomputer (1520, 1560) initializes the distance value Wd and determines an amount of movement for the displayed images based on the value Wa and the initialized value Wd. Each of the display drivers 1530 and 1570 moves the displayed images based on the determined movement amount and displays the moved images on each of the display screens 1540 and 1580, In one embodiment of the invention, each microcomputer (1520, 1560) may incorporate the function of each of the display drivers 1530 and 1570. In that situation, the display drivers 1530 and 1570 may be omitted.
Referring to
It is then determined whether Wa equals Wd (1640). If Wa equals Wd, no movement of the displayed images is made (1680). In this situation, since the distance (Wa) between the center points of the viewer's eye is the same as the distance (Wd) between the center points of the displayed images, no adjustment of the displayed images is made.
If Wa does not equal Wd, it is determined whether Wa is greater than Wd (1650). If Wa is greater than Wd, the distance (Wd) needs to be increased until Wd equals Wa. In this situation, the left image 1750 displayed in the left screen 1540 is moved to the left side and the right image 1760 displayed in the right screen 1580 is moved to the right side until the two values are substantially the same as shown in
With regard to the HMD system shown in
If it is determined that Wa is not greater than Wd, the distance Wd needs to be reduced until Wd equals Wa. Thus, the left image 1770 displayed in the display device 1260 is moved to the right side and the right image 1780 displayed in the display device 1280 is moved to the left side until the two values are substantially the same as shown in
The microcomputer 1820 determines an amount of the movement for the display devices 1840 and 1845 based on the provided value Wa such that the Wd value is substantially the same as the Wa value. In one embodiment of the invention, the microcomputer 1820 initializes the distance value (Wd) and determines an amount of the movement for the display devices 1840 and 1845 based on the value Wa and the initialized value Wd. Each of the servo mechanisms 1830 and 1835 moves the display devices 1840 and 1845, respectively, based on the determined movement amount.
Referring to
In another embodiment of the invention, the distance (Va) is automatically detected using a known eye lens motion detector. This embodiment of the invention will be described referring to
The detected distance and location values are provided to a microcomputer 2120. The microcomputer 2120 receives the distance value Va and determines an amount of movement for the displayed images or an amount of movement for the display devices similarly as described with regard to
The motion detection devices 2220 and 2230 detect the motion of the cameras 2200 and 2210, respectively. The motion of the cameras 2200 and 2210 may comprise motions for upper and lower directions, and left and right directions as shown in
The combiners 2240 and 2250 combine the image data and the motion detection data, and provide the combined data 2260 and 2270 to the transmitters 2280 and 2290, respectively. If the combiners 2240 and 2250 receive information data representing no motion detection from the motion detection devices 2220 and 2230, or if the combiners 2240 and 2250 do not receive any motion data, each combiner (2240, 2250) provides only the image data to the transmitters 2280 and 2290 without motion detection data. In one embodiment of the invention, each of the combiners 2240 and 2250 comprises a typical multiplexer. Each of the transmitters 2280 and 2290 transmits the combined data 2260 and 2270 to the display site through a communication network (not shown).
Each of the receivers 2300 and 2310 receives the combined data transmitted from the camera system, and provides the received data to the data separators 2320 and 2330, respectively. Each of the data separators 2320 and 2330 separates the image data and the motion detection data from the received data. The image data are provided to the image processors 2340 and 2360. The motion detection data are provided to the microcomputers 2350 and 2370. The image processors 2340 and 2360 perform typical image data processing for the image data, and provide the processed data to the combiners 2380 and 2400, respectively.
Each of the microcomputers 2350 and 2370 determines camera motion information from the motion detection data. In one embodiment of the invention, each microcomputer (2350, 2370) determines camera motion information for at least four directions, e.g., upper, lower, left, right. The microcomputers 2350 and 2370 provide the determined camera motion information to the OSD circuits 2390 and 2410, respectively. Each of the OSD circuits 2390 and 2410 produces OSD data representing camera motion based on the determined motion information. In one embodiment of the invention, the OSD data comprise arrow indications 2442-2448 showing the motions of the cameras 2200 and 2210. The arrows 2442 and 2448 mean that each camera has moved to the upper and lower directions, respectively. The arrows 2444 and 2446 mean that each camera has moved to the directions in which the distance between the cameras is widened and narrowed, respectively.
The combiners 2380 and 2400 combine the processed image data and the OSD data, and provide the combined image to the display drivers 2420 and 2430. Each of the display drivers 2420 and 2430 displays the combined image in each of the display screens 2440 and 2450.
Referring to
The transmitted data from the camera system are provided to the data separators 2320 and 2330 via the receivers 2300 and 2310 (2500). The image data and the motion detection data are separated in the data separators 2320 and 2330 (2510). The image data are provided to the image processors 2340 and 2360, and each of the processors 2340 and 2360 processes the image data (2520). The motion detection data are provided to the microcomputers 2350 and 2370, and each of the microcomputers 2350 and 2370 determines motion information from the motion detection data (2520).
OSD data corresponding to motion information are generated based on the determined motion information in the OSD circuits 2390 and 2410 (2530). The processed image data and the OSD data are combined together in the combiners 2380 and 2400 (2540). The combined data are displayed in the display screens 2440 and 2450 (2550). When the OSD data are displayed on the display screens 2440 and 2450, this means that at least one of the cameras 2200 and 2210 has moved. Thus, the image also moves in the direction in which the cameras 2200 and 2210 have moved. This is for guiding a viewer's eye lenses to track the motion of the cameras 2200 and 2210. In one embodiment of the invention, the arrows 2442-2448 are displayed right before the image is moved by the movement of the cameras so that a viewer can expect the movement of the images in advance.
In another embodiment of the invention, the display system may allow the viewer to know the movement of the cameras 2200 and 2210 by providing a voice message that represents the movement of the cameras. By way of example, the voice message may be “the stereoscopic cameras have moved in the upper direction” or “the cameras have moved in the right direction.” In this embodiment of the invention, the OSD circuits 2390 and 2410 may be omitted. In another embodiment of the invention, both of the OSD data and voice message representing the movement of the cameras may be provided to the viewer.
In one embodiment of the invention, the camera and display systems shown in
Another aspect of the invention provides a 3D display system that controls the movement of the cameras according to a viewer's eye lens movement. Before describing the aspect of the invention, the relationship between a viewer's eyes and a set of stereoscopic cameras will be described by referring to
The rotation axes C3L and C3R do not move and are the axes around which the cameras 30 and 32 rotate. The rotation axes C3L and C3R allow the cameras 30 and 32 to rotate by behaving like a car windshield wiper, respectively, as shown in
A3L and A3R represent rotation axes of the eyes 38 and 40, respectively. The rotation axes A3L and A3R are the axes around which the eyes 38 and 40 rotate. The rotation axes A3L and A3R allow the eyes 38 and 40 to rotate as shown in
SAL represents the line connecting A2L and A3L. SAR represents the line connecting A2R and A3R. As shown in
Referring to
When a viewer sees an object that is located in front of him or her and is closer than, for example, “10 m,” the viewer's left eye rotates in a clockwise direction and right eye rotates in a counter clockwise direction as shown in
The display site comprises an eye lens motion detecting device 3000, a transmitter 3010, a pair of display devices 2980 and 2990, a pair of receivers 2960 and 2970, and a V shaped mirror 2985. When the camera site transmits stereoscopic images through a pair of transmitters 2900 and 2930 to the display site, the display site receives the images and displays through the display devices 2980 and 2990. A viewer sees stereoscopic images through the V shaped mirror that reflects the displayed image to the viewer. While the viewer is watching the images, the viewer's eye lenses may move in directions, e.g., latitudinal (upper or lower) and longitudinal (clockwise or counterclockwise) directions. Once again, another display device such as a HMD, or a projection display device as discussed above, may be used.
The eye lens motion detecting device 3000 detects motions of each of a viewer's eye lenses while a viewer is watching 3D images through the V shaped mirror. The motions may comprise current locations of the eye lenses. The detecting device 3000 is substantially the same as the device 2100 shown in
The transmitter 3010 transmits the eye lens motion data to the camera site through a communication network 3015. The detection data may comprise identification data that identify each of the left and right eye lenses in the camera site. In one embodiment of the invention, the display site may comprise a pair of transmitters each transmitting left and right eye lens motion data to the camera site. In one embodiment of the invention, before transmitting the motion data, data modification such as encoding and/or modulation adapted for transmitting may be performed.
The camera site comprises a set of stereoscopic cameras 30 and 32, a receiver 2950, a microcomputer 2940, a pair of camera controllers 2910 and 2920, the pair of transmitters 2900 and 2930. The receiver 2950 receives the eye lens motion data from the display site, and provides the data to the microcomputer 2940. The microcomputer 2940 determines each of the eye lens motion data from the received data, and provides the left and right eye lens motion data to the camera controllers 2910 and 2920, respectively. In one embodiment of the invention, the camera site may comprise a pair of receivers each of which receives left and right eye lens motion data from the display site, respectively. In that situation, each receiver provides each eye lens detection data to corresponding camera controllers 2910 and 2920, respectively, and the microcomputer 2940 may be omitted.
The camera controllers 2910 and 2920 control each of the cameras 30 and 32 based on the received eye lens motion data. That is, the camera controllers 2910 and 2920 control movement of each of the cameras 30 and 32 in substantially the same directions as each of the eye lenses 42 and 44 moves. Referring to
The eye lens motion data are provided to the servo controller 3140 (3210). In one embodiment of the invention, the eye lens motion data comprise (x,y) coordinate values, where x and y represent the horizontal and vertical motions of each of the eye lenses, respectively.
The servo controller 3140 determines camera adjusting values (X, Y) based on the provided eye lens motion data. It is determined whether X equals “0” (3230). If X is “0,” the servo controller 3140 does not move the horizontal motor 3120 (3290). If X is not “0,” it is determined whether X is greater than “0” (3240). If X is greater than “0,” the servo controller 3140 operates the horizontal motor 3120 to move the camera 30 in the right direction (3270). As exemplified in
If X is not greater than “0,” meaning this means that X is less than “0,” the servo controller 3140 operates the horizontal motor 3120 so as to move the camera 30 in a counterclockwise (θ1) direction (3260). Referring to
Similarly, it is determined whether Y equals “0” (3300). If Y is “0,” the servo controller 3140 does not move the vertical motor 3130 (3290). If Y is not “0,” it is determined whether Y is greater than “0” (3310). If Y is greater than “0,” the servo controller 3140 operates the vertical motor 3130 to move the camera 30 to +latitudinal (upper: θ2) direction (3320). If the value Y is, for example, “2,” the movement is “4°” in the upper direction.
If Y is not greater than “0,” the servo controller 3140 operates the vertical motor 3130 so as to move the camera 30 in the lower direction (3330). If the value Y is, for example, “−3,” the movement amount is “6°,” and the direction is in a −latitudinal (lower: θ4) direction.
Now, the entire operation of the system shown in
The receiver 2950 of the camera site receives the eye lens motion data from the display site (3050). The camera adjusting values are determined based on the eye lens motion data (3060). The stereoscopic cameras 30 and 32 are controlled by the determined camera adjusting values (3070). In this way, the stereoscopic cameras 30 and 32 are controlled such that the cameras keep track of the eye lens motion. In terms of the viewer, he or she notices that as soon as his or her eye lenses are moved to a certain direction, stereoscopic images are also moved in the direction to which the eye lenses has moved.
The system comprises a microcomputer 3430, a memory 3440, camera selectors 3420 and 3425, and plural sets of stereoscopic cameras 30a and 32a, 30b and 32b, and 30c and 32c. The memory 3440 stores a table as shown in
The microcomputer 3430 determines camera adjusting values based on the received eye lens motion data. The microcomputer 3430 also determines first and second camera selection signals based on the table stored in the memory 3440. The first selection signal is determined based on the movement of a viewer's left eye lens, and used for controlling the camera selector 3420. The second selection signal is determined based on the movement of a viewer's right eye lens, and used for controlling the camera selector 3425. The microcomputer 3430 provides each of the selection signals to the camera selectors 3420 and 3425, respectively.
The camera selectors 3420 and 3425 select the respective camera based on the selection signal. In one embodiment of the invention, a base set of cameras (e.g., C33) shown in
Referring to
Regarding the embodiments described with regard to
The system comprises a set of stereoscopic cameras 30 and 32, a pair of transmitters 2900 and 2930, a set of display devices 3900 and 3910, a 3D mouse 3920, and an input device 3990. The stereoscopic cameras 30 and 32, a pair of transmitters 2900 and 2930, and a pair of receivers 2960 and 2970 are the same as those shown in
In one embodiment of the invention, the input of the 3D mouse is provided to both the display devices 3900 and 3910 as shown in
In one embodiment of the invention, the shape of the 3D mouse cursor comprises a square, an arrow, a cross, a square with a cross therein as shown in
When a viewer wants to see the mountain image 3810 shown in
When the viewer sets the distance value (d1) to, for example, “100 m,” and sees the tree image 3820, Md has Md1 value which is less than Md0 as shown in
When the viewer sets a smaller distance value (d2) to, for example, “5 m” and sees the house image 3830, Md has Md2 value which is less than Md1 as shown in
When the viewer sets a distance value (d3) to the distance between the viewer and the screen, as exemplified as “50 cm,” the mouse cursors 400 and 420 overlap with each other as shown in
As seen in
When the viewer sets the distance value to a value ( d4) less than “d3,” the viewer's sight lines are converged in front of the screen and crossed to each other as shown in
As shown in
The display device 3900 comprises a display screen 3930, a display driver 3940, a microcomputer 3950, a memory 3960 and Interfaces 3970 and 3980. The display device 3900 adjusts the distance (Md) between a pair of 3D mouse cursors 400 and 420 according to the distance value set as shown in
The 3D mouse 3920 detects its movement amount. The detected movement amount is provided to the microcomputer 3950 via the interface 3970. The distance value that the viewer sets is provided to the microcomputer 3950 via the 3D mouse 3920 and the interface 3970. In one embodiment of the invention, the interface 3970 comprises a mouse controller. In another embodiment of the invention, the distance value may be provided to the microcomputer 3950 via the input device 3990 and the interface 3980.
The input device 3990 provides properties of the 3D mouse such as minimum detection amount (Am), movement sensitivity (Bm), and the mouse cursor size (Cm), the viewer-screen distance (d), and viewer's eye data such as Wa and SAL and SAR to the microcomputer 3950 via the interface 3980. The minimum detection amount represents the least amount of movement which the 3D mouse can detect. That is, when the 3D mouse moves only more than the minimum detection amount, the movement of the 3D mouse can be detected. In one embodiment of the invention, the minimum detection amount is set when the 3D mouse is manufactured. The movement sensitivity represents how sensitive the mouse cursors move based on the movement of the 3D mouse. This means that the scroll button of the 3D mouse has different movement sensitivity, i.e., being either more sensitive or less sensitive, according to the distance value. For example, if the distance value is greater than 1,000 m, a “1 mm turn” of the scroll button may increase or decrease the distance by 2,000 m distance. If the distance value is between 100 m and 1,000 m, a “1 mm turn” of the scroll button may increase or decrease distance by 100 m. Similarly, if the distance value is less than 1 m, a “1 mm turn” of the scroll button may increase or decrease the distance by 10 cm.
In one embodiment of the invention, the mouse cursor size may also be adjusted. The distance (d) represents the distance between the middle point of the viewer's eyes and the screen as exemplified in
Also, the input device 3990 provides display device properties to the microcomputer 3950 through the interface 3980. In one embodiment of the invention, the display device properties comprise the display device resolution and screen size of the display device 3900. The resolution represents the number of horizontal and vertical pixels of the device 3900. For example, if the resolution of the display device 3900 is 640×480, the number of the horizontal pixels is 640, and the number of the vertical pixels is 480. The size may comprise horizontal and vertical lengths of the display device 3900. With the resolution and screen size of the display device 3900, the length of one pixel can be obtained as, for example, “1 mm” per 10 pixels.
In one embodiment of the invention, the input device 3990 comprises a keyboard, a remote controller, and a pointing input device, etc. In one embodiment of the invention, the interface 3980 comprises the input device controller. In one embodiment of the invention, the properties of the 3D mouse are stored in the memory 3960. In one embodiment of the invention, the viewer's eye data are detected using a detection device for eye lens movement or provided to the display device 3900 by the viewer.
The microcomputer 3950 determines the mouse cursor distance (Md) based on the distance value set by the viewer. A table (not shown) showing the relationship between the distance value and the Md value as shown in
Referring to
Display device properties are provided to the display devices 3900 and 3910 (4205). In one embodiment of the invention, the display device properties may be stored in the memory 3960.
The viewer's eye data are provided to the display devices 3900 and 3910 (4210). As discussed above, the viewer's eye data may be automatically detected by a detection device or provided to the display devices 3900 and 3910 by the viewer. In one embodiment of the invention, the viewer's eye data comprise the distance (Wa) between the center points of the eyes, and the SA (SAL and SAR) value which is the distance between the eye lens center point (A2) and the eye center point (A3).
A viewer-screen distance (d) is provided to each of the display devices 3900 and 3910 via, for example, the input device 3990 (4220).
The mouse cursor location and distance value are initialized (4230). In one embodiment of the invention, the initialization is performed in an infinity distance value. In this situation, left and right mouse cursors are located at (−Wa/2, 0, 0) and (Wa/2, 0, 0), respectively, where the origin of the coordinate system is O (0, 0, 0) point as shown in
3D image and 3D mouse cursors are displayed in each of the display devices 3900 and 3910 (4240). In one embodiment of the invention, 3D mouse cursors 400 and 420 are displayed on each of the 3D images. Since the mouse cursor location has been initialized, the adjusted mouse cursors 400 and 420 are displayed on the images.
It is determined whether initialized distance value has been changed to another value (4250). When the viewer may want to set different distance value from the initialized distance value, he or she may provide the distance value to the display devices 3900 and 3910.
If the initialized distance value has been changed, 3D mouse cursor distance (Md) is adjusted and the 3D mouse cursor location is reinitialized based on the changed distance value (4260). For example, in case that the initial location is (0, 0, 10,000 m), if another distance value (e.g., 100 m) as shown in
If the initialized distance value has not been changed, it is determined whether 3D mouse movement has been detected (4270).
If the 3D mouse movement has been detected, a new location of the 3D mouse cursors 400 and 420 is determined (4280). In one embodiment of the invention, the new location of the mouse cursors is determined as follows. First, the number of pixels on which the mouse cursors have moved in the x-direction is determined. For example, left direction movement may have “−x” value and right direction movement may have “+x” value. The same applies to “y” direction, i.e., “−y” value for lower direction movement and “+y” value for upper direction movement. The “z” direction movement is determined by the distance value.
The locations of the center points of the display images to be adjusted are calculated based on the new location of the 3D mouse cursors 400 and 420 (4290). In one embodiment of the invention, the locations of the center points of the display images are obtained from the location values of each of the eye lenses, respectively. In this embodiment of the invention, the location values of the eye lenses are obtained using Equations VII and VIII as described below. Referring to
First, the value for ZL is obtained from Equation VII.
In
Second, center point locations [(x1, y1, z1); (x2, y2, z2)] for each eye lens are obtained from Equation VIII. A2L (X1, y1, z1) is the center point location of the left eye lens, and A2R (x2, y2, z2) is the center point location of the right eye lens, as shown in
In one embodiment of the invention, a digital signal processor may be used for calculating the locations of the eye lenses.
Each of the center points of the displayed images is moved to the locations (x1, y1) and (x2, y2), respectively as shown in
It is determined whether the 3D mouse movement has been completed (4310). If the 3D mouse movement has not been completed, procedures 4280-4300 are performed until the movement is completed. This ensures that the displayed images are moved so long as the viewer is moving the mouse cursor.
By using the above calculation method, the distance between two locations can be measured. Referring to
In this embodiment, the microcomputer 3950 is programmed to calculate the distance between two locations, or may comprise a distance measure mode. In this situation, when a viewer designates a first location (A: middle point of two mouse cursors 400 and 420), the location is determined and stored in the memory 3960. In one embodiment, the location value may be displayed in the display screen 3930 or may be provided to a viewer via voice signal. This applies to a second location (B). In this way, the values of the first and second locations (A, B) are determined and the distance between the locations (A, B) is calculated.
Method and System for Controlling the Motion of Stereoscopic Cameras Using a Three-Dimensional Mouse
The system comprises a camera site and a display site. The display site comprises a pair of transmitters/receivers 4530 and 4540, a set of display devices 4510 and 4520, and an input device 3990 and a 3D mouse 3920.
The input device 3990 and 3D mouse 3920 are substantially the same as those of the system shown in
The interface 4810 may modify the location values adapted for transmission, and provide the modified data to the transmitter 4530. The transmitter 4530 transmits the modified location data to the camera site.
Referring to
The servo mechanisms 4580 and 4590 control the cameras 30 and 32 based on the received location data, respectively. In one embodiment of the invention, the servo mechanisms 4580 and 4590 control the cameras 30 and 32 such that the longitudinal and latitudinal values of the center points of the object lenses (C2L, C2R;
Referring to
It is determined whether 3D mouse movement is detected (4650). If movement is detected, the new location of the 3D mouse cursors is determined (4660). The location values of the center points of the viewer's eye lenses are calculated based on the new location of the mouse cursors, respectively (4670). The new location and movement of the mouse cursors (400, 420) are illustrated in
The location value data are transmitted to the camera site through each of the transmitter/receivers 4530 and 4540 (4680). As discussed above, the location values are calculated so long as the mouse cursor is moving. Thus, the location values may comprise a series of data. In one embodiment of the invention, the location values are serially transmitted to the camera site so that the cameras 30 and 32 are controlled based on the received order of the location values. In another embodiment of the invention, the sequence of the generated location values may be obtained and transmitted to the camera site so that the cameras 30 and 32 are controlled according to the sequence. In one embodiment of the invention, the location value data are digital data and may be properly modulated for transmission.
The location value data are received in each of the receivers 4550 and 4560 (4690). In one embodiment of the invention, one transmitter may be used instead of the two transmitters 4530 and 4540. In that situation, one receiver may be used instead of the receivers 4550 and 4560.
Camera adjusting values are determined based on the location values and the stereoscopic cameras 30 and 32 are controlled based on the camera adjusting values (4700). Each of the servo controllers 4580 and 4590 controls the respective camera 30 and 32 such that each of the center points of the cameras object lenses keeps track of the movement of the center points of each eye lens (4710). As shown in
While each of the servo controllers 4580 and 4590 is controlling the stereoscopic cameras 30 and 32, the cameras 30 and 32 are photographing an object. The photographed image is transmitted to the display site and displayed in each of the display devices 4510 and 4520 (4720, 4730).
Regarding the embodiments described with regard to
The system comprises a camera site and a display site. The display site comprises an input device 4910, a set of display devices 4920 and 4930, a transmitter 4950, and a pair of receivers 4940 and 4960.
The input device 4910 provides a viewer's eye distance value (Wa) as shown in
At least one of the display devices 4920 and 4930 displays the space magnification adjusting data that are provided through the input device 4910. The at least one of the display devices 4920 and 4930 provides the space magnification adjusting data and eye distance value (Wa) to the transmitter 4950. The transmitter 4950 transmits the magnification adjusting data and the value Wa to the camera site. In one embodiment of the invention, the space magnification adjusting data and the value Wa may be provided directly from the input device 4910 to the transmitter 4950 without passing through the display devices 4920 and 4930.
The receiver 4970 receives the space magnification adjusting data and Wa from the transmitter 4950, and provides the data to the camera controller 4990. The camera controller 4990 controls the camera distance based on the space magnification adjusting data and the value Wa. The camera controller 4990 comprises a servo controller 4985 and a horizontal motor 4975 as shown in
The servo controller 4985 initializes camera distance (C1), for example, such that C1 is the same as Wa (5100). The space magnification relates to the camera distance (C1) and the eye distance value (Wa). When C1 is the same as Wa, the space magnification is “1,” which means that a viewer sees the same size of the object that is photographed by the cameras 30 and 32. When C1 is greater than Wa, the space magnification is less than “1,” which means that a viewer perceives a smaller space than a space that is imaged by the cameras 30 and 32. When C1 is less than Wa, the space magnification is greater than “1,” which means that a viewer perceives a larger sized object than is imaged by the cameras 30 and 32.
The space magnification adjusting data (SM) are provided to the servo controller 4985 (5110). It is determined whether the adjusting data is “1” (5120). If the adjusting data are “1,” no adjustment of the camera distance is made (5160). If the adjusting data are not “1,” it is determined whether the adjusting data is greater than “1.” If the adjusting data are greater than “1,” the servo controller 4985 operates the motor 4975 so as to narrow C1 until the requested space magnification is obtained (5150). Referring to
If the adjusting data are less than “1,” the servo controller 4985 operates the motor 4975 so as to widen C1 until the requested space magnification is obtained (5140). As exemplified in
Referring to
Regarding the embodiments described with regard to
The system comprises a camera site and a display site. The camera site comprises a set of stereoscopic cameras 30 and 32, a pair of direction detection devices 5410 and 5420, transmitters 5430 and 5440. In this embodiment of the invention, the cameras 30 and 32 may not be parallel to each other as shown in
Each of the transmitters 5430 and 5440 transmits the detected direction data of the cameras 30 and 32 to the display site. If it is detected that only the camera 32 is tilted as shown in
The display site comprises a pair of receivers 5450 and 5460, a pair of display device controllers 5470 and 5500, and a set of display devices 5480 and 5490. Each of the receivers 5450 and 5460 receives the detected tilting data of the cameras 30 and 32, and provides the data to each of the display device controllers 5470 and 5500. The display device controllers 5470 and 5500 determine display adjusting values based on the received camera tilting data. The display adjusting values represent movement amounts to be adjusted for the display devices 5480 and 5490. In one embodiment of the invention, the display device controllers 5470 and 5500 determine display adjusting values based on a table as shown in
Referring to
At least one of the display device controllers 5470 and 5500 determines the display device adjusting values based on the retrieved DDD (5560). The at least one of the display device controllers 5470 and 5500 adjusts the display angle with respect to the viewer's eye lenses by moving a corresponding display device (5570). The display devices 5480 and 5490 display the received stereoscopic images (5580).
The system shown in
Referring to
Referring to
Referring to
In one embodiment, stereoscopic images 624 are produced from a pair of stereoscopic cameras (not shown) and combined with the photographing ratio 622. In one embodiment of the invention, the stereoscopic (broadcasting, advertisement, or movie, etc.) images 624 and the photographing ratio 622 may be transmitted from an Internet server, or a computing device of a broadcasting company. The Internet server may be operated by an Internet broadcasting company, an Internet movie company, an Internet advertising company or an Internet shopping mall company. In another embodiment, the photographing ratio is not combined, and rather, is transmitted separately from the stereoscopic images. However, for convenience, the explanation below will be mainly directed to the combined method.
The combined data 620 are transmitted to a computing device 627 at a display site via a network 625. In one embodiment of the invention, the network 625 may comprise the Internet, a cable, a PSTN, or a wireless network. Referring to
The computing device 627 also provides the left and right images to the display devices 626 and 628, respectively. In one embodiment of the invention, the data format may be constituted such that the computing device 627 can identify the left and right images of the stereoscopic images 624 when the device 627 retrieves the images 624 such as predetermined order or data tagging. In one embodiment of the invention, the computing device 627 may comprise any kind of computing devices that can download the images 624 and ratio 622 either in a combined format or separately via the network 625. In one embodiment, a pair of computing devices each retrieving and providing left and right images to the display devices 626 and 628, respectively may be provided in the display site.
The display devices 626 and 628 display the received stereoscopic images such that the screen ratios (D1:E1:F1, D2:E2:F2) of each of the display devices 626 and 628 are substantially the same as the photographing ratio (A:B:C). In one embodiment of the invention, the screen ratios (D1:E1:F1, D2:E2:F2) are the same (D1:E:F1=D2:E2:F2=D:E:F). The display devices 626 and 628 may comprise the elements of the display devices 86 and 88 disclosed in
In another embodiment of the invention, as shown in
The recording medium 630 is inserted into the medium retrieval device 632 that retrieves the stereoscopic images 624 and photographing ratio 622. In one embodiment of the invention, the medium retrieval device 632 may comprise a CD ROM driver, a DVD ROM driver, or a hard disk driver (HDD), and a host computer for the drivers. The medium retrieval device 632 may be embedded in a computing device (not shown).
The medium retrieval device 632 retrieves and provides the stereoscopic images 624 and photographing ratio 622 to the display devices 634 and 636, respectively. The exemplified data format shown in
In one embodiment of the invention, the data format recorded in the medium 630 is constituted such that the medium retrieval device 632 can identify the left and right images of the stereoscopic images 624. The operation of the display devices 634 and 636 is substantially the same as that of the devices 626 and 628 as discussed with regard to
The pair of digital cameras 640 and 642 produce stereoscopic images of a scene or an object and photographing ratios thereof. In one embodiment of the invention, each of the cameras 640 and 642 comprises substantially the same elements of the camera 20 shown in
The distance input portion 648 is provided with the distance values (similar to screen-viewer distances F1 and F2 in
The device 67 comprises a pair of digital cameras 664, 666, a pair of display screens 654, 656, a distance input portion 658, an eye interval input portion 660, and a space magnification input portion 662. The device 67 also comprises a receiver and a transmitter, or a transceiver (all not shown).
The pair of digital cameras 664 and 666 produce stereoscopic images of a scene or an object and photographing ratios thereof. In one embodiment of the invention, each of the cameras 664 and 666 comprises substantially the same elements of the camera 20 shown in
The distance input portion 658, the eye interval input portion 660, and the space magnification input portion 662 are substantially the same as those of the device 65.
The system shown in
In one embodiment of the invention, the space magnification adjusting data and photographing ratios have a standard data format so that the devices 65 and 67 can identify the data easily.
The Devices Displaying Stereoscopic Images are Implemented Such That the Photographing Ratio is Substantially the Same as the Screen Ratio
The camera portion 700 produces and transmits stereoscopic images and photographing ratios thereof to the device 67. As discussed above, the communication between the devices 65 and 67 may be performed via at least one base station (not shown). The camera portion 700 comprises the pair of digital cameras 640, 642, and a transmitter 710. Each of the digital cameras 640 and 642 produces stereoscopic images and photographing ratios thereof, and combines the images and ratios (combined data 702 and 704). In one embodiment of the invention, the photographing ratios provided in the combined data 702 and 704 are the same. Each of the digital cameras 640 and 642 may comprise the elements of the camera 20 shown in
The production of the stereoscopic images and the calculation of the photographing ratios, and the combining of the images and ratios have been explained in detail with regard to
In one embodiment of the invention, the transmitter 710 may comprise two transmitting portions that transmit the combined data 702 and 704, respectively. The device 67 receives and displays the stereoscopic images transmitted from the device 65 such that the received photographing ratio is substantially the same as the screen ratio of the device 67.
The display portion 720 receives combined data 714 and 716 of stereoscopic images and photographing ratios thereof from the device 67, and displays the stereoscopic images such that the received photographing ratio is substantially the same as the screen ratio of the device 65.
The display portion 720 comprises a pair of display devices 706, 708, and a receiver 712. The receiver 712 receives the combined data 714 and 716 that the device 67 transmitted, and provides the combined data 714, 716 to the display devices 706, 708, respectively. In one embodiment of the invention, the receiver 712 may comprise two receiving portions that receive the combined data 714 and 716, respectively. In another embodiment, the images and photographing ratios are separately received as they are not combined in transmission.
Each of the display devices 706 and 708 separates the provided images and ratios from the receiver 712. The devices 706 and 708 also display the stereoscopic images such that the photographing ratios are substantially the same as the screen ratios of the display devices 706 and 708, respectively. Each of the display devices 706 and 708 may comprise substantially the same elements of the display device 86 or 88 shown in
The microcomputer 740 controls the operation of the camera portion 700 and display portion 720, and data communication with the device 67. In one embodiment of the invention, the microcomputer 740 is programmed to control the camera portion 700 such that the digital cameras 640 and 642 produce stereoscopic images and photographing ratios thereof, and that the transmitter 710 transmits the images and ratios to the device 67 when the communication link is established between the devices 65 and 67. In another embodiment of the invention, the microcomputer 740 is programmed to control the power of the camera portion 700 and the display portion 720 independently. In this embodiment, even when the cameras 640 and 642 are turned off, the display devices 706 and 708 may display the stereoscopic images received from the device 67. Also, when the display devices 706 and 708 are turned off, the cameras 640 and 642 may produce stereoscopic images and photographing ratios thereof, and transmit the images and ratios to the device 67. In this embodiment, the device 65 may comprise an element that performs a voice signal communication with the device 67.
The device 65 may include a volatile memory such as a RAM and/or a non-volatile memory such as a flash memory or a programmable ROM that store data for the communication. The device 65 may comprise a power supply portion such as a battery.
In another embodiment of the invention, the device 65 may include a transceiver that incorporates the transmitter 710 and receiver 712. In this situation, the transmitter 710 and receiver 712 may be omitted.
Though not specifically shown, the device 67 may be configured to comprise substantially the same elements and perform substantially the same functions as those of the device 65 shown in
In one embodiment of the invention, the device 65 moves the stereoscopic images displayed in the display screens 644 and 646 such that the distance (Wd) between the center points of the displayed stereoscopic images is substantially the same as the Wa distance. The device 65 comprises an eye interval input portion 650, a data processor 722, e.g., a microcomputer, a pair of display drivers 724, 726, and a pair of display screens 644, 646. The eye interval input portion 650 and the pair of display screens 644 and 646 are substantially the same as those of
The microcomputer 722 controls the display drivers 724 and 726 based on the received Wa distance such that the Wd distance is substantially the same as the Wa distance. Specifically, the display drivers 724 and 726 moves the stereoscopic images displayed in the display screens 644 and 646 until Wd is substantially the same as Wa. The detailed explanation with regard to the movement of the stereoscopic images has been provided in connection with
In another embodiment of the invention, as shown in
The microcomputer 732 controls the servo mechanisms 734 and 736 based on the received Wa distance such that the Wd distance is substantially the same as the Wa distance. Specifically, the servo mechanisms 734 and 736 move the display screens 644 and 646 until Wd is substantially the same as Wa. The detailed explanation with regard to the movement of the display screens has been provided with regard to
Though not specifically shown, the device 67 may comprise substantially the same elements and performs substantially the same f unctions as those of the device 65 shown in
The camera portion 760 comprises a pair of digital cameras 640, 642, a camera controller 742, and a transceiver 744. The transceiver 744 receives adjusting data for space magnification from the device 67, and provides the adjusting data (C) to the camera controller 742. Space magnification embodiments have been explained in detail with respect to
The camera controller 742 controls the distance (interval) between the digital cameras 640 and 642 based on the provided adjusting data (C). In one embodiment of the invention, the camera controller 742 comprises a motor that adjusts the camera distance, and a servo controller that controls the motor (both not shown). The operation of the camera controller 742 is substantially the same as that of the controller 4990 described in connection with
In another embodiment, space magnification adjusting data (A) may be provided to the camera controller 742, for example, through the space magnification input portion 652 shown in
The display portion 780 comprises a pair of display screens 644, 646, and a transceiver 746. Space magnification (SM) adjusting data (B) are provided to the transceiver 746 from a user of the device 65. The SM adjusting data (B) are used to adjust the interval between the cameras 664 and 666 of the device 67 (
The device 67 receives the SM adjusting data (B) and adjusts the interval between the cameras 664 and 666 of the device 67 based on the adjusting data (B). Also, the device 67 transmits stereoscopic images produced in adjusted space magnification to the device 65. The transceiver 746 receives left and right images from the device 67 and provides the images to the display screens 644 and 646, respectively. The display screens 644 and 646 display the stereoscopic images. In one embodiment, each of the devices 65 and 67 of
The microcomputer 750 controls the operation of the camera portion 760 and display portion 780, and data communication with the device 67. In one embodiment of the invention, the microcomputer 750 is programmed to control the camera portion 760 and display portion 780 such that after the communication link between the devices 65 and 67 is established, the SM adjusting data (B, C) are transmitted or received from or to each other. In another embodiment of the invention, the microcomputer 750 is programmed to control the camera portion 760 such that the camera controller 742 adjusts the interval between the digital cameras 640 and 642 based on the SM adjusting data (A) even when the communication link between the devices 65 and 67 is not established.
The device 65 may include a volatile memory such as a RAM and/or a non-volatile memory such as a flash memory or a programmable ROM that store data for the communication. The device 65 may comprise an element that performs a voice signal transmission.
Though not specifically shown, embodiments of the device 67 comprise substantially the same elements and perform the same functions as those of the device 65 shown in
In another embodiment of the invention, the communication device 65 comprises a goggle shaped display device 649 as shown in
The device 67 may be applied to the embodiments described with regard to
As one example, the three-dimensional structural data comprise pixel values (e.g., RGB pixel values) ranging from, for example, (0000, 0000, 0000) to (9999, 9999, 9999) in the locations from (000, 000, 000) to (999, 999, 999) in a 3D coordinate system (x, y, z). In this situation, Table 1 exemplifies data #1—data #N of the 3D structural data.
In one embodiment of the invention, as shown in
In another embodiment of the invention, as shown in
A method of producing stereoscopic images from the three-dimensional structural data is, for example, disclosed in U.S. Pat. No. 6,005,607, issued Dec. 21, 1999, which is incorporated by reference herein.
This aspect of the invention may be applied to all of the aspects of the invention described above. In some embodiments, however, some modification may be made. As one example, the photographing ratios of the imaginary cameras (projection portions, view points) may be calculated by calculating horizontal and vertical lengths of a photographed object or scene and the distance between the cameras and the object (scene), using the location of the cameras and object in the projected coordinate system.
As another example, the control of the motions of the imaginary cameras may be performed by a computer software that identifies the location of the imaginary cameras and controls the movement of the cameras.
As another example, the control of the space magnification may be performed by adjusting the interval between the imaginary cameras using the identified location of the imaginary cameras in the projected coordinate system.
The above systems have been described showing a communication location connecting the display to a remote camera site. However, these various inventions can be practiced without a receiver/a transmitter and network so that functions are performed at a single site. Some of the above systems also have been described based on a viewer's eye lens motion of location. However, the systems can be practiced based on a viewer's eye pupils or cameras.
While the above description has pointed out novel features of the invention as applied to various embodiments, the skilled person will understand that various omissions, substitutions, and changes in the form and details of the device or process illustrated may be made without departing from the scope of the invention. Therefore, the scope of the invention is defined by the appended claims rather than by the foregoing description. All variations coming within the meaning and range of equivalency of the claims are embraced within their scope.
Claims
1. A method of displaying stereoscopic images, comprising:
- displaying at least one stereoscopic image on a set of display devices, the stereoscopic image comprising a pair of two-dimensional plane images;
- providing, at the set of display devices, adjustment data for spatial magnification, the spatial magnification relating to a size of a space that is imaged;
- transmitting the magnification adjustment data to a set of remotely located stereoscopic cameras; and
- adjusting the distance between the stereoscopic cameras based on the magnification adjustment data.
2. The method of claim 1, further comprising:
- imaging a scene using the adjusted cameras;
- transmitting the image to the set of display devices; and
- receiving and displaying the image on the set of display devices.
3. The method of claim 1, further comprising:
- providing a distance (Wa) between the center points of a viewer's eyes;
- initializing a distance between the stereoscopic cameras to be the same as the Wa value;
- determining whether the magnification adjustment data is greater than 1; and
- narrowing the camera distance if the magnification adjustment data is greater than 1.
4. The method of claim 3, further comprising widening the camera distance if the magnification adjustment data is less than 1.
5. The method of claim 1, further comprising displaying the adjusted spatial magnification on the set of display devices.
6. The method of claim 1, further comprising providing a voice signal representing the adjusted spatial magnification.
7. The method of claim 1, wherein the set of display devices comprises a unitary display device adapted to sequentially display the two-dimensional plane images.
8. The method of claim 1, wherein the set of display devices comprises a pair of display devices configured to simultaneously display the two-dimensional plane images, respectively.
9. A system for displaying stereoscopic images, comprising:
- a set of display devices configured to display at least one stereoscopic image, the stereoscopic image comprising a pair of two-dimensional plane images;
- an input device configured to provide adjustment data for spatial magnification to at least one of the display devices, the spatial magnification relating to a size of a space that is imaged;
- a transmitter configured to transmit the magnification adjustment data to a set of remotely located stereoscopic cameras;
- a receiver configured to receive the magnification adjustment data; and
- a camera controller configured to adjust the distance between the stereoscopic cameras based on the magnification adjustment data.
10. The system of claim 9, wherein the camera controller comprises:
- a servo controller configured to determine an adjustment amount based on the spatial magnification adjustment data; and
- a horizontal motor configured to adjust the camera distance based on the determined adjustment amount.
11. The system of claim 9, wherein the set of display devices is selected from one of the following: a head mount display, a projection display device, a LCD device, a CRT device, and a plasma display panel device.
12. The system of claim 9, wherein the set of the display devices is configured to display the adjusted spatial magnification.
13. The system of claim 9, wherein the set of display devices comprises a unitary display device adapted to sequentially display the two-dimensional plane images.
14. The system of claim 9, wherein the set of display devices comprises a pair of display devices configured to simultaneously display the two-dimensional plane images, respectively.
15. A camera system for communicating data with a set of display devices, comprising:
- a set of stereoscopic cameras configured to produce at least one stereoscopic image;
- a receiver configured to receive adjustment data for spatial magnification from the set of display devices, the space magnification relating to a size of the space that is imaged by the set of stereoscopic cameras; and
- a camera controller configured to adjust the distance between the set of stereoscopic cameras based on the spatial magnification adjustment data.
16. The camera system of claim 15, further comprising a transmitter configured to transmit the adjusted stereoscopic image to the set of display devices.
17. The camera system of claim 15, wherein the receiver is configured to receive a distance (Wa) between the center points of a viewer's eyes and the camera controller is configured to adjust a distance between the stereoscopic cameras based on the magnification adjustment data and the Wa value.
18. A method of controlling a set of stereoscopic cameras, comprising:
- providing a set of stereoscopic cameras;
- generating at least one stereoscopic image, the stereoscopic image comprising a pair of two-dimensional plane images generated by the set of stereoscopic cameras, respectively;
- receiving adjustment data for spatial magnification from a set of remotely located display devices, the spatial magnification relating to a size of the space that is imaged by the set of stereoscopic cameras; and
- adjusting the distance between the stereoscopic cameras based on the spatial magnification adjustment data.
19. The method of claim 18, further comprising:
- initializing the camera distance such that the spatial magnification is 1;
- determining whether the magnification adjustment data is greater than 1; and
- narrowing a distance between the stereoscopic cameras if the magnification adjustment data is greater than 1.
20. The method of claim 19, further comprising widening the camera distance if the magnification adjustment data is less than 1.
21. A method of displaying stereoscopic images, comprising:
- providing a pair of projection portions spaced at a predetermined distance apart from each other and configured to project three-dimensional structural data of a scene into a pair of two-dimensional planes;
- producing at least one stereoscopic image from the three-dimensional structural data by the pair of projection portions spaced at a predetermined distance apart from each other, the stereoscopic image comprising a pair of two-dimensional plane images;
- displaying the pair of two-dimensional plane images in a pair of display devices, respectively;
- providing adjustment data for space magnification to the display devices, the space magnification relating to a size of the scene; and
- adjusting the distance between the projection portions based on the magnification adjustment data.
22. The method of claim 21, wherein the spatial magnification is related to a size of the scene that is perceived by a viewer
23. A system for displaying stereoscopic images, comprising:
- a pair of projection portions spaced at a predetermined distance apart from each other and configured to produce at least one stereoscopic image from three-dimensional structural data of a scene, the stereoscopic image comprising a pair of two-dimensional plane images;
- a pair of display devices configured to display the pair of two-dimensional plane images, respectively;
- an input device configured to provide adjustment data for spatial magnification to the display devices, the spatial magnification relating to a size of the scene; and
- a computing device configured to adjust the distance between the projection portions based on the magnification adjustment data.
24. A system for controlling a set of stereoscopic cameras, comprising:
- means for providing a set of stereoscopic cameras;
- means for generating at least one stereoscopic image, the stereoscopic image comprising a pair of two-dimensional plane images generated by the set of stereoscopic cameras, respectively;
- means for receiving adjustment data for spatial magnification from a set of remotely located display devices, the spatial magnification relating to a size of the space that is imaged by the set of stereoscopic cameras; and
- means for adjusting the distance between the stereoscopic cameras based on the spatial magnification adjustment data.
Type: Application
Filed: Oct 19, 2006
Publication Date: Feb 15, 2007
Inventor: Byoungyi Yoon (Nam-gu)
Application Number: 11/583,542
International Classification: H04N 13/02 (20060101); H04N 13/04 (20060101);