Omniview motionless camera orientation system

A device for omnidirectional image viewing providing pan-and-tilt orientation, rotation, and magnification within a hemispherical field-of-view that utilizes no moving parts. The imaging device is based on the effect that the image from a fisheye lens, which produces a circular image of at entire hemispherical field-of-view, which can be mathematically corrected using high speed electronic circuitry. More specifically, an incoming fisheye image from any image acquisition source is captured in memory of the device, a transformation is performed for the viewing region of interest and viewing direction, and a corrected image is output as a video image signal for viewing, recording, or analysis. As a result, this device can accomplish the functions of pan, tilt, rotation, and zoom throughout a hemispherical field-of-view without the need for any mechanical mechanisms. The preferred embodiment of the image transformation device can provide corrected images at real-time rates, compatible with standard video equipment. The device can be used for any application where a conventional pan-and-tilt or orientation mechanism might be considered including inspection, monitoring, surveillance, and target acquisition.

Skip to:  ·  Claims  ·  References Cited  · Patent History  ·  Patent History

Claims

1. A device for providing perspective corrected views of a selected portion of a hemispherical view in a desired format that utilizes no moving parts, which comprises:

a camera imaging system for receiving optical images and for producing output signals corresponding to said optical images;
fisheye lens means attached to said camera imaging system for producing said optical images, throughout said hemispherical field-of-view, for optical conveyance to said camera imaging system;
image capture means for receiving said output signals from said camera imaging system and for digitizing said output signals from said camera imaging system;
input image memory means for receiving said digitized signals;
image transform processor means for processing said digitized signals in said input image memory means according to selected viewing angles and magnification, and for producing output transform calculation signals according to a combination of said digitized signals, said selected viewing angles and said selected magnification;
output image memory means for receiving said output signals from said image transform processor means;
input means for selecting said viewing angles and magnification;
microprocessor means for receiving said selected viewing angles and magnification from said input means and for converting said selected viewing angles and magnification for input to said image transform processor means to control said processing of said transform processor means; and
output means connected to said output image memory means for recording said perspective corrected view according to said selected viewing angles and magnification.

2. The device of claim 1 wherein said output means includes image display means for providing a perspective corrected image display according to said selected viewing angle and said magnification.

3. The device of claim 1 wherein said input means further provides for input of a selected portion of said hemispherical view to said transform processor means.

4. The device of claim 1 wherein said input means further provides for input of a selected tilting of said viewing angle through 180 degrees.

5. The device of claim 1 wherein said input means further provides for input of a selected rotation of said viewing angle through 360 degrees to achieve said perspective corrected view.

6. The device of claim 1 wherein said input means further provides for input of a selected pan of said viewing angle through 180 degrees.

7. The device of claim 1 wherein said output transform calculation signals of said image transform processor means are produced in real-time at video rates.

8. The device of claim 1 wherein said input means is a user-operated manipulator switch means.

9. The device of claim 1 wherein said image transform processor means is programmed to implement the following two equations: ##EQU7## where:

R=radius of the image circle
.beta.=zenith angle
.differential.=Azimuth angle in image plane
.O slashed.=Object plane rotation angle
m=Magnification
u,v=object plane coordinates
x,y=image plane coordinates

10. A device for providing perspective corrected views of a selected portion of a hemispherical view in a desired format that utilizes no moving parts, which comprises:

a camera imaging system for receiving optical images and for producing output signals corresponding to said optical images;
fisheye lens means attached to said camera imaging system for producing said optical images, throughout said hemispherical field-of-view, for optical conveyance to said camera imaging system;
image capture means for receiving said output signals from said camera imaging system and for digitizing said output signals from said camera imaging system;
input image memory means for receiving said digitized signals;
image transform processor means for processing said digitized signals in said input image memory means according to selected viewing angles and magnification, and for producing output signals, said selected viewing angles and said selected magnification, according to the equations; ##EQU8## where:
R=radius of the image circle
.beta.=zenith angle
.differential.=Azimuth angle in image plane
.O slashed.=Object plane rotation angle
m=Magnification
u,v=object plane coordinates
x,y=image plane coordinates
output image memory means for receiving said output signals from said image transform processor means;
input means for selecting said viewing angles and magnification;
microprocessor means for receiving said selected viewing angles and magnification from said input means and for converting said selected viewing and magnification for input to said image transform processor means to control said processing of said transform processor means; and
output means connected to said output image means for recording said perspective corrected views according to said selected viewing angles and implementation.

11. A device for providing perspective corrected views of a selected portion or a hemispherical view in a desired format that utilizes no moving parts, which comprises:

a camera imaging system for receiving optical images and for producing output signals corresponding to said optical images;
fisheye lens means attached to said camera imaging system for producing said optical images, throughout said hemispherical field-of-view, for optical conveyance to said camera imaging system;
image capture means for receiving said output signals from said camera imaging system and for digitizing said output signals from said camera imaging system;
input image memory means for receiving said digitized signals;
image transform processor means for processing said digitized signals in said input image memory means according to selected viewing angles and magnification, and for producing output transform calculation signals in real-time at video rates according to a combination of said digitized signals, said viewing angles and said selected magnification;
user operated input means for selecting said viewing angles and magnification;
microprocessor means for receiving said selected viewing angles and magnification from said user operated input means and for converting said selected viewing angles and magnification for input to said image transform processor means to control said processing of said transform processor means;
output image memory means for receiving said output transform calculation signals in real-time and at video rates from said image transform processor means; and
output means connected to said output image memory means for recording said perspective corrected views according to said selected viewing angles and magnification.
Referenced Cited
U.S. Patent Documents
4772942 September 20, 1988 Tuck
5023725 June 11, 1991 McCutchen
5067019 November 19, 1991 Juday et al.
5068735 November 26, 1991 Tuchiya et al.
Foreign Patent Documents
011909 June 1984 EPX
2-127877 May 1990 JPX
WO 82/03712 October 1982 WOX
Other references
  • Two (2) Japanese prior art articles authored by Dr. Murio Kuno (1980). Heckbert, "Fundamentals of Texture Mapping and Image Warping", Report No. UCB/CSD 89/516, Jun. 1989. Heckbert, "The PMAT and Poly User's Manual", NYIT Document, 1983. Transcript of trial testimony of Dr. Ned Greene, Interactive Pictures Corporation, f/k/a Omniview, Inc., v. Infinite Pictures, Inc. and Bill Tillman, Civil Action No. 3-96-849, U.S. District Court, Eastern District of Tennessee, Feb. 2, 1998. Upstill, Steve, Building Strong Images, UNIX Review, Oct. 1988, pp. 63-73., Greene, Ned, A Method of Modeling Sky for Computer Animation, Proceedings for a computer animation conference, 1984. Greene, Ned and Heckbert, Mark, Creating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter, IEEE Computer Graphics and Applications, Jun. 1986, pp. 21-27. Greene, Ned, Environment Mapping and Other Applications of World Projections, IEEE Computer Graphics and Applications, Nov. 1986, pp. 21-29. Color copies of 12 prior art slides (1984-86) shown and described in the Greene trial testimony in Transcript pages identified in captions, Feb. 2, 1998. Block diagram of Greene/NYIT system (1986), Greene testimony from Transcript, Feb. 2, 1998, pp. 33-37. Diagrams of the geometry employed in the "Fisheye to Box" and "Poly" software used in the DX 280 system, Greene trial testimony from Transcript, Feb. 2, 1998, pp. 43-50. Heckbert, Paul, NYIT PMAT and Poly Users Manual, 1983. Heckbert, Paul, Fundamentals of Texture Mapping and Image Warping, Computer Science Division, University of California, Berkeley, Masters Thesis, Jun. 1989. Transcript of trial testimony of Steven D. Zimmermann, Interactive Pictures Corporation, f/k/a Omniview, Inc. v. Infinite Pictures, Inc. and Bill Tillman, Civil Action No. 3-96-849, U.S. District Court, Eastern District of Tennessee, Jan. 6, 1998, pp. 77-142, 152-156. Drawing from Zimmermann testimony, Jan. 6, 1998. Zimmermann et al, excerpts from Phase I NASA Test Report, Aug. 1988. TMC2302 ASICS Data Sheets, TRW LSI Products, Inc., LaJolla, CA, 1990. Transcripts of relevant trial testimony of Dr. Douglas Birdwell: Transcript of Jan. 7, 1998, pp. 61-72, Transcript of Jan. 8, 1998, pp. 27-42, Transcript of Feb. 5, 1998, pp. 65-165. Two sketches drawn by Dr. Douglas Birdwell, Transcript, Jan. 8, 1998, pp. 27-41. Function, Statistics and Trigonometry, Scott, Foresman & Company, 1992, pp. i-x, 143, 709-720. U.S. Geological Survey Professional Paper 1395, Map Projections--A Working Manual, 1987, pp. viii-ix, 3-10, 33-35, 90-91, 164-168. Plaintiff's exhibits PX 559 and PX560 showing the failure of perspective correction of a fisheye image of Hawthorne Bridge side rail using '667 patent test program (previously submitted) when actual image radius R.about.256 pixels is used and with a reduced radius, R.about.230 pixels, to get a somewhat less distorted output image. (Birdwell Feb. 5, 1998 Transcript at pp. 149-150, line 1). Color image of the Hawthorne Bridge showing distortion at different magnification (Birdwell Transcript of Feb. 5, 1998, pp. 149-152). TMC2301 ASICS Data Sheet, TRW LSI Products, Inc., LaJolla, CA, 1988. DX 402- showing similarity of transforms performed by TRW TMC2302 and TMC 2301 ASICs TRW LSI Products, Inc., LaJolla, CA, 1988. (Birdwell Transcript, Feb. 5, 1998. pp. 157-160). G. Wolberg, "Digital Image Warping", IEEE Computer Society Press, 1988. F. Kenton Musgrave, "A Panoramic Virtual Screen for Ray Tracing", Graphics Gems, 1992, pp. 288-294. J. D. Foley et al., "Computer Graphics: Principles and Practice", 1990, 1996, pp. 229-381. Intel Corporation, "Action Media 750 Production Tool Reference", 1988, 1991. S. Morris, "Digital Video Interactive--A New Integrated Format for Multi-Media Information", Microcomputer for Information Management, Dec. 1987, 4(4):249-261. "Declaration of Scott Gilbert in Support of Defendant Infinite Pictures' Memorandum in Opposition to Plaintiff's Motion for Preliminary Injunction", Omniview, Inc. v. Infinite Pictures, Inc., Civ. Action No. 3-96-849 G. David Ripley, "DVI--A Digital Multimedia Technology", Communications of the ACM Jul. 1989, vol. 32, No. 7, pp. 811-822. M. Onoe et al., "Digital Processing of Images Taken by Fish-Eye Lens", IEEE: Proceedings, New York, 1982, vol. 1, pp. 105-108. N. Greene, "Environment Mapping and Other Applications of World Projections", IEEE Computer Graphics and Applications, Nov. 1986, pp. 21-29. N. Greene, "A Method of Modeling Sky for Computer Animations", Proc. First Int'l. Conf. Engineering and Computer Graphics, Aug. 1984, pp. 297-300. J. Blinn et al., "Texture and Reflection in Computer Generated Images," Comm. ACM, vol. 19, No. 10, 1976, pp. 542-547. N. Greene et al., "Creating Raster Omnimax Images from Multiple Perspective Views Using the Elliptical Weighted Average Filter", IEEE Computer Graphics and Applications, Jun. 1986, pp. 21-27. R. Kingslake, "Optical System Design", Academic Press, 1983, pp. 86-87. S. Ray, "The Lens in Action", Hastings House, 1976, pp. 114-117. F. Pearson II, "Map Projections Theory and Applications", CRC Press, Inc., 1990, pp. 215-345. A. Paeth, "Digital Cartography for Computer Graphics", Graphics Gems, 1990, pp. 307-320.
Patent History
Patent number: RE36207
Type: Grant
Filed: Jul 12, 1996
Date of Patent: May 4, 1999
Assignee: Omniview, Inc. (Oak Ridge, TN)
Inventors: Steven D. Zimmermann (Knoxville, TN), H. Lee Martin (Knoxville, TN)
Primary Examiner: Glenton B. Burgess
Law Firm: Banner & Witcoff, Ltd.
Application Number: 8/662,410
Classifications
Current U.S. Class: 348/207; Panoramic (348/36); Observation Of Or From A Specific Location (e.g., Surveillance) (348/143); 382/43
International Classification: H04N 530;