Digital security multimedia sensor
A fully digital camera system provides high-resolution still image and streaming video signals via a network to a centralized, server supported security and surveillance system. The digital camera for collects an image from one or more image transducers, compressing the image and sending the compressed digital image signal to a receiving station over a digital network. A plurality of image transducers or sensors may be included in a single camera unit, providing array imaging such as full 360 degree panoramic imaging, universal or spherical imaging and field imaging by stacking or arranging the sensors in an array. The multiple images are then compressed and merged at the camera in the desired format to permit transmission of the least amount of data to accomplish the desired image transmission. The camera also employs, or connects to, a variety of sensors other than the traditional image sensor. Sensors for fire, smoke, sound, glass breakage, motion, panic buttons, and the like, may be embedded in or connected to the camera. Data captured by these sensors may be digitized, compressed, and networked to detect notable conditions. An internal microphone and associated signal processing system may be equipped with suitable signal processing algorithms for the purpose of detecting suitable acoustic events and their location. In addition, the camera is equipped with a pair of externally accessible terminals where an external sensor may be connected. In addition, the camera may be equipped with a short-range receiver that may detect the activation of a wireless ‘panic button’ carried by facility personnel. This ‘panic button’ may employ infrared, radio frequency (RF), ultrasonic, or other suitable methods to activate the camera's receiver.
1. Field of the Invention
The invention is a continuation of co-pending patent application Ser. No. 09/593,361, filed Jun. 14, 2000 “Digital Security Multimedia Sensor.” The subject invention is generally related to digital cameras and sensors and is specifically directed to a multimedia sensor of use in connection with a digital networked surveillance system. The subject invention in it's preferred embodiment is a networked appliance.
2. Discussion of the Prior Art
Security of public facilities such as schools, banks, airports, arenas and the like is a topic of increasing concern in recent years. Over the past few years, a number of violent incidents including bombings, shootings, arson, and hostage situations have occurred. In addition, agencies responsible for public security in these facilities must cope with more commonplace crimes, such as drug dealing, vandalism, theft and the like.
Such facilities frequently employ monitoring and surveillance systems to enhance security. This has been common practice for an umber of years. Such systems generally have a centralized monitoring console, usually attended by a guard or dispatcher. A variety of sensors are located throughout the facility, such as smoke detectors, fire detectors, motion sensors, glass breakage detectors, badge readers at various access points, and sometimes, video cameras and/or microphones. These prior-art systems often use technologies that are somewhat dated. Sensors are not ‘intelligent’ in the modern sense; they merely provide an ‘ON/OFF’ indication to the centralized monitoring system. The sensors are not ‘networked’ in the modern sense; they are generally hard-wired to the centralized monitoring system via a ‘current loop’ or similar arrangement, and do not provide situational data other than their ON/OFF status.
Video systems in common use today are particularly dated—they are generally of low quality, using analog signals conveyed over coaxial or, occasionally, twisted-pair cabling to the centralized monitoring facility. Such visual information is generally archived on analog video recorders. Further, such systems generally do not have the ability to ‘share’ the captured video, and such video is generally viewable only on the system's control console.
Prior art systems have typically employed analog cameras, using composite video at frame rates up to the standard 30 frames/second. Many such systems have been monochrome systems, which are less costly and provide marginally better resolution with slightly greater sensitivity under poor lighting conditions than current analog color systems. Traditional video cameras have used CCD or CMOS area sensors to capture the desired image. The resolution of such cameras is generally limited to the standard CCTV 300-350 lines of resolution, and the standard 480 active scan lines.
Such cameras are deployed around the area to be observed, and are connected to a centralized monitoring/recording system via coaxial cable or, less often, twisted-pair (UTP) wiring. The signals conveyed over such wiring are almost universally analog, composite video. Baseband video signals are generally employed, although some such systems modulate the video signals on to an RF carrier, using either AM or FM techniques. In each case, the video is subject to degradation due to the usual causes—crosstalk in the wiring plant, AC ground noise, interfering carriers, and so on.
More recently, security cameras have employed video compression technology, enabling the individual cameras to be connected to the centralized system via telephone circuits. Due to the bandwidth constraints imposed by the public-switched telephone system, such systems are typically limited to low-resolution images, or to low frame rates, or both.
Prior-art surveillance systems were oriented towards delivering a captured video signal to a centralized monitoring facility or console. In the case of analog composite video signals, these signals were transported as analog signals over coaxial cable or twisted-pair wiring, to the monitoring facility. In other systems, the video signals were compressed down to very low bit rates, suitable for transmission over the public-switched telephone network.
Each of these prior-art systems suffers functional disadvantages. The composite video/coaxial cable approach provides full-motion video but can only convey it to a local monitoring facility. The low-bit rate approach can deliver the video signal to a remote monitoring facility, but only with severely degraded resolution and frame rate. Neither approach has been used to provide access to any available video source from several monitoring stations.
Another commonplace example is the still-image compression commonly used in digital cameras. These compression techniques may require several seconds to compress a captured image, but once done the image has been reduced to a manageably small size, suitable for storage on inexpensive digital media (e.g., floppy disk) or for convenient transmission over an inexpensive network connection (e.g. via the internet over a 28.8 kbit/sec modem).
Prior-art surveillance systems have been oriented towards centralized monitoring of the various cameras. While useful, this approach lacks the functional flexibility possible with more modern networking technologies.
SUMMARY OF THE INVENTIONThe subject invention is directed to a fully digital camera system having the capability of providing high resolution still image and/or streaming video signals via a network to a centralized, server supported security and surveillance system. A suitable security and surveillance system and related appliances are shown and described in my copending application entitled: “Multi-media Surveillance and Monitoring System including Network Configuration”, filed on even date herewith, and incorporated by reference herein. The digital camera of the subject invention is adapted for collecting an image from one or more image transducers, compressing the image and sending the compressed digital image signal to one or more receiving stations over a digital network.
Recent advances in the art have produced commercially available area sensors with resolutions of 1024×1024, 1280×1024, 3072×2048, and more. These resolutions are continuing to increase, driven in part by the consumer market for digital cameras. As applied to a security camera, such improved resolution provides a significant improvement in the quality of the captured images. Such improved quality allows greater accuracy in recognizing persons or events.
In addition, visual information captured by these sensors is commonly converted to digital form either on the sensor itself, or by an immediate subsequent analog to digital converter device. In digital form, the captured visual information is largely immune to the degradations that plague the prior-art systems. In addition, such digitized visual information is readily amenable to subsequent processing and networking.
This disclosure describes techniques and systems for applying modern image capture, compression, and networking techniques to a camera used in a security monitoring and surveillance network. The camera described herein may employ a high-resolution imager, a CMOS or CCD area sensor capable of capturing images or video at resolutions much higher than existing CCTV-grade cameras. Such resolution is advantageous when attempting to analyze a situation or when reconstructing an event which has been captured and archived. The camera advantageously converts the captured visual information into digital form. This renders it suitable for further processing and networking without risk of visual degradation often seen in analog systems.
The described camera uses video compression techniques to reduce the amount of image data that must be conveyed by the network. Over recent years, a number of image and video compression techniques have been perfected, which may be advantageously employed to significantly reduce the amount of visual data, while preserving the visual quality.
The camera described herein is designed to transport the captured and compressed visual information over a modern digital network. Modern data networks provide connected devices with high bit rates and low error rates, suitable for the transport of compressed visual data streams. Modern networks also employ protocols that render such data streams suitable for addressing and routing over interconnected networks. Modem protocols also allow connected devices to send their data to more than one destination address. These techniques, applied to security and monitoring cameras, overcome the limitation of prior-art systems that supported only one monitoring console.
The described camera also employs, or connects to, a variety of sensors other than the traditional image sensor. Sensors for fire, smoke, sound, glass breakage, gunshot detection, motion, panic buttons, and the like, as described in my aforementioned copending application, may be embedded in or connected to the camera. Data captured by these sensors may be digitized, compressed, and networked, as described therein.
The digital camera system of the subject invention generates the image signal by applying a visual image to an imaging device, preferably a CMOS or CCD area sensor. Suitable sensors are available from a variety of manufacturers, in various sizes, resolutions, sensitivities, and image and signal formats. The image, as applied to the sensor, is converted into an electrical signal. Subsequent processing digitizes the video signal for subsequent compression and networking.
Preferably, the camera uses a very-high resolution imager, with resolutions of 1024×1024 or greater. New imager technologies provide resolutions up to approximately 2K×2 k. This represents an improvement over prior-art systems; prior art surveillance networks are limited to typically 300 TV lines of resolution. This improved resolution allows far greater accuracy in recognizing people or in reconstructing events, and can reduce overall system cost by reducing the number of physical cameras required to achieve a given area coverage at a given resolution.
In the described invention, images captured by the area sensor using high-quality, possibly low-loss techniques, such as to preserve image detail. A variety of compression techniques are currently in use. When used with adequate transmission bandwidth, or given adequate compression time, these compression techniques may produce virtually low-loss results. A commonplace example is the DSS broadcast system, which produces broadcast-quality video at bit rates of 1 to 4 Mbits/sec using MPEG-2 compression.
It is an important feature of the invention that a plurality of sensors may be included in a single camera unit, providing array imaging such as full 360 degree panoramic imaging, universal or spherical imaging and wide angle high resolution flat field imaging by stacking or arranging the sensors in an array. The multiple images are then compressed and merged at the camera or image-processing device connected to the network in the desired format to permit transmission of the least amount of data to accomplish the desired image transmission.
The camera may contain a microphone, audio digitizer, and compressor that allow captured audio to be conveyed, over the attached network along with the captured video. Audio and video samples are time-stamped to allow accurate synchronization at the monitoring station(s).
A variety of suitable audio compression methods exist. The captured audio is of sufficient quality that the (attached) monitoring server may, upon analysis, accurately discern sonic patterns indicative of various disturbances such as glass breakage, gunshots, and the like.
As an alternative, acoustic signal analysis may be performed inside the camera by a suitable signal processing system, so as to trigger the camera when a suitable acoustic event is detected.
In the invention, the digitized and compressed audiovisual signals are fed into a digital network, capable of flexible routing and transport of the signals. While the described invention uses Ethernet as a transport medium for the audiovisual signals, any equivalent digital network may be used.
In addition, the communication protocols used by the network and attachments thereunto embed addressing and routing information into the individual signals. This allows the digital information, produced by the attached cameras, to be efficiently routed and disseminated. An example of this protocol is TCP/IP, commonly used in the Internet.
An advantage of such a network and protocol is that the audiovisual signals, produced by the various cameras, may be accessible by any suitable terminal attached to the network. In particular, cameras are accessible by Internet Browsers and search engines. This is an advantageous contrast to the prior art, where the audiovisual signals produced by the cameras were viewable only on a centralized monitoring station.
As a further refinement, enhanced communications protocols may be employed, which provide more efficient transport of real-time asynchronous signals such as the audiovisual signals produced by the various cameras. Protocols such as Real-Time Protocol (RTP), Real Time Control Protocol (RTCP), IP Multicast Protocols, and others, may be used to reduce overall network bandwidth and provide reliable delivery of the audiovisual data to one or more client recipients.
As a further refinement, the digital networking system used may be a wireless network. Such a network would be of advantage in older institutions where the cost of adding network cabling might be prohibitive or hazardous. Wireless networking also allows cameras or monitoring stations to be mobile. A camera might be temporarily installed in some location for special events, without the time and expense of adding network cabling. Or, a facility guard, on foot, may be able to select and view any particular camera during his rounds.
As a further refinement, the various cameras may synchronize themselves to a master clock using a suitable protocol, such as NTP or SNTP. Over a localized network within a facility, camera time bases may thus be synchronized to within 1 to 10 milliseconds of a master clock. This is advantageous during an event reconstruction, where recorded images or videos from the vantage point of different cameras may be compared. Such camera-to-camera synchronization may also be used for accurately measuring time-of-arrival differences between cameras, thereby allowing the location of said event to be calculated using well-known triangulation techniques.
As a further refinement, an internal data storage device such as a small disk drive may be embedded into the camera. This allows the camera to collect images and/or video and audio from cameras, which may be located at some inaccessible distance from the facility's data network. Stored images or video & audio may be later retrieved for analysis or archival, either by removal of the storage media or by transfer of the stored data over the network.
An additional feature of the present invention is the inclusion of additional sensors to detect notable conditions. Examples might include a smoke or fire detector, an alarm pull-handle, a glass breakage detector, a motion detector, and so on. Additionally, the internal microphone and associated signal processing system may be equipped with suitable signal processing algorithms for the purpose of detecting suitable acoustic events. In addition, the camera may be equipped with a pair of externally accessible terminals where an external sensor may be connected. In addition, the camera may be equipped with a short-range receiver that may detect the activation of a wireless ‘panic button’ carried by facility personnel. This ‘panic button’ may employ infrared, radio frequency (RF), ultrasonic, or other suitable methods to activate the camera's receiver.
In normal operation, the camera is in two-way communication with a suitable server via the digital network. The camera possesses a unique address and is thus distinguishable from other cameras or attached devices.
During normal times, when the camera is powered-on, it may be triggered by various alarms in order to initiate transmission, or triggered by commands sent by the server. Conversely, it may be pre-programmed to transmit at certain times or intervals. Both still and motion video may be transmitted alternately or simultaneously. An onboard archival system may be included to permit temporary storage of data prior to transmission, permitting transmission of pre-event data. The on-board archival system also permits internal storage of images or video at a different resolution than that which is transmitted over the network. This allows pre- and post-event analysis of video at higher resolutions than that transmitted. The on-board storage also allows the device to store data during times where a network connection is absent or intermittent.
Where desired, a local illumination system may be incorporated in the camera for low ambient lighting conditions. This may be infrared, if desired. As described in my aforementioned copending application, various other sensor appliances such as acoustic detectors, motion sensors and the like may activate the camera. These adjunct sensors may be used to trigger the on-board illumination, or the illumination may be on at all times. In addition, the camera and/or lighting can be controlled by manual or automated commands from the server or a workstation on the network.
Various geometries or configurations may be incorporated in the camera design. Specifically, the capability for placing multiple sensors in a single enclosure or unit greatly increases the resolution and/or viewing range of the camera without duplicating the per unit cost associated with prior art cameras by permitting all of the sensors to communicate directly to a single processor, compressor, transmitter circuit. Also, the higher-resolution of this multi-sensor camera can eliminate the need for expensive pan/tilt/zoom mechanisms. It also allows the periodic capture of a wide-field high-resolution view that is not possible with conventional CCTV cameras. In addition, other configurations which can be combined in a single or multiple sensor array are pan, tilt, rotate and zoom features, a single backup power supply for multiple sensor units and the like. The camera can be adapted for wireless communication and can be portable where desired.
It is, therefore, an object and feature of the subject invention to provide a high resolution digital camera for providing both high resolution still and streaming video images in a digital format.
It is another object and feature of the subject invention to provide a digital camera having a plurality of image sensors positioned to provide a predetermined viewing pattern of an area greater than the area of a single sensor, wherein the multiple images may be merged, compressed and transmitted as a single image data signal.
It is an additional object and feature of the subject invention to provide a digital camera that is capable of converting an analog image signal to a digital signal for compression and transmission.
It is another object and feature of the subject invention to provide a digital camera adapted for being incorporated in a multimedia sensor system, wherein other sensors activate the camera for initiation of transmission.
It is yet another object and feature of the subject invention to provide for a digital camera that is suitable for connection to a server supported network wherein the camera may communicate with the server for sending image signals and the server can communicate various control, command and updating signals to the camera.
It is a further object and feature of the subject invention to provide onboard storage capability for storing image data at the camera for recall when transmission is activated.
Other objects and features of the invention will be readily apparent from the drawings and the following detailed description of the preferred embodiments.
BRIEF DESCRIPTION OF THE DRAWINGS
With specific reference to
It should be noted that the sensor 10 could be either an analog camera system in combination with an analog-to-digital converter or a digital camera imager which employs an integral analog-to-digital converter. Where greater resolution is desired, direct digital imaging is the preferred mechanism. A variety of high-resolution digital imagers are currently available, such as the VCA1280C from Symagery Microsystems, or the PCS2112 from Pixelcam Inc.
As shown in
The configuration of the system of
As depicted in
Turning now to
Various implementation schemes for the sensor system are shown in
A block circuit diagram of a useful configuration is shown in
The various sensors and triggering units associated with the camera are not required to be physically located on one camera unit. As shown in
The panel camera configurations of
One of the important features of the various camera configurations is the ability to reconstruct the entire area being covered and to map an event as it progresses. Illustrations of this feature are shown in
A flow chart for the system of
Selection of active cameras is made by the processor 15 or by the network connected server in response to predetermined trigger conditions or programmed controls. This would apply to any co-housed array of spherical, panoramic and panel cameras, and could apply to multiple camera installations as well. It is particularly useful when more than one zone is hot at one time, as described in accordance with
As shown in
Each acoustic detector in
An alternative configuration is shown in
Another alternative configuration is shown in
These various configurations permit the monitor setups as shown in
Cameras designed to render color images typically suffer from reduced luminous sensitivity, compared with monochrome cameras. A method to overcome this deficiency is illustrated in
An optical method for fusing monochrome and color imagers is depicted in
While certain features and embodiments of the invention have been described in detail herein it should be understood that the invention includes all improvements, modifications and enhancements within the scope and spirit of the following claims.
Claims
1. A digital security camera capable of generating and transmitting digital high resolution image signals in both a full motion video format and a still image frame format, the camera comprising:
- a. an image transducer;
- b. a motion video compressor associated with the image transducer for compressing full motion video images for generating a compressed full motion video image data signal;
- c. a still frame compressor associated with the image transducer for compressing still frame images for generating a compressed still frame image data signal;
- d. a multiplexer for merging the compressed full motion video image data signal and the compressed still frame image data signal into a single, combined image data signal;
- e. a processor associated with the multiplexer for generating a conditioned output image signal suitable for transmission over a network; and
- f. a network gateway.
2. The digital camera of claim 1, wherein the compressed still frame image data signal is of a higher resolution than the compressed full motion video image data signal.
3. The digital camera of claim 1, further including an activation mechanism for activating the camera to collect images in response to an activation signal.
4. The digital camera of claim 3, wherein the activation mechanism is an event detector adapted for generating an activation signal in response to the detection of an event.
5. The digital camera of claim 4, wherein the event detector is a manually operated switch.
6. The digital camera of claim 4, wherein the event detector is a sensor adapted for automatically responding to the occurrence of an event.
7. The digital camera of claim 6, wherein the event detector is a smoke detector.
8. The digital camera of claim 6, wherein the event detector is an acoustic event detector.
9. The digital camera of claim 6, wherein the event detector is motion detector.
10. The digital camera of claim 6, wherein the event detector is an alarm trigger switch.
11. The digital camera of claim 3, further including a wireless receiver and wherein the activation signal generator is a remote device having a wireless transmitter for generating an activation signal upon the occurrence of an event.
12. The digital camera of claim 1, further including a plurality of image transducers each adapted for collecting digital high resolution image signals, and a second multiplexer for merging all of said signals into a combined data signal.
13. The digital camera of claim 12, further including a motion compressor and a still frame compressor associated with each transducer and between the transducer and the second multiplexer.
14. The digital camera of claim 12, further including a single motion compressor and a single still frame compressor associated with all of the transducers and positioned between the first mentioned multiplexer and the second multiplexer.
15. The digital camera of claim 12, further including a cylindrical housing for housing the plurality of transducers, each of the transducers mounted in the cylindrical housing such that they are angularly spaced and aimed radially outward from the housing in a manner to collect a combined image representing a full panoramic view of an area within the normal range of the transducers.
16. The digital camera of claim 15, wherein all of the transducers are mounted in a common plane generally perpendicular to the axis of the cylindrical housing.
17. The digital camera of claim 16, further including another plurality of sensors, each of said second plurality of sensors mounted in the cylindrical housing such that they are angularly spaced and aimed radially outward from the housing in a manner to collect a combined image representing a full panoramic view of an area within the normal range of the transducers, said second plurality of sensors mounted in a common plane generally perpendicular to the axis of the cylindrical housing and axially spaced from said first mentioned common plane.
18. The digital camera of claim 12, further including a planar housing for supporting the plurality of sensors mounted in the housing on a planar surface thereof and spaced to provide full image collection coverage for a predetermined area.
19. The digital camera of claim 18, wherein all of the plurality of transducers are mounted in a straight line on the planar surface.
20. The digital camera of claim 19, further including a second plurality of transducers mounted in a second straight line on the planar surface of the housing, said second line being parallel to and spaced from said first mentioned line.
21. The digital camera of claim 12, further including a spherical housing for supporting the plurality of sensors mounted in the housing in angularly spaced, radially projecting relationship to provide full image collection coverage for a predetermined three dimensional space.
22. The digital camera of claim 12, further including a housing comprising an axial sliced cylinder having a planar wall and a partially cylindrical wall, the planar wall adapted for mounting the housing on a relatively flat surface, the plurality of transducers mounted in the cylindrical portion of the housing such that they are angularly spaced and aimed radially outward from the housing in a manner to collect a combined image representing a full panoramic view of an area within the normal range of the transducers.
23. The digital camera of claim 22, wherein all of the transducers are mounted in a common plane generally perpendicular to the axis of the cylindrical housing.
24. The digital camera of claim 23, further including another plurality of sensors, each of said second plurality of sensors mounted in the cylindrical housing such that they are angularly spaced and aimed radially outward from the housing in a manner to collect a combined image representing a full panoramic view of an area within the normal range of the transducers, said second plurality of sensors mounted in a common plane generally perpendicular to the axis of the cylindrical housing and axially spaced from said first mentioned common plane.
25. The digital camera of claim 15, the cylindrical housing further including a stand for supporting the housing on the floor with the transducer plane parallel to the floor.
26. The digital camera of claim 25, including cable and wire passageways in the stand.
27. The digital camera of claim 25, including a power supply for powering the camera housed within the stand.
28. The digital camera of claim 27, wherein the power supply is a self-contained, rechargeable power supply.
29. The digital camera of claim 15, the cylindrical housing including means for supporting the camera from the ceiling with the transducer plane parallel to the ceiling.
30. The digital camera of claim 13, the housing further housing a removable hard drive for storing the image data collected by the transducers.
31. The digital camera of claim 13, the housing further housing a WLAN transceiver.
32. The digital camera of claim 1, wherein the full motion video compressor is an MPEG chip.
33. The digital camera of claim 1, wherein the full motion video compressor is a JPEG chip.
34. A method for monitoring an area and producing a pictorial representation thereof for real time surveillance and for archiving and later retrieval of image data, the method comprising:
- a. placing a plurality of image collectors in such a manner as to provide full coverage of the area being monitored;
- b. assigning a zone to each collector;
- c. temporarily locally storing the data collected at each zone;
- d. transmitting the locally stored data at a specific zone to a central base when a triggering event occurs;
- e. transmitting additional data on a real time basis until the triggering event is terminated.
35. The method of claim 34, further comprising the step of shifting from zone to zone as an event progresses through zones.
36. The method of claim 34, wherein the triggering event is an acoustic event.
37. The method of claim 36, including the steps of:
- a. placing a plurality of acoustic event detectors in the area being monitored;
- b. upon occurrence of an event utilizing the time differential among the acoustic event detectors to triangulate and locate the precise location of the event;
- c. selecting the transducer covering the zone where the event occurred; and
- d. initiating transmission of the image data collected by the selected transducer.
38. The method of claim 34, including the step of mapping the area to be monitored by transducer zone.
39. The method of claim 38, further including the step of tracking an event from zone to zone and providing a moving icon on the map to indicate the zone wherein the event is occurring on a real time basis.
40. The method of claim 34, further including the step of activating a plurality of transducers when an event is occurring in more than one zone.
Type: Application
Filed: Feb 14, 2005
Publication Date: Sep 22, 2005
Inventor: David Monroe (San Antonio, TX)
Application Number: 11/057,814