Systems and methods for triggering an out of focus alert
In order to trigger an out of focus alert when the focus level of a video frame meets a focus criteria, a method is performed including the operations of: receiving a video frame, partitioning the video frame into a plurality of blocks, calculating an array of discrete cosine transformation (DCT) coefficients for at least one of the plurality of blocks using a DCT, classifying each of the at least one of the plurality of blocks based on the array of DCT coefficients for that block, calculating a focus level of the video frame from the block classifications, and triggering an out of focus alert if the focus level meets a focus criteria.
Latest Verint Systems Inc. Patents:
The present application claims priority pursuant to 35 U.S.C. §119(e) to U.S. Provisional Application Ser. No. 60/952,896 entitled, “Video Processing,” filed on Jul. 31, 2007, which is hereby incorporated by reference in its entirety.
TECHNICAL FIELDThis disclosure is related to the field of video imaging, and in particular, to the automatic triggering of an alert when a video frame is determined to be out of focus.
BACKGROUNDCurrently video cameras are used for a variety of purposes, including security cameras, data collection, web cameras, and many other uses. Often, a video camera will be configured to continually capture video data without any interaction with a user. For example, some zoos include on their web sites, images from video cameras configured to capture video of one or more of their animals while the animals are in a secluded den or other area out of the normal view of visitors. This allows the visitors to observe the animals without disrupting the normal habits of these animals. In such a case, the camera is set up in a fixed or a panning mode of operation and left to display its video without any further user interaction. However, it is possible for this camera to loose its focus due to a variety of reasons, such as being bumped by the animal, vibrations from other equipment, or one or more of a wide variety of other causes. In such a case, the out of focus camera may not be recognized for a period of time, since visitors may assume that the video quality is always poor and may not alert the zoo to the problem with the camera.
Similar problems arise is other video camera configurations, such as in the case of security cameras. These cameras may also go out of focus for a wide variety of reasons, including natural causes, and in some instances due to people tampering with the device. Often, security cameras are not actively monitored, and their outputs are simply recorded for later viewing if a need arises. However, if the cameras have been tampered with, or otherwise are out of focus, the recordings may be useless for later investigations.
SUMMARYIn this regard, systems and methods for triggering an out of focus alert are provided. An exemplary embodiment of such a method comprises receiving a video frame, partitioning the video frame into a plurality of blocks, calculating an array of discrete cosine transformation (DCT) coefficients for at least one of the plurality of blocks using a DCT, classifying each of the at least one of the plurality of blocks based on the array of DCT coefficients for that block, calculating a focus level of the video frame from the block classifications, and triggering an out of focus alert if the focus level meets a focus criteria.
An exemplary embodiment of a system comprises a storage system containing software, and a processing system coupled to the storage system. The processing system is instructed by the software to receive a video frame, partition the video frame into a plurality of blocks, calculate an array of discrete cosine transformation (DCT) coefficients for at least one of the plurality of blocks using a DCT, classify each of the at least one of the plurality of blocks based on the array of DCT coefficients for that block, calculate a focus level of the video frame from the block classifications, and trigger an out of focus alert if the focus level meets a focus criteria.
An exemplary embodiment of a computer-readable medium of instructions for triggering an out of focus alert in a computer system comprises receiving a video frame, partitioning the video frame into a plurality of blocks, calculating an array of discrete cosine transformation (DCT) coefficients for at least one of the plurality of blocks using a DCT, classifying each of the at least one of the plurality of blocks based on the array of DCT coefficients for that block, calculating a focus level of the video frame from the block classifications, and triggering an out of focus alert if the focus level meets a focus criteria.
Other systems, methods, features and/or advantages of this disclosure will be or may become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features and/or advantages be included within this description and be within the scope of the present disclosure
Many aspects of the disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views. While several embodiments are described in connection with these drawings, there is no intent to limit the disclosure to the embodiment or embodiments disclosed herein. On the contrary, the intent is to cover all alternatives, modifications, and equivalents.
As discussed above, video cameras may go out of focus for a wide variety of reasons, including tampering. In order to alert a user when a camera goes out of focus, a method for monitoring the focus level of a video camera, and automatically triggering an alert when the focus level meets a focus criteria is presented. While some current cameras have the ability to auto-focus, this feature is not perfect. For example, the camera may focus on an object at a different distance from the camera, other than the object of interest in the video frame. For a wide variety of reasons, it may be desired to configure a video camera for manual focus on a particular area of interest. Unless this camera is continually monitored, it is possible for the camera to go out of focus without being observed. By measuring the focus level of a video frame, and comparing this focus level to a criteria, an apparatus or method is configured to trigger an alert when the focus level is reduced to a level meeting the criteria.
In operation 106, the at least one of the plurality of blocks is classified into one of a number of different block types based on the arrays of DCT coefficients for each block. These block types and an example method of classification will be discussed in detail below, with respect to
α=
α=255[maxInt−(minInt+1)].
The intensity of each pixel is then scaled by α as follows:
Int=α*(Int−minInt).
Other embodiments may scale the luminance in other ways, or may not scale the luminance at all, all within the scope of the present invention.
In operation 606, the video frame is partitioned into a plurality of non-overlapping blocks. In some example embodiments, the video frame may be partitioned into 8 pixel by 8 pixel blocks, while other embodiments may use other block sizes and shapes all within the scope of the present invention. In this example, the video frame block size matches the mask block size so that masked blocks are not processed. In operation 608, an array of discrete cosine transform (DCT) coefficients are calculated for at least one of the plurality of blocks using a discrete cosine transform. DCT coefficients are not calculated for masked blocks. In an example embodiment, a type-II DCT is used to calculate the array of DCT coefficients while other embodiments may use other types of discrete cosine transforms all within the scope of the present invention.
In operation 610, the array of DCT coefficients is weighted by the contents of a weight array, such as the array illustrated in
The scalar is calculated using the following formula:
The lowScore is calculated using the following formula:
The highScore is calculated using the following formula:
In an operation 702, the dark_value is compared to a dark threshold. If the dark_value is less than the dark threshold, the block is classified as a dark block in operation 704. and the classification of the present block is completed 720. If the dark_value is greater than or equal to the dark threshold, in an operation 706, the noise_value is compared to a noise threshold. If the noise_value is less than the noise threshold, the block is classified as a noisy block in operation 708, and the classification of the present block is completed 720. If the noise_value is greater than or equal to the noise threshold, in an operation 710, the lowScore is compared to a plain threshold. If the lowScore is less than the plain threshold, the block is classified as a plain block in operation 712, and the classification of the present block is completed 720. If the lowScore is greater than or equal to the plain threshold, in an operation 714, the lowScore is compared to the highScore. If the lowScore is less than the highScore, the block is classified as an unsharp block in operation 716, and the classification of the present block is completed 720. If the lowScore is greater than or equal to the highScore, the block is classified as a sharp block in operation 718, and the classification of the present block is completed 720.
Communication interface 801 could comprise a network interface, modem, port, transceiver, or some other communication device. Communication interface 801 may be distributed among multiple communication devices. Processing system 802 could comprise a computer microprocessor, logic circuit, or some other processing device. Processing system 802 may be distributed among multiple processing devices. User interface 803 could comprise a keyboard, mouse, voice recognition interface, microphone and speakers, graphical display, touch screen, or some other type of user device. User interface 803 may be distributed among multiple user devices. Storage system 804 could comprise a disk, tape, integrated circuit, server, or some other memory device. Storage system 804 may be distributed among multiple memory devices. The memory can include any one or combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, etc.)) and nonvolatile memory elements (e.g., ROM, hard drive, tape, CDROM, etc.). Moreover, the memory may incorporate electronic, magnetic, optical, and/or other types of storage media. Note that the memory can have a distributed architecture, where various components are situated remote from one another, but can be accessed by the processor. Additionally, the memory includes an operating system, as well as instructions associated with methods for image processing. Exemplary embodiments of each of which are described above.
Processing system 802 retrieves and executes software 805 from storage system 804. Software 805 may comprise an operating system, utilities, drivers, networking software, and other software typically loaded onto a computer system. Software 805 could comprise an application program, firmware, or some other form of machine-readable processing instructions. When executed by processing system 802, software 805 directs processing system 802 to operate as described herein. In this example embodiment of the present invention, the software 805 may be configured to cause the processing system 802 to execute the operations of the methods illustrated in
One should note that the flowcharts included herein show the architecture, functionality, and/or operation of a possible implementation of software. In this regard, each block can be interpreted to represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of the order. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
One should note that any of the programs listed herein, which can include an ordered listing of executable instructions for implementing logical functions (such as depicted in the flowcharts), can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a “computer-readable medium” can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a nonexhaustive list) of the computer-readable medium could include an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM or Flash memory) (electronic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) (optical). In addition, the scope of the certain embodiments of this disclosure can include embodying the functionality described in logic embodied in hardware or software-configured mediums.
It should be emphasized that the above-described embodiments are merely possible examples of implementations, merely set forth for a clear understanding of the principles of this disclosure. Many variations and modifications may be made to the above-described embodiments without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure.
The above description and associated figures teach the best mode of the invention. The following claims specify the scope of the invention. Note that some aspects of the best mode may not fall within the scope of the invention as specified by the claims. Those skilled in the art will appreciate that the features described above can be combined in various ways to form multiple variations of the invention. As a result, the invention is not limited to the specific embodiments described above, but only by the following claims and their equivalents.
Claims
1. A method for triggering an out of focus alert, comprising:
- receiving a video frame;
- partitioning the video frame into a plurality of blocks;
- calculating an array of discrete cosine transformation (DCT) coefficients for at least one of the plurality of blocks using a DCT;
- generating block classifications by classifying at least one block of the at least one of the plurality of blocks as noisy based on the array of DCT coefficients for that block;
- calculating a focus level of the video frame based at least in part on the block classifications; and
- triggering an out of focus alert if the focus level meets a focus criteria.
2. The method of claim 1, further comprising:
- biasing the luminance of the video frame.
3. The method of claim 1,
- wherein each block is classified as one block type from the group of block types consisting of: dark, noisy, plain, unsharp, or sharp.
4. The method of claim 3,
- wherein the focus level is calculated by dividing the number of sharp blocks by the sum of the number of unsharp blocks and sharp blocks.
5. The method of claim 1,
- wherein the focus criteria is a focus threshold.
6. The method of claim 1,
- wherein the focus criteria includes a persistence level.
7. The method of claim 1,
- wherein the array of DCT coefficients for a block comprises 64 coefficients.
8. The method of claim 7, further comprising:
- calculating a dark value for the block; and
- classifying the block as a dark block if the dark value is less than a dark threshold.
9. The method of claim 8,
- wherein the dark value is equal to dct[0], and
- wherein dct[0:63] is the array of DCT coefficients, and dct[0] is the first element in the array of DCT coefficients.
10. The method of claim 1, ∑ k = 0 9 d c t [ k ] ∑ k = 48 63 d c t [ k ], and
- wherein the noise value is equal to
- wherein dct[0:63] is the array of DCT coefficients.
11. The method of claim 7, further comprising:
- scaling the array of DCT coefficients by a weight factor contained in a weight array.
12. The method of claim 11, further comprising:
- calculating a scalar from the array of DTC coefficients;
- calculating a lowScore from the array of DTC coefficients; and
- calculating a highScore from the array of DTC coefficients.
13. The method of claim 12, scalar = min ( 16, ∑ k = 48 63 d c t [ k ] + 2 ); lowScore = ( ∑ k = 1 5 [ d c t [ k ] * weight [ k ] scalar ] ) * scalar; highScore = ( ∑ k = 6 63 [ d c t [ k ] * weight [ k ] scalar ] ) * scalar; and
- wherein calculating a scalar uses the formula:
- wherein calculating a lowScore uses the formula:
- wherein calculating a highScore uses the formula:
- wherein dct[0:63] is the array of DCT coefficients, and weight[0:63] is the weight array.
14. The method of claim 12, further comprising:
- classifying the block as a plain block if lowScore is less than a plain threshold.
15. The method of claim 12, further comprising:
- classifying the block as an unsharp block if lowScore is less than highScore.
16. The method of claim 12, further comprising:
- classifying the block as a sharp block if: lowScore is greater than or equal to highScore, lowScore is greater than or equal to a plain threshold, the dark value is greater than or equal to a dark threshold, and the noise value is greater than or equal to a noise threshold.
17. The method of claim 1, further comprising:
- masking the video frame.
18. A computer system, comprising:
- a storage system containing software; and
- a processing system coupled to the storage system;
- wherein the processing system is instructed by the software to: receive a video frame; partition the video frame into a plurality of blocks; calculate an array of discrete cosine transformation (DCT) coefficients for at least one of the plurality of blocks using a DCT; generate block classifications by classifying at least one block of the at least one of the plurality of blocks as noisy based on the array of DCT coefficients for that block; calculate a focus level of the video frame based at least in part on the block classifications; and trigger an out of focus alert if the focus level meets a focus criteria.
19. A non-transitory computer-readable medium of instructions for triggering an out of focus alert in a computer system, the instructions comprising:
- receiving a video frame;
- partitioning the video frame into a plurality of blocks;
- calculating an array of discrete cosine transformation (DCT) coefficients for at least one of the plurality of blocks using a DCT;
- generating block classifications by classifying at least one block of the at least one of the plurality of blocks as noisy based on the array of DCT coefficients for that block;
- calculating a focus level of the video frame based at least in part on the block classifications; and
- triggering an out of focus alert if the focus level meets a focus criteria.
5065246 | November 12, 1991 | Takemoto et al. |
5347310 | September 13, 1994 | Yamada et al. |
5363137 | November 8, 1994 | Suga et al. |
5363138 | November 8, 1994 | Hayashi et al. |
5835163 | November 10, 1998 | Liou et al. |
5905535 | May 18, 1999 | Kerdranvat |
6067118 | May 23, 2000 | Chen et al. |
20050006598 | January 13, 2005 | Pearl |
20060104621 | May 18, 2006 | Nakata |
20060127077 | June 15, 2006 | Kim |
20070115356 | May 24, 2007 | Kang et al. |
1450559 | August 2004 | EP |
2006118559 | November 2006 | WO |
- Murino, V. et al., “Visual surveillance by depth from focus”, Industrial Electronics, Control and Instrumentation, 1994, IECON 1994, 20th International Conference on Bologna, Italy, Sep. 5-9, 1994, New York, USA, IEEE, vol. 2 dated Sep. 5, 1994, pp. 998-1003, XPO10137463, ISBN: 978-0-7803-1328-6.
Type: Grant
Filed: Nov 9, 2007
Date of Patent: Jan 1, 2013
Patent Publication Number: 20090033796
Assignee: Verint Systems Inc. (Santa Clara, CA)
Inventors: Fuk Sang Mak (Hong Kong), Hing Yip Chung (Hong Kong), Tao Luo (Hong Kong), Wai Shing Yiu (Hong Kong), Chi Kin Lee (Hong Kong), Shiu Hang Tsang (Hong Kong)
Primary Examiner: Pankaj Kumar
Assistant Examiner: Sahar Baig
Application Number: 11/937,553
International Classification: H04N 5/00 (20110101);