Imaging System
Signals from an object in a field of view are detected by an array of directional sensors and “re-converged” to create a three-dimensional image as the object changes contrast or moves relative to the sensors. Each sensor is oriented along an axis toward the field of view. Sensor signals are converted to logarithms thereof and transients are detected and compared to background signals. Resulting signals are connected in overlapping groups to coincidence detectors in a matrix. Each point in the field of view where two or more sensor axes intersect is represented by a coincidence detector, which is connected to the corresponding group of sensors. If a threshold number of sensors in the group detects a transient, the corresponding point in the image is deemed to be “contrasty” and can be made visible or otherwise perceivable by a human or can be further processed by a computer or other circuit.
This application is a divisional of U.S. patent application Ser. No. 10/918,151, filed Aug. 13, 2004, titled “Imaging System,” the entire contents of which are hereby incorporated herein for all purposes, which claims the benefit of U.S. Provisional Patent Application No. 60/494,717 titled “Vision System,” filed Aug. 13, 2003.
TECHNICAL FIELDThe present invention relates to imaging systems and, more particularly, to systems capable of imaging hidden or otherwise invisible objects. There exists a large and growing demand for devices that can detect, and display images of, hidden or otherwise invisible objects. Security applications include airport baggage screening and searching for contraband at border crossings. Military applications include detecting hidden weapons and soldiers. Public safety applications include locating victims in burning buildings. Commercial applications include locating pipes, wires or other infrastructure within walls or under roads.
Existing systems, such as x-ray and computerized tomography (CT), capable of imaging hidden objects are large and complex. These systems typically consume large amounts of energy and are typically not portable. Existing portable devices, such as electronic stud sensors, use radar or measurements of dielectric constants to detect hidden objects, however these devices provide essentially binary results, not images of the hidden objects. Existing infrared, ultrasonic and other imaging systems produce adequate images, however, these systems typically operate over relatively small dynamic ranges of signal flux or require re-calibration or relatively slow automatic gain control (AGC) circuits. Background Art
SUMMARY OF EMBODIMENTSThe present invention provides methods and apparatus for acquiring signals from an object within a field of view over a wide dynamic range of signal flux using an array of directional sensors. The methods and apparatus “re-converge” signals from the sensors to create a three-dimensional image of the object, as points on the object change contrast or the object moves relative to the array of sensors (or the array of sensors moves relative to the object). The image of the object can be made visible to a user or it can be further processed, such as by a computer.
The signals from the object can be in the form of transverse or longitudinal waves, such as visible light, infrared light, radio frequency (RF), sound or ultrasound, or any other type of signal that can be detected by a sensor, such as pressure or a chemical attribute. Any combination of signal types can be used. The signals from the object can be reflected, refracted, and/or radiated from the object. In addition, signals from around a backlit object, as well as shadows cast by such an object, can be detected. The invention encompasses active embodiments, which illuminate the object and detect reflections from the object, as well as passive embodiments, which do not illuminate the object and rely on ambient illumination or radiation from the object.
Systems according to the present disclosure can include features to modify signals in ways that simplify subsequent processing of the signals. For example, sensor signals can be converted to signals representing logarithms thereof to enable systems to operate over wide dynamic ranges of flux impinging on the sensors, without requiring automatic gain control (AGC), time to adapt to changes in background flux or time to recover from overloads.
Other signal modifications can include removing unnecessary information from the signals. For example, signals can be modified so that only contrast-changing or moving objects are detected. In this context, portions of an object that change contrast over time (such as blink), as well as edges between the object and its contrasting background and edges between two contrasting portions of the object, are highlighted in the object's image, as the object changes contrast or moves relative to the array of sensors. Circuits detect changes (transients) in the signals received by the sensors from the object to detect these changes in contrast.
Another signal modifying technique involves removing background (common mode) information from the sensor signals. This can be accomplished by comparing the signal from each directional sensor to a background (ambient) signal, such as from a non-directional sensor or an average of several of the directional sensors, and retaining only differences between the compared signals.
In addition to simplifying the detected signals, systems according to the present disclosure can create images for viewing by a human or for subsequent processing by a computer or other circuit. This can be accomplished by detecting coincident transients (i.e., transients coincidentally detected by a plurality of directional sensors), determining points in the field of view where the transients occur and mapping these points to points in the image.
To detect transients in the field of view, each directional sensor is oriented along an axis toward the field of view. The directional sensors are oriented in diverse directions and collectively cover all or portions of the field of view. Outputs from the directional sensors are connected in overlapping groups to coincidence detectors in a coincidence detector matrix. Each point in the field of view where two or more directional sensor axes intersect is represented by a corresponding coincidence detector, which is connected to the two or more intersecting directional sensors. The two or more intersecting directional sensors are referred to as a “group.”
If a predetermined number of the sensors in the group detects a transient within a predetermined time (or another predetermined threshold is reached), the corresponding point in the image is deemed to be “contrasty” (such as an edge point). Alternatively, contrast can be a continuous attribute of the point in the field of view. In addition, the point in the image can be labeled to indicate whether the contrast of the corresponding point in the field of view is increasing, decreasing or cycling. Contrasty points can be made visible or otherwise perceivable by a human, such as by connecting each coincidence detector to a light emitting diode (LED) or other output device. Optionally or alternatively, the outputs of the coincidence detectors can be processed by a computer or other circuit.
Several points in the field of view can lie along the axis of a first directional sensor at different distances from the sensor. Other sensors' axes can intersect the first sensor's axis at these points. Thus, the first directional sensor can detect contrast changes related to objects at various points (along the sensor's axis) in the field of view. Consequently, the first directional sensor can be a member of several groups, and the sensor can be connected to several different coincidence detectors.
Thus, systems according to the present disclosure include one or more aspects of logarithmic transient comparison re-convergence.
These and other features, advantages, aspects and embodiments of the present invention will become more apparent to those skilled in the art from the following detailed description of an embodiment of the present invention when taken with reference to the accompanying non-scale drawings, in which the first digit, or first two digits, of each reference numeral identifies the figure in which the corresponding item is first introduced and in which:
The contents of U.S. Provisional Patent Application No. 60/494,717, titled “Vision System,” filed Aug. 13, 2003, are hereby incorporated by reference herein.
The present disclosure provides methods and apparatus for acquiring signals from an object within a field of view over a wide dynamic range of signal flux using an array of directional sensors (i.e., sensors having relatively small reception angles, or what physicists would refer to as “visual angles”). The methods and apparatus “re-converge” signals from the sensors to create a three-dimensional image of the object, as points on the object change contrast or the object moves relative to the array of sensors (or the array of sensors moves relative to the object). The image of the object can be made visible to a user or it can be further processed, such as by a computer. Applications of these methods and apparatus include viewing devices that enable a user to view images of hidden or otherwise invisible objects, such as wires or pipes within a wall, hidden soldiers or contraband inside luggage.
The signals from the object can be in the form of transverse or longitudinal waves, such as visible light, infrared light, x-rays, radio frequency (RF), sound or ultrasound, or any other type of signal that can be detected by a sensor, such as pressure or a chemical attribute. As shown in
Output signals from the sensors represent the flux impinging on the sensors or are otherwise analogous to the input signals 100. The output signals from the sensors can be processed, such as to convert the sensor signals into signals representing logarithms thereof, detect transients in the sensor signals and/or compare the signals to background flux, as described in more detail below. The processed or unprocessed sensor signals are forwarded to a coincidence detector matrix 104.
The coincidence detector matrix 104 includes a plurality of coincidence detectors. Each coincidence detector corresponds to one of the groups of sensors in the sensor array 102, i.e. to a point in the field of view. The sensor signals from the sensors of each group are connected to the corresponding coincidence detector. The coincidence detector is configured to detect a coincidence of signals from its corresponding sensors, in some embodiments based on a threshold. For example, the coincidence detector can be configured to detect whether a predetermined fraction of the sensors in its corresponding group of sensors produces output signals within a predetermined time. In other embodiments, the coincidence detectors determine a degree of coincidence.
The output of the coincidence detectors can produce a user-perceivable output 106, such as by a visual display device or an audio or tactile output unit. For example, each coincidence detector can drive a point source of light, which collectively renders a three-dimensional image of objects in the field of view. Optionally or alternatively, the output of the coincidence detectors can be further processed, such as by a computer.
Implementations of the components shown schematically in
To handle a wide dynamic range of flux impinging on sensors, sensor signals can be converted to signals representing logarithms thereof. To reduce the amount of processing performed on the output signals from the sensors, only changes in the flux impinging on the sensors need to be processed. Furthermore, each sensor's output signal can be compared to a background (ambient) signal, and common mode characteristics of the sensor's signal can be discarded.
The directional sensor 200 is shown as a collimated sensor, however other configurations of directional sensors are acceptable, as long as the reception angles of the sensors are sufficiently small. In one embodiment, the collimated sensor 200 includes an infrared-sensitive photodiode 200a and a tube 200b. The inside surface of the tube 200b is constructed or treated to reduce reflections, such as by painting the inside surface of the tube flat black. The length-to-diameter ratio of the tube 200b can be selected to achieve a desired reception angle of the directional sensor 200.
The sensor complex detects changes in contrast along the axis of the directional sensor 200. These changes in contrast can be caused by movement of contrasty (relative to the background) points of an object into or out of the reception angle of the directional sensor 200. For example, as points along edges of the object (as viewed against a contrasting background) move into or out of the reception angle of the directional sensor 200, the sensor complex detects these edge points. Similarly, the sensor complex can detect boundaries between two contrasting portions of the object, as points along these boundaries move into or out of the reception angle of the directional sensor 200. The sensor complex can also detect points on a stationary object that change contrast over time, such as because the points blink or because of a change in the amount of radiation illuminating the object, e.g. if the points are more reflective than the rest of the object.
As noted, the sensor complex detects differences between signals detected by the directional sensor 200 and a background (ambient) signal. A non-directional sensor 204 detects background flux 206. Alternatively, the background flux 206 can be ascertained by averaging signals from a plurality of directional sensors. If a plurality of directional sensors is used to ascertain the background flux, the sensors should be oriented in diverse directions.
Output signals from the directional sensor 200 and the non-directional sensor 204 are fed to respective logarithmic converters 208 and 210. Converting signals representing the flux impinging on the sensors 200 and 204 into signals representing logarithms thereof enables the sensor complex to operate over a wide dynamic range of flux impinging on the sensors, without requiring automatic gain control (AGC), time to adapt to changes in the background flux or time to recover from overloads. For example, sensor signals that span a 12-decade range of flux can be processed by subsequent circuits.
The outputs of the logarithmic converters 208 and 210 are fed to respective transient detectors (differentiators) 212 and 214. As noted above, only changes in the flux need to be processed. The outputs of the transient detectors 212 and 214 are fed to a comparator 216. The output of the comparator 216 represents a difference between transients detected along the axis of the directional sensor 200 and transients in the background flux. In one embodiment, the output of the comparator 216 indicates whether the sensor complex detects an increase or a decrease in contrast. Alternatively, the output of the comparator 216 simply indicates that the sensor complex detects a change in contrast, without indicating whether the contrast increased or decreased. In yet other embodiments, the signal detected by the directional sensor 200 is also fed to a subsequent circuit, as described below.
Processing transients in sensor signals makes systems insensitive to DC or slowly changing components of the signals. For example, current flowing through a photodiode slowly heats the photodiode and changes its response characteristics. This effect is commonly called “drift.” However, systems according to the present disclosure are insensitive to this drift, because these systems process transients, not steady-state information, in sensor signals. Comparing changes in flux impinging on the directional sensor 200 with changes in background flux makes systems according to the present disclosure insensitive to changes in illumination on an object (assuming a uniformly reflective object). Furthermore, the ratio of the logarithms of the amount of illumination reflected from two contrasting portions of an object remains constant, even if the amount of illumination impinging on the object varies. In general, “contrast” herein means a ratio of logarithms. Although all three forms of signal processing (logarithmic conversion, transient detection and comparison) provide advantages, systems according to the present disclosure can use any combination of these signal processing techniques.
Appropriate exemplary circuits for the logarithmic converters 208 and 210, the transient detectors 212 and 214 and the comparator 216 are described below. However, other circuits that perform the above-described functions are acceptable, regardless of whether the circuits are implemented using electronic, photonic, hydraulic, molecular or other technologies. Furthermore, these functions can be implemented in hardware, software, firmware or combinations thereof.
As noted, the transients detected by the sensor complex represent contrasty points on objects, as the points move into or out of the reception angle of the directional sensor 200, or as stationary points within the reception angle change contrast. The contrasty points on the objects often correspond to points along the periphery or other edges of the objects. Detecting and displaying a plurality of these periphery or edge points, such as by a three-dimensional display device, can enable a user to identify the objects. To facilitate such detection and display, outputs of a plurality of sensor complexes are fed to a coincidence detector matrix, as discussed below.
Coincidence Detector MatrixOutputs from the sensor complexes are connected in overlapping groups to coincidence detectors in a three-dimensional coincidence detector matrix to detect the plurality of periphery or edge points of objects. Each coincidence detector is connected to a different group of spaced-apart sensor complexes. The directional sensors in each such group are oriented and arranged in the sensor array such that they simultaneously acquire signals from a common point in the three-dimensional field of view. (The term “point” in the field of view refers to a volume of space. The volume is generally larger at progressively greater distances from the directional sensor, because the sensor's reception angle (reception cone) includes more volume at progressively greater distances.) Thus, each coincidence detector corresponds to a point in the three-dimensional field of view. If a predetermined number of these sensors detect a transient within a predetermined time (or another threshold is reached), a corresponding point is deemed to be “contrasty” (an edge point), and the point can be made visible in a three-dimensional image display. In addition, the point in the image can be displayed so as to indicate whether the contrast of the corresponding point in the field of view is increasing, decreasing or cycling. Alternatively, the coincidence detector can use a continuous or stepped scale to represent an amount of contrast change for the point in the field of view, and a visual display can display the point with a brilliance proportional to the point's contrast change or color coded to represent the contrast change. A user could then identify a more contrasty point from less contrasty points, because the point would be displayed with more or less brilliance than nearby points. Similarly, a computer or other circuit can compare a point's contrast to near by points' contrasts.
The sensor complexes are distributed over the sensor surface 304 such that the directional sensors that are parts of groups of sensor complexes are oriented towards common points in the field of view. For example, the directional sensors in sensor complexes 306 and 308 are oriented so an object (or a point of an object) 310 is within the reception angles of the respective directional sensors, i.e. signals 312 and 314 from the object are detected by the directional sensors in sensor complexes 306 and 308.
Note that each sensor complex can be a member of multiple groups. For example, sensor complex 308 is a member of a second group, which also includes sensor complex 316. The directional sensors of sensor complexes 308 and 316 are oriented toward a second object 318. Thus, each group of sensor complexes corresponds to a point in the field of view.
Output signals from the sensor complexes descend at “projection” angles, such as angle 319, into the coincidence detector matrix 300 along respective transient signal lines, such as transient signal line 320. The angle of descent of each transient signal line coincides with (but is not necessarily equal to) the angle of the axis of the respective directional signal sensor (relative to the sensor surface 304), as measured at the respective sensor complex, i.e., the “elevation” angle of the sensor.
At each point where two or more transient signal lines coincide (at a “coincidence point”), the coincidence detector matrix 300 includes a coincidence detector, such as coincidence detector 322. Transient signal lines coincide where signals from more than one sensor complex (that are caused by a common object in the field of view) occur roughly simultaneously or meet some other criteria, as discussed in more detail below. For example, transient signal lines 324 and 326 descend from respective sensor complexes 306 and 308 and coincide at a coincidence detector 328. Coincidence detector 328 corresponds to point 310 in the field of view.
All the directional sensors can be oriented at a common elevation angle. Alternatively, the directional sensors can be oriented at a variety of elevation angles. If three or more sensor complexes are distributed over a three-dimensional sensor surface, or directional sensors on a planar sensor surface are oriented at a variety of elevation angles, more than two transient signal lines can coincide at each coincidence point. If all the directional signal sensors are oriented at a common elevation angle, and signals travel along all the transient signal lines at a common speed, then coincidence points lie along a line (such as line 329) that is normal to the sensor surface and that passes through the corresponding object.
In the preceding discussion, the transient signal lines are described as descending into the coincidence detector matrix at projection angles. Embodiments can be constructed in which the transient signal lines literally descend at these angles, as described in more detail below. However, in other embodiments, the transient signal lines need not be straight. For example, in embodiments in which the transient signal lines are optical fibers, wires or other components that carry electronic signals or other structures that can carry signals over nonlinear paths, the transient signal lines can extend from the sensor complexes to the coincidence detectors at various angles or along curved paths, and the “projection angles” discussed above are used simply to ascertain the location of the coincidence detectors within the coincidence detector matrix. In yet other embodiments, in which the locations of the coincidence detectors within the coincidence detector matrix do not coincide with the coordinates of the object (for example, embodiments in which outputs from the coincidence detectors are processed by a computer, instead of being used to render a visual display within the coincidence detector matrix), these angles are simply conceptual (not necessarily physical) attributes used to associate the coincidence detectors with points in the field of view. Projections into the coincidence detector need not be along straight lines, but sensors having common elevation angles should have common projection angles.
In some embodiments, each coincidence detector is configured to produce an output signal when at least a threshold amount of signal information is received from the respective sensor complexes. For example, the coincidence detector can be configured to produce the output signal if it receives signals from at least a predetermined number of sensor complexes within a predetermined time. For example, in one embodiment, each coincidence detector is connected to seven sensor complexes. The coincidence detector produces an output signal if it receives signals from at least four of the seven sensor complexes within a 1 ms time window. In some embodiments, the length of the time window can be adjusted, e.g. by a user. Alternatively, the coincidence detector can be configured with an adjustable or a “sliding scale” threshold. For example, the coincidence detector can be configured to produce the output signal if it receives at least a relatively small number of signals from the sensor complexes within a relatively short time or if it receives a relatively large number of signals within a relatively longer time. This kind of threshold can accommodate variations in transmission rates (i.e., descent times) of the respective transient signal lines or misaligned directional sensors (where not all the directional sensors of a group receive signals from an object simultaneously).
The threshold can also be based on the magnitudes, rates or directions (i.e., increasing or decreasing contrast) of the transients detected by the sensor complexes. Coincidence detection need not be binary, and the signals from sensor complexes need not be binary. As noted, signals from sensor complexes can indicate a degree of detected contrast change, and the coincidence detectors can process these signals in a variety of ways. For example, the signal values can be summed, and the sum can be used to control the brilliance or color of a respective light source.
An appropriate exemplary circuit for the coincidence detectors is described below. However, other circuits that perform the above-described functions are acceptable, regardless of whether the circuits are implemented using electronic, photonic, hydraulic, molecular or other technologies. Furthermore, these functions can be implemented in hardware, software, firmware or combinations thereof.
Essentially, there exists a physical or logical mapping between physical or logical points in the field of view and the coincidence detectors. The coincidence detector matrix performs a transformation between physical or logical coordinates of the points in the field of view and physical or logical coordinates of a coincidence detector. For example, a coincidence detector matrix can be implemented in software using Basic Linear Algebra Subprograms (BLAS), which are building block routines for performing vector and matrix operations. In such an embodiment, matrix indices are used to represent coordinates in the field of view and in the coincidence detector matrix, and BLAS is used to translate between these sets of indices.
In some embodiments, the output signals from the coincidence detectors drive optical elements located within the coincidence detector matrix 300 (or elsewhere) to produce a three-dimensional visual image, as described in more detail below. For example, each coincidence detector output signal can drive a light emitting diode (LED), liquid crystal display (LCD) element or other light-emitting element or element that is capable of charging an optical characteristic. If the signals sent along the transient signal lines indicate whether the detected changes are increases or decreases in contrast, different color LEDs can be used to show increasing or decreasing contrast, respective. For example, red LEDs can be used to signify decreasing contrast, and green LEDs can be used to signify increasing contrast. In such an embodiment, a coincidence point that rapidly changes from increasing to decreasing (or decreasing to increasing) contrast illuminates both color LEDs in rapid succession, thereby creating an illusion of a third color.
In embodiments that produce visible displays, it can be advantageous to include a visible light filter 330 below the sensor surface 304 to prevent ambient light from contaminating the visual image, particularly if the sensor surface is translucent. Optionally or alternatively, the output signals from the coincidence detectors can be fed to a computer or other circuit for further processing, as indicated at 332. Thus, the coincidence detector matrix 300 is referred to as producing an image of objects within the field of view, whether a visual image is produced or signals are provided to a computer or other circuit.
Hand-Held EmbodimentA sensor surface 406 includes a plurality of sensor complexes, such as exemplary sensor complex 408. The sensor surface 406 can be made in any of a variety of shapes. For example, the sensor surface 406 can be a portion of the surface of a sphere or other smooth shape. Alternatively, the sensor surface 406 can be planar or a portion of the surface of a regular or irregular geometric shape, such as a polyhedron. In other embodiments, the sensor surface 406 can include a plurality of regular or irregular polygonal or curved faces, on or in which the sensor complexes are mounted. Not all the faces of the sensor surface 406 need necessarily be the same size or shape. In the embodiment illustrated in
As noted above with respect to the coincidence detector matrix 300 (
Returning to
Similarly, the directional sensors that are oriented toward object 2 lie along a different circle 602 and are connected to a different coincidence detector. As noted, a directional sensor can be oriented toward multiple points in the field of view, as discussed with respect to sensor complexes 306, 308 and 316 (
The coincidence detectors can be used to construct a three-dimensional image of objects in the field of view. In some of the embodiments discussed thus far, each coincidence detector is physically located within the three-dimensional coincidence detector matrix 300 at coordinates that correspond to coordinates of the imaged points in the field of view.
At the current position of object 1, signals from the object, such as signals 700 and 702, are detected by directional sensors on circle 600 (
As object 1 moves along an object path, different groups of directional sensors detect signals from the object. For example, when object 1 reaches a future position, object signals 714 and 716 are detected by directional sensors along a different circle (not shown) on the sensor surface, such as sensors 718 and 720. Sensor complexes associated with sensors 718 and 720 send signals along transient signal lines 722 and 724, respectively. These signals coincide at a different object coincidence point 726 and trigger a different coincidence detector. Thus, the coincidence point that corresponds to object 1 in the coincidence detector matrix 707 changes position in a manner analogous to the movement of the object in the field of view, and the image produced by the coincidence detector matrix 707 similarly changes. In addition, if output signals from the coincidence detectors produce a visible display (as discussed above), the visible image of object 1 changes in a way that corresponds to the movement of the object. The visible image can be observed through a viewing port 728.
As object 1 moves along a path at a constant distance from the sensor surface 604, the corresponding coincidence points occur on what can be considered a surface at a constant depth below the sensor surface within the detector matrix 707. Similarly, as object 1 moves so the distance between the object and the sensor surface 604 changes, corresponding coincidence points occur at correspondingly varying depths within the detector matrix 707, as illustrated in
In the embodiment shown in
Detecting coincident signals to create an image of objects in a field of view is referred to herein as re-convergence. Embodiments of the disclosed coincidence detector have a plurality of invariant coincidence points, each coincidence point corresponding to a point in the field of view. These points are invariant, in that they do not change under varying illumination conditions, in the face of failed detectors or other variations. Whereas a parabolic reflector has a single, invariant focal point, embodiments of the disclosed system can be though of as having a plurality of focal points, one for each coincidence point.
Viewing PortVisible images produced within the coincidence detector matrix 707 can be viewed through the viewing port 728, which in some embodiments is simply an aperture. Optionally, the viewing port 728 can be fitted with an eyepiece, camera, or other optics. One such embodiment is shown in
As noted, images produced by embodiments of the invention identify contrast-changing points in the field of view. These points can be edge points of objects, as the objects move relative to a contrasting background. These points can also be points along a boundary between contrasting portions of the objects, as the objects move within the field of view. These points can also be points that change contrast (such as blink) on stationary objects. As noted above, light emitting elements, such as LEDs, can be used to make these points visible.
Assuming the object has a surface that is higher in contrast or more reflective than the background, as the object moves to the right, a leading edge 1200 of the object is displayed as a plurality of open circles. Trailing edges 1202 and 1204 of the object are displayed as pluralities of Xs. Thin lines 1206 on the object and small protrusions, such as 1208, are displayed in both colors, because these points are displayed as transitions to a higher reflectivity or contrast and then almost immediately they are displayed as transitions to a lower reflectivity or contrast (or vice versa). Using such a display, a user can readily identify the object.
Logarithmic Converter CircuitAs noted above, some embodiments convert sensor signals to signals representing logarithms thereof to enable these embodiments to handle a wide dynamic range of flux impinging on the sensors.
Photodiodes and other sensors can typically operate in one of a plurality of modes. For example, photodiodes can operate in a current mode, where a closed-circuit current produced by the photodiode has a non-linear relationship to the amount of flux impinging on the photodiode. In a voltage mode, an open-circuit voltage produced by the photodiode has a non-linear relationship to the amount of flux impinging on the photodiode. In a charge displacement mode, an open-circuit charge displacement has a linear relationship to the amount of flux impinging on the photodiode. Embodiments of the presently disclosed system can utilize sensors operated in either of these modes, although linear operation is preferred.
As is well-known to those of skill in the art, variations on the schematic shown in
As noted above with respect to
Outputs from the transient detectors 1408 and 1410 are fed to comparator 1412. The comparator 1412 includes two back-to-back current mirrors 1414 and 1416. The output of the comparator 1412 is taken from a junction of the outputs of the two current mirrors 1414 and 1416.
The schematic of
As is well-known to those of skill in the art, variations on the schematics shown in
As noted above with respect to
In one embodiment, if at least a predetermined number of sensor complexes 51, S2, . . . Sn produce coincidence transient detection signals within a predetermined time window, the charge displacements from the sensor complexes accumulate on the capacitor C1, and the voltage across the capacitor reaches a predetermined threshold (“trigger”) value. The voltage across the capacitor C1 is available at the coincidence output port to drive a subsequent circuit, such as a Schmidt trigger, a four-layer diode with leads to all four layers (such as a Shockley diode or a silicon control switch (SCS)) or other voltage threshold detector (not shown). Leakage current through a second group of three diodes D1502 discharges the capacitor C1, so the coincidence output signal lasts a finite amount of time, resetting the coincidence detector.
If, however, fewer than the predetermined number of sensor complexes S1, S2, . . . Sn produce transient detection signals within the time window, the capacitor C1 does not reach the trigger value, because an insufficient number of charge displacements accumulate on the capacitor. The charges that do accumulated on the capacitor C1 leak off the capacitor through the three diodes D1502, resetting the coincidence detector.
Similarly, if the predetermined number of sensor complexes do produce transient detection signals, but over a time period that exceeds the time window, then the capacitor C1 does not reach the trigger value, because charges accumulated on the capacitor leak off through the three diodes D1502 faster than charges are added to the capacitor.
Thus, the reverse current characteristics of the three diodes D1502 and the number of these diodes determine a maximum time window over which the coincident transients must occur, and the trigger voltage determines the minimum number coincident transients that must occur, to trigger the coincidence detector. Other numbers of diodes can, of course, be used.
In embodiments that utilize ranges of contrast values, the circuit of
Sensor complexes, each of which includes a directional sensor and a non-directional sensor, are described above with reference to
Detector cluster 1700, depicted in
Sensor complex clusters, such as those depicted in
Embodiments of the presently disclosed system can be constructed using a combination of technologies. For example, electronic photodiodes can be used with a coincidence detector matrix/visual display that includes nanodots of fluorochromes.
As shown in the insert on
The fluorochromes used in the embodiment of
The averager 2136 and the comparators 2138-2148 are optional. If these components are omitted, the transient detectors 2124-2134 directly drive the LEDs 2150-2172. In this case, the transient detectors 2124-2134 can provide respective up and down transient signals to drive the LEDs 2150-2172. Similarly, the logarithmic converters and the transient detectors are each optional.
Optionally, threshold detectors (not shown) can be interposed before the LEDs 2150-2172, such as between the comparators 2138-2148 and the respective LEDs. Without the threshold detectors, the LEDs can be driven with continuously variable signals that control the brilliance of the LEDs. With the threshold detectors, the LEDs 2150-2172 can be controlled to be either on or off.
Circuits, software or firmware described above or other function equivalents can be used for the logarithmic converters, transient detectors, comparators and threshold detectors or combinations thereof.
Each comparator 2138-2148 drives one pair of LEDs. For example, LED 2150 is driven when comparator 2138 detects an up transient. The wavelength of the LED 2150 is chosen so it excites fluorochrome type C. The other LED 2152 of the pair is driven when comparator 2138 detects a down transient. The wavelength of the other LED 2152 is chosen so it excites fluorochrome type E. LEDs 2150 and 2152 are arranged to illuminate the end of the nanodot line that corresponds to photodiode 2100. Thus, when a transient is detected, one of the LEDs 2150 or 2152 lights, exciting fluorochrome type C or E in the nanodot line.
For simplicity, an LED that emits a wavelength that excites fluorochrome type X is referred to as “LED X.” Depending on the associated photodiode 2100-2110, an up transient causes either a C or D LED to be driven, while a down transient causes either an E or F LED to be driven.
As noted, the nanodot lines 1204 descend at angles into a coincidence detector matrix.
Assuming two collimated sensors coincidentally detect flux transients that differ from transients in the background flux,
A second signal 2310, initiated by a D LED at the end of the second nanodot line 2304, propagates down the second nanodot line, exciting the D-type fluorochromes 2312. At the coincidence point 2300, coincidental emissions from the C-type fluorochromes 2308 in the first nanodot line 2302 and from the D-type fluorochromes 2312 in the second nanodot line 2304 excite A-type fluorochromes 2314 in one or both nanodot lines. Consequently, the A-type fluorochromes 2314 emit visible light and provide a visible display of a contrasty point on an object.
The nanodot lines 2302 and 2304 and E-type and F-type nanodots therein operate similarly when an opposite transient is detected to excite B-type fluorochromes to emit visible light of a different color.
Nanodot lines can be fabricated in a viewing device, such as the device illustrated in
Alternatively, an embodiment that does not distinguish between up and down transients can be constructed. In such an embodiment, the transient detectors 2124-2134 need not output signals for both types of transients, and only half the number of LEDs 2150-2172 and half the number of fluorochrome types are needed.
SensorsAs noted, the presently disclosed system has been described with reference to infrared sensors, although other sensor types and combinations of sensor types can be used. For example, sensors that detect radio frequency (RF) signals, microwaves (antennas, waveguides, etc.), visible light, ultraviolet or other invisible frequencies of light (photodiodes, photocells, photomultipliers, etc.), x-rays, sonic, or ultrasonic (piezoelectric devices, electret, ribbon or dynamic microphones, etc.) signals can be used. Various well-known techniques can be used to make such signal sensors directional. For example, parabolic dishes can be used to make sonic or ultrasonic sensors directional, and lenses can be used to reduce reception angles of light sensors.
Although embodiments that employ separate comparators are disclosed, other embodiments integrate comparators with sensors. For example,
As discussed with reference to
In general, a higher density of sensor complexes oriented towards a region of the field of view yields higher resolution images of objects in the region. In one embodiment, a 32.times.32 array of sensor cluster complexes is distorted and distributed across a portion of a spherical sensor surface, and each sensor cluster complex includes seven directional sensors.
Output DevicesSome embodiments of the presently disclosed system produced non-visual, humanly-perceptible outputs. For example, outputs from the coincidence detectors can drive tactile, piezoelectric or other sound-generating devices or produce electrical signals that directly stimulate a user's nervous system to cause or simulate perception. The sensor complexes can provide signals in addition to, or in place of, transient signals. For example, the directional sensors can provide transient signals (as described above) and detected audio signals to the coincidence detector matrix. Coincidence detectors that detect a threshold can pass the audio signals to the sound-generating device(s). In this way, the embodiment selectively passes detected sounds from detected objects or locations within the field of view, while blocking ambient sounds. Such an embodiment can be used as a positional hearing aid that automatically selects a listening direction.
The coincidence detectors can use coincidences of sounds to make the threshold determinations. The sensor complexes or the coincidence detectors can include signal processing capabilities to filter out selected types of signals or to select only certain types of signals, such as gun shot sounds. Thus, a device to locate the source of a sound, such as a gun shot, can be constructed.
Alternatively, rather than using the coincidence detector to automatically select a direction from which to pass sound or other signals, a user can identify a point or region in the field of view, thereby selecting one or more groups of directional sensors. Signals from the selected directional sensors can be passed to an output device, such as a speaker. Thus, a “positional microphone” can be constructed. Such a positional microphone picks up sounds from a specified point or region in the field of view. This contrasts with a directional microphone, which picks up sounds from any point along its axis.
If the sensor surface contains a mixture of sensor types, such as light and sound sensors, the coincidence detectors can use one or more types of signals to make the threshold determinations and pass one or more of the types of signals to the humanly-perceptible output device or to a computer or other circuit for further processing. Thus, embodiments can correlate signals from a plurality of sensor types to detect signals, directions or objects of interest. However, the coincidence determinations need not be made using the same type of signal as is ultimately provided to a user. Furthermore, different types of signals can be involved in each of various stages in a system: sensing signals in the field of view, sending transient detection information to coincidence detectors, detecting coincidences, delivering human-perceivable outputs and/or further processing.
Auxiliary ComponentsAs noted with respect to
Embodiments for detecting signals from objects have been described. Optionally, some or all of the directional signal sensors can be paired with directional light sources or other directional wave or particle beam generators, weapons, etc. (collectively “response generators”) oriented in the same directions or toward the same points in the field of view as the respective signal sensors. The paired response generators can be co-located with the directional signal sensors, or the response generators can be located elsewhere. A user can identify a point of interest in the field of view, such as by specifying the corresponding point in the coincidence detector matrix through a user input device. Alternatively, the system can automatically identify the point of interest, such as with pattern matching components, filters, signal processors or other decision logic implemented in hardware, software, firmware or combinations thereof. In either case, the corresponding response generators can illuminate or fire on the identified point or region in the field of view. That is, the device can activate the response generators that correspond to the group of directional signal sensors associated with the point in the field of view, and outputs from the response generators converge on the point of interest in the field of view.
Depending on the nature of the response generators, such a device can function as a directional projector into the field of view (not to be confused with projection angles into a coincidence detector) or a lethal or non-lethal weapon. If the system automatically identifies the point of interest, the projector or weapon can aim itself. For example, a nightlight device can detect an approaching pedestrian and illuminate the pedestrian with a visible lights as he or she walks along an arbitrary path. Furthermore, a control system within the nightlight device can delay extinguishing the response generators (lights) for a predetermined amount of time, such as 30 seconds, after motion ceases to be detected, e.g. to enable the pedestrian to locate his or her keys and unlock a door.
Depending on the nature of the detected object and a confidence level determined by the decision logic, the system can automatically activate the response generators or one of several different types of response generators. For example, if an unidentified person or vehicle is detected, the system can illuminate the object with an infrared or visible light. On the other hand, if, either before or after illuminating the object, the system identifies the approaching object as an enemy, the system can automatically fire a weapon.
Other EmbodimentsAlthough a hand-held embodiment is disclosed, other embodiments can be larger or smaller than a hand-held unit. A sensor array can be any size. For example, microscopic or nano-scale sensor arrays can be used. On the other hand, larger sensor arrays, up to and including astronomic-scale sensor arrays, can be used. A sensor array up to about one hundred meters in diameter can be mounted on a vehicle, such as a truck or rail car. Larger fixed or mobile sensor arrays are also possible. Furthermore, individual sensors, sensor clusters or pluralities thereof can be mounted together on portions of a discontiguous sensor surface, and signals from the sensors can be sent, such as via a local or wide area computer network, to a central location for processing. Some or all of the signal processing can be performed locally (i.e., local to the respective sensors), along the network, at the central location or elsewhere. For example, sensor clusters can be located at various fixed or mobile locations across portions of the earth, other planets, natural or artificial satellites or elsewhere, and signals from these sensor clusters can be processed in a central location. The locations of the sensors need not remain fixed with respect to each other. For example, some sensors of an array can move with an artificial satellite, while other sensors of the array remain stationary on earth.
As noted, in some embodiments, most or all of the directional sensors have a common elevation angle. In other embodiments, the elevation angles of the directional sensors can be dynamically adjusted, such as under software control. For example, to increase the acuity of an image, more directional sensors can be dynamically aimed at points or a region of interest in the field of view. In general, shallower elevation angles (i.e. elevation angles closer to a tangent to the sensor surface) have more sensors in each circle (
Other aspects of systems according to the present disclosure can also be dynamically changed while the systems are in use. For example, the reception angle of one or more of the directional sensors can be changed to adjust the acuity of the system. In one embodiment (similar to the embodiment shown in
Although three-dimensional sensor surfaces, coincidence detector matrices and visual displays have been described, other embodiments can include two-dimensional counterparts.
Those skilled in the art should readily appreciate that the functions of the logarithmic converters, transient detectors, comparators, coincidence detector matrices and other components disclosed herein can be implemented in hardware, software, firmware or a combination thereof. When implemented in hardware, embodiments can include discrete circuits, integrated circuits (including Application Specific Integrated Circuits (ASICSs) and Field Programmable Gate Arrays (FPGAs)), vacuum tubes, photonic circuits, hydraulic circuits, molecular, bioengineered or nano-circuits or other hardware. Although some functional components have been described as diodes, these functions can be implemented with transistors or other circuits, as is well known. Thus, where a term such as “diode” is used, it also means a functionally equivalent component or circuit. Other identified components can be likewise replaced by functional equivalents.
When implemented in software or firmware, embodiments can be stored or delivered in many forms, including, but not limited to, information permanently stored on non-writable storage media (e.g. read-only memory devices for use with a computer, such as a ROM or CD-ROM disk); information alterably stored on writable storage media (e.g. floppy disks, hard drives or flash memory); or information conveyed to a computer or other device through a communication medium, such as a baseband or broadband signal, including a carrier wave, such as over computer or telephone networks via modems.
While the invention is described through the exemplary embodiments described above, it will be understood and appreciated by those of ordinary skill in the art that modification to, and variation of, the illustrated embodiments may be made without departing from the inventive concepts disclosed herein. Furthermore, combinations and sub-combinations of the disclosed embodiments can be utilized. Accordingly, the invention should not be viewed as limited except by the scope and spirit of the appended claims.
Claims
1. A device for generating an image of points in a field of view, comprising: a plurality of directional signal sensors; and a plurality of coincidence detectors, each coincidence detector being connected to receive a signal from each directional signal sensor of a respective group of the directional signal sensors.
2. The device of claim 1, wherein each coincidence detector is configured to produce an output signal upon detecting coincidental signals from at least some of the directional signal sensors of the group.
3. The device of claim 2, wherein the coincidence detector is configured to produce the output signal upon detecting the coincidental signals within a predetermined time.
4. The device of claim 2, wherein the output signal indicates a degree of coincidence among the coincidental signals from the directional signal sensors of the group.
5. The device of claim 2, wherein the output signal does not indicate a degree of coincidence among the coincidental signals from the directional signal sensors of the group.
6. The device of claim 1, wherein each coincidence detector is configured to produce an output signal upon detecting coincidental signals from at least a predetermined number of the directional signal sensors of the group.
7. The device of claim 6, wherein the coincidence detector is configured to produce the output signal upon detecting the coincidental signals within a predetermined time.
8. The device of claim 1, wherein the directional signal sensors of the group lie along at least a portion of a circle.
9. The device of claim 1, wherein the directional signal sensors of the group are oriented toward a common point in the field of view.
10. The device of claim 9 wherein the directional sensors of the group lie along at least a portion of a circle.
11. The device of claim 1, further comprising a plurality of comparators, each comparator being connected between at least one of the directional signal sensors and at least one of the coincidence detectors to provide a signal indicative of a difference between a flux transient detected by the directional signal sensor and a background flux transient.
12. The device of claim 11, further comprising a plurality of background signal sensors, wherein each comparator is connected to at least one of the background signal sensors to receive a signal indicative of the background flux.
13. The device of claim 11, wherein each comparator is also connected to a respective subset of the plurality of directional sensors to obtain a signal indicative of the background flux.
14. The device of claim 1, further comprising a transient detector along a signal path from each of the directional signal sensors to a respective coincidence detector.
15. The device of claim 14, wherein the transient detector is configured to detect up transients and down transients.
16. The device of claim 1, further comprising a logarithmic converter along a signal path from each of the directional signal sensors to a respective coincidence detector.
17. The device of claim 1, further comprising a plurality of visible display elements, each visible display element being connected to one of the plurality of coincidence detectors and configured to provide a visible display when the coincidence detector detects coincidental signals from at least two directional signal sensors of the group.
18. The device of claim 17, wherein each visible display element comprises a light emitting diode.
19. The device of claim 17, wherein each visible display element comprises a liquid crystal display element.
20. The device of claim 17, wherein each visible display element is configured to indicate up transients in the coincidental signals differently than the visible display element indicates down transients in the coincidental signals.
21. The device of claim 17, wherein each visible display element indicates a degree of coincidence among the coincidental signals.
22. The device of claim 2, 3, 6 or 7, further comprising a plurality of visible display elements, each visible display element being configured to provide a visible display in response to the production an output signal by at least one of the plurality of coincidence detectors.
23. The device of claim 22, wherein each visible display element comprises a light emitting diode.
24. The device of claim 1, wherein the plurality of directional sensors is distributed across a two-dimensional surface.
25. The device of claim 1, wherein the plurality of directional sensors is distributed across a three-dimensional surface.
26. The device of claim 25, wherein the three-dimensional surface comprises at least a portion of a geodesic dome.
27. The device of claim 25, wherein the three-dimensional surface comprises a piecewise planar surface.
28. The device of claim 25, wherein the three-dimensional surface comprises at least a portion of a sphere.
29. The device of claim 25, wherein the three-dimensional surface comprises a smooth surface.
30. The device of claim 25, wherein the three-dimensional surface comprises at least a portion of a planet.
31. The device of claim 25, wherein the three-dimensional surface comprises at least a portion of a natural satellite.
32. The device of claim 25, wherein the three-dimensional surface comprises at least a portion of an artificial satellite.
33. The device of claim 1, wherein the plurality of sensors is distributed across a plurality of discontiguous surfaces.
34. A device for generating an image of points in a field of view, comprising: a plurality of directional signal sensors, a plurality of logarithmic converters, each of the plurality of directional signal sensors being connected to a corresponding one of the plurality of logarithmic converters; a plurality of transient detectors, each of the plurality of logarithmic converters being connected to a corresponding one of the transient detectors; a plurality of comparators, each comparator being connected to compare a signal from a corresponding one of the plurality of transient detectors and a signal indicative of a background flux transient to provide a difference signal corresponding to the respective one of the directional signal sensors; and a plurality of coincidence detectors, each coincidence detector being associated with a group of the plurality of directional signal sensors, each coincidence detector being connected to receive the difference signals corresponding to the associated group of directional signal sensors.
35. The device of claim 34, wherein the group of the plurality of directional signal sensors contains at least one directional signal sensor of another group of directional signal sensors.
36. A comparator for producing a difference signal that represents a difference between a first input signal and a second input signal, comprising: a first current mirror having an input port for connection to the first input signal and a PNP transistor driving an output port; and a second current mirror having an input port for connection to the second input signal and an NPN transistor driving an output port; wherein the output port of the first current mirror is connected to the output port of the second current mirror to provide the difference signal.
37. The comparator of claim 36, further comprising: a first logarithmic converter interposed between the first current mirror and the first input signal, the first logarithmic converter having: an input port for connection to the first input signal; and an output port connected to the input port of the first current mirror; and a second logarithmic converter interposed between the second current mirror and the second input signal, the second logarithmic converter having: an input port for connection to the second input signal; and an output port connected to the input port of the second current mirror.
38. The comparator circuit of claim 37, wherein the first and second logarithmic converters each comprise: a field effect transistor having a source and a drain defining a source-drain path therethrough and a gate, wherein the input port of the respective logarithmic converter is connected to the gate of the field effect transistor; a first diode connected in series with the source-drain path of the field effect transistor; a second transistor having an emitter and a collector defining an emitter-collector path therethrough and a base, the base being connected to a junction between the first diode and the field effect transistor; and a second diode connected in series with the emitter-collector path of the second transistor; wherein the output port of the respective logarithmic converter is connected to a junction between the second diode and the emitter-collector path of second transistor.
39. The comparator circuit of claim 37, wherein the first and second logarithmic converters each comprise: a first transistor having a current path therethrough and a high-impedance control port, resistance of the current path being controlled by at least one of a voltage, a current and a charge applied to the first control port, wherein the input port of the respective logarithmic converter is connected to the first control port of the first transistor; a first diode connected in series with the current path through the first transistor; a second transistor having a current path therethrough and a second control port, the second control port being connected to a junction between the first diode and the first transistor; and a second diode connected in series with the current path of the second transistor; wherein the output port of the respective logarithmic converter is connected to a junction between the second diode and the current path of second transistor.
40. A coincidence detector, comprising: a first line comprising a plurality of first, second, third and fifth materials; and a second line comprising a plurality of first, second, fourth and sixth materials and oriented such that a portion of the second line is proximate a portion of the first line; wherein: when excited, the third material radiates a third-type signal that excites the third material; when excited, the fourth material radiates a fourth-type signal that excites the fourth material; when excited, the fifth material radiates a fifth-type signal that excites the fifth material; when excited, the sixth material radiates a sixth-type signal that excites the sixth material; when excited by a coincidence of the third-type and fourth-type signals, the first material radiates a visible light in a first color; when excited by a coincidence of the fifth-type and sixth-type signals, the second material radiates a visible light in a second color; whereby, if the first line is excited by the third-type signal and the second line is coincidentally excited by the fourth-type signal, or the first line is excited by the fifth-type signal and the second line is coincidentally excited by the sixth-type signal, the coincidence detector produces visible light of the first or second color, respectively, in a region where the first line is proximate the second line.
41. The coincidence detector of claim 40, wherein the first, second, third, fourth, fifth and sixth materials each comprise a plurality of nanodots.
42. The coincidence detector of claim 40, wherein the coincidence of the third-type and fourth-type signals comprises an occurrence of the third-type signal within a predetermined time of an occurrence of the fourth-type signal and the coincidence of the fifth-type and sixth-type signals comprises an occurrence of the fifth-type signal within a predetermined time of an occurrence of the sixth-type signal.
43. A coincidence detector, comprising: a first line comprising a plurality of first and second materials; and a second line comprising a plurality of first and third materials and oriented such that a portion of the second line is proximate a portion of the first line; wherein: when excited, the second material radiates a second-type signal that excites the second material; when excited, the third material radiates a third-type signal that excites the third material; when excited by a coincidence of the second-type and third-type signals, the first material radiates a visible light; whereby, if the first line is excited by the second-type signal and the second line is coincidentally excited by the third-type signal, the coincidence detector produces visible light in a region where the first line is proximate the second line.
44. The coincidence detector of claim 43, wherein the first, second and third materials each comprise a plurality of nanodots.
45. The coincidence detector of claim 43, wherein the coincidence of the second-type and third-type signals comprises an occurrence of the second-type signal within a predetermined time of an occurrence of the third-type signal.
Type: Application
Filed: Aug 12, 2010
Publication Date: Dec 2, 2010
Inventor: Jonathan D. Lettvin (Watertown, MA)
Application Number: 12/855,061
