LIQUID CRYSTAL FOURIER TRANSFORM IMAGING SPECTROMETER
A hyperspectral imaging system has a processor to receive hyperspectral imaging parameters and produce a series of images to be acquired at a series of retardances at a series of retardance times, a hyperspectral imaging component having an input polarizer to polarize an incoming beam of light, a liquid crystal variable retarder to receive the polarized beam of light and to produce wavelength-dependent polarized light, an output polarizer to receive the wavelength-dependent polarized light and to convert polarization state information into a form detectable as light intensity, a voltage source connected to the liquid crystal variable retarder, and a retardance controller. The retardance controller receives the series of retardances at a series of retardance times and produces a series of voltages at a series of voltage times to apply to the liquid crystal variable retarder. A focal plane array, synchronized with the retardance controller, receives the light in a form detectable as light intensity and converts the light to a series of images.
This application is a continuation-in-part of, and claims priority to, U.S. patent application Ser. Nos. 14/527,347, and 14/527,378, filed Oct. 29, 2014, which are incorporated herein in their entirety.
BACKGROUNDHyperspectral imaging (HSI) collects and processes information from across the ultraviolet (UV), visible, and infrared portions of the electromagnetic spectrum. A hyperspectral imaging camera obtains hyperspectral image data or HSI data which consists of spectral information at each point in a scene. HSI is frequently employed to increase the depth of information in a scene's image, or to increase the image's visual contrast beyond what can be recorded with a conventional monochrome or color camera. This enhanced contrast can be used to detect hard to find or camouflaged objects obscured by visual noise; it can also aid in materials identification. It can be used to assess detailed information about the state of a subject, such as the ripeness of a piece of fruit. Well-known applications of HSI abound for domains as diverse as industrial and agricultural sorting, remote sensing for agriculture and defense, threat identification, and even medicine.
The advent of smartphone technology has provided powerful, mobile platforms that a significant fraction of the world's population carries on their person at most times. There is a trend toward increasing the number and types of sensors present on smartphones, and the computing power of these phones is correspondingly increasing. Smartphones already include multiple image sensors, but they are not currently thought of as candidates for HSI cameras because of the prohibitive size and cost of existing HSI technologies.
SUMMARYAn embodiment is a hyperspectral imaging system, including a processor to receive hyperspectral imaging parameters and to convert the imaging parameters to a series of images to be acquired at a series of retardances at a series of retardance times, a hyperspectral imaging component having at least one input polarizer, wherein the input polarizer receives and polarizes an incoming beam of light, a liquid crystal variable retarder arranged adjacent the input polarizer opposite the incoming beam of light to receive the polarized beam of light from the input polarizer and to change polarization of the light to produce wavelength-dependent polarized light, an output polarizer arranged to receive the wavelength-dependent polarized light and to convert polarization state information of the light into a form detectable as light intensity, a voltage source electrically connected to the liquid crystal variable retarder, and a retardance controller, the controller to receive the series of retardances at a series of retardance times and to produce a series of voltages at a series of voltage times to be applied to the liquid crystal variable retarder, and a focal plane array synchronized with the retardance controller to receive the light in a form detectable as light intensity as a function of retardance of the liquid crystal variable retarder and convert the light to an output signal as a series of images, wherein the processor performs transformations of the series of images to hyperspectral image data.
Another embodiment is a method of producing hyperspectral image data, including receiving a set of hyperspectral imaging parameters at a processor, generating, with the processor, a series of retardances at a series of retardance times corresponding to the hyperspectral imaging parameters, sending the series of retardances at the series of retardance times to a retardance controller, generating, with the retardance controller, a series of voltages at a series of voltage times to be applied to a liquid crystal variable retarder, applying the series of voltages at the series of voltage times to a liquid crystal variable retarder, capturing a series of images through the liquid crystal variable retarder with a focal plane array, and using the processor to generate hyperspectral image data from the series of images.
Hyperspectral imaging has many promising use cases such as for sorting, remote sensing and medical applications. The cost, size, and usability of this technology have limited the applications. If one could include a hyperspectral imaging camera into many common systems, hyperspectral imaging (HSI) could become much more widespread. For example, smartphones are readily available technology platforms for HSI cameras. Inclusion on the smartphone could push the boundaries of what is possible with HSI, as the smartphone is a general-purpose platform carried by most individuals for which it is easy to develop new applications.
A current state-of-the-art HSI camera might work by scanning a slit across a scene and dispersing the light transmitted through the slit via a grating. Or, it might work by placing liquid crystal tunable filters in front of a focal plane array that allow instantaneous imaging at one wavelength band. As used in this discussion, the term focal plane array, or FPA, is one or more light sensing elements arranged in an array, where an array could be an array of one. Examples of FPAs include charge-coupled devices (CCDs), CMOS image sensors, etc. Light emanating from a scene falls upon the FPA and generates signals that are processed to produce an image of the scene. Lower-cost hyperspectral imaging cameras might tile optical bandpass filters on the focal plane array itself, similar to existing red-green-blue cameras but with more discrete optical bands, trading off spatial resolution for greater spectral resolution. While liquid crystal tunable filters and optical bandpass filters allow one to obtain an image in a given wavelength band immediately, they do this at the expense of discarding all out-of-band wavelengths.
Typically, one wants to image a scene over a range of wavelengths. It is therefore desirable to record light from all the wavelengths at once, rather than only from one spectral band at a time. This can be achieved using optically multiplexed forms of hyperspectral imaging, such as Fourier transform hyperspectral imaging, which encodes wavelength information into a time signal that is recorded by a detector. Any optically multiplexed technique, such as those exemplified in the current embodiments, has gains in optical throughput, a feature known as Fellgett's advantage.
Systems that use gratings and two-dimensional focal plane arrays, also known as dispersive systems, can record light from all wavelengths within a given band and along one spatial axis simultaneously. Spatial information along the other spatial axis is built up by scanning a slit over the object to be imaged in a pushbroom manner. Spatial information is instantaneously obtained in the long direction of the slit and the short direction is dispersed in wavelength with a grating or prism so the FPA can record all of the wavelengths at once. Spatial information in the short direction of the slit is collected as the slit is scanned in time across the object. Dispersive systems are often high in cost, size, or complexity. The embodiments here do not incur these penalties because they merely add a controllable liquid crystal variable retarder in the optical path before the FPA. In addition, the use of a slit in dispersive systems restricts the total optical throughput at any given time to the light that passes through the slit. By foregoing the use of slits, Fourier transform hyperspectral imaging systems and other multiplexed systems gain a throughput advantage known as the Jacquinot advantage.
The conventional way to perform Fourier transform hyperspectral imaging is to use an imaging Michelson interferometer, which is a device that splits an imaging path into two arms, and that varies the length of one of the arms while recording the recombined light on a FPA. This kind of HSI gains from both the Fellgett and Jacquinot advantages, but it is bulky, costly, and sensitive to vibration and misalignment. The current embodiments suffer from none of these drawbacks.
The embodiments here enable a new class of HSI cameras that have the potential to be extremely small and low-cost, and are capable of being integrated anywhere cameras are currently used, including on smartphones. The embodiments have advantages in a general-purpose host platform like a smartphone because performance parameters such as wavelength resolution, imaging speed, and spatial resolution can be selected in software and are not fixed by the hardware configuration. The HSI components described in the embodiments here can be operated in conjunction with existing camera components of the host system, such as FPAs, optics, and processors, to form an HSI camera in combination. The HSI components and the camera components of the host system can be triggered with the same camera button, and the HSI components can be configured in software to allow the camera components of the host system to take normal, non-hyperspectral images. The embodiments leverage the increasing prevalence of computing power by shifting most of the system complexity to the electronic/software side, therefore keeping overall system cost low.
In addition, the use of a smartphone or other portable device having wireless or wired communications capability allows the device to communicate the raw HSI data. Alternatively, the device would transmit the analyzed or processed HSI data for use with larger devices, such as medical equipment, or as part of a diagnostic routine.
A portable device-based HSI camera enables current applications of HSI at the consumer level. More importantly, as HSI becomes widely deployed on a mobile platform and as mobile app developers learn to exploit the capabilities of HSI, they undoubtedly will uncover novel and interesting uses for HSI.
The following discussion uses several terms having the following definitions. A “liquid crystal variable retarder” refers to at least one liquid crystal (LC) cell, comprising liquid crystal material sandwiched between two transparent, typically glass, substrates. Transparent electrode layers deposited on the glass substrates, typically made of a transparent conductor such as indium tin oxide or ITO, provide for the generation of an electric field within the liquid crystal cell, which is used to vary the orientation of the liquid crystal molecules and hence the optical retardance of the liquid crystal variable retarder. Additional layers may be provided interior to the cell, such as polyimide alignment layers deposited on the electrode layer that are rubbed in a preferred alignment direction to orient the liquid crystal molecules. Instead of a single LC cell comprising the liquid crystal variable retarder, there may be several.
A “hyperspectral imaging component” or “HSI component” refers to a component having an input polarizer or polarizing device, a liquid crystal variable retarder, an output polarizer or analyzer, a retardance controller and a voltage source. The input polarizer is between the LC variable retarder and the source of light. The output polarizer resides on the side of the LC variable retarder opposite the first polarizer. The retardance controller controls the voltages applied to the LC variable retarder to achieve a desired retardance of light passing through the LC retarder.
A “hyperspectral imaging system” or “HSI system” refers to a combination of the HSI component, a focal plane array as defined above, and a processor. The processor may reside in a stand-alone device, meaning a device that contains all of the elements of the system including the processor and the FPA, and functions independently of a host. Alternatively, the processor and/or the FPA may be components of a host system, with all of the associated combinations. A “host system” or “host” is an optional device that sends HSI parameters to an HSI camera or an HSI system and receives HSI data in return. An “HSI camera” is an HSI system with the necessary optics to record an image.
An example of a device that is a host system having an HSI component (LC variable retarder, retardance controller and voltage source) is shown in
In certain embodiments of the HSI component, the optical axis of the LC variable retarder is nominally at 45 degrees with respect to the polarizers. For a given wavelength of incident light that passes through the input polarizer, the HSI component oscillates between transmitting and not transmitting the light as the optical retardation increases. This oscillation occurs because the LC variable retarder periodically alters the polarization state of the light as the retardance increases, and the output polarizer functions to alternately block or pass the light after the retarder based on its polarization state. Retardance may be described as optical path delay or optical phase delay, as discussed below, where optical phase delay is proportional to the optical path delay divided by the wavelength.
The intensity oscillations as a function of optical retardance, collectively called the interferogram, occur with a period that depends on the incident wavelength. Each unique incident wavelength oscillates in intensity as a function of retardance at a different rate, and the intensity oscillations from a combination of incident wavelengths sum together linearly. The wavelengths can be separated by Fourier transform of the received light as a function of optical retardance, in this case optical path delay, yielding an optical spectrum. Since the liquid crystal variable retarder is placed in the optical path of the focal plane array, the FPA can independently sample and record the intensity oscillations or interferogram at each point in an image and use this information to calculate hyperspectral image data.
The spectral resolution Δλ at each wavelength and each point in the hyperspectral image data is given by the formula, Δλ=2πλ/Δϕ, where Δϕ is the range of optical phase delays at which the interferogram is recorded, expressed in radians. From this formula it is apparent that to resolve wavelength differences significantly smaller than the center wavelength, interferograms must be recorded with ranges of optical phase delay Δϕ>>2π. This requirement differentiates the current embodiments from liquid crystal tunable filters that typically do not need to scan the phase delays of their constituent liquid crystal stages beyond a range of 2π. Furthermore, the range of phase delay is a function of wavelength λ liquid crystal birefringence Δn, and position-dependent effective thickness of the liquid crystal layers β, expressed with the following equation: Δϕ=2πΔn(λ,T,V)β(x,y)/λ. Here, birefringence is a function of wavelength, temperature T, and time-dependent liquid crystal voltage V, and expresses the birefringence between two rays normally incident to the liquid crystal variable retarder with ordinary and extraordinary polarization. The change in incidence angle of the chief ray with position of a given pixel on the FPA and the corresponding position dependence of the optical phase delay range are incorporated into the position-dependent effective thickness β.
The optics 22 may include a relay lens or other relay optical device as will be discussed in more detail later. The optics may also include an imaging lens. Typically, imaging lenses are fixed relative to the FPA. However, as the HSI camera may consist of or be present on a hand-held device, the imaging lens may be movable or the device may have other optical techniques to allow for image stabilization that would compensate for unintended hand movement. Other optical elements may be included in the optics or elsewhere in the system, such as a chromatic compensation device. The optics may be under control of processor 28.
Alternatively, the processor may receive the output signal from the FPA and perform image stabilization on the output signals. The output signals will typically represent frames of image data detected by the FPA with each frame acquired at a particular retardance state of the liquid crystal variable retarder. The processor may receive at least two output signals from the FPA, each representing a frame of image data detected at the FPA. Ideally, there is no relative motion between a scene or objects in a scene and the FPA during the acquisition of output signals corresponding to a single HSI dataset. However, because this case of no relative motion is not always practical, the processor may perform an image analysis to determine and apply registration of regions of the image or of objects in the image, compensating such motion. This compensation may eliminate or mitigate the imaging artifacts of images captured while the system is unstable, images of moving objects, and even relative scene motion caused by the beam walk-off phenomenon, to be described subsequently.
After passing through the imaging optics 22, the light travels through an input polarizer 40, the liquid crystal variable retarder 42, an optional compensation layer 43, an optional waveplate 45, and an output polarizer or analyzer 44. The light then strikes the FPA 24. The light may take a second path to a photodetector 47. In some embodiments, the resulting detection signals are extracted by a retardance extractor 27 and sent to the processor 28 and the retardance controller 50. The retardance extractor can extract the retardance of the liquid crystal variable retarder from signals generated by the photodetector 47, or the FPA 24, or both. It can consist of a separate electronic circuit or processor, or it can be a function that resides within the camera processor 28. The detection signals are generally sent directly to the processor 28 from the FPA 24 so they can be processed into hyperspectral image data before being sent to the requesting system or host, if there is one. In addition, as will be discussed in more detail later, the liquid crystal variable retarder may include a temperature sensor, such as a thermometer, 30. Such a thermometer can be fabricated within a LC cell of the LC variable retarder by patterning a thermistor into the electrode. It also may be a thermistor or a similar electronic component that is in thermal contact, meaning that the component is thermally connected, with the LC variable retarder or that senses the approximate temperature of one or more LC cells. The HSI data is acquired by processing multiple images taken by the camera at multiple times and multiple retardances provided by the liquid crystal variable retarder under the control of the retardance controller.
As mentioned above, optional elements of the HSI imager may consist of a waveplate 45, and a retardance or phase compensation layer or compensator such as 43. A waveplate allows for phase shifting between the polarization components to allow alteration of the polarization of the light. A retardance, or phase compensation layer applies a static path delay at each point in an image, enabling sampling of a different portion of the interferogram than would normally be sampled by applying a voltage waveform to the liquid crystal variable retarder. One embodiment of such a compensation layer would achieve net zero retardance in combination with the liquid crystal variable retarder when the HSI component is inactive, and could additionally be designed to compensate for non-zero retardance at any, some, or all wavelengths of interest. This would prevent the HSI component from interfering with the normal non-hyperspectral operation of a camera.
Another kind of compensation layer would consist of multiple regions of different path delays tiled across the FPA. An example compensation layer consists of two regions, one of path delay 0 and the other of path delay D. If a LC variable retarder can generate path delays from 0 to D, the region of the image corresponding to the region of the compensation layer with path delay 0 will produce an interferogram with path delay from 0 to D, whereas the other region will produce an interferogram with path delay from D to 2D. If a special lens were used that produced a duplicate image of a scene with one image per compensation layer region, then an interferogram of this scene with path delay from 0 to 2D could be formed in software by stitching together the two image duplicates acquired at different path delay ranges. This would provide hyperspectral image data with twice the spectral resolution of an interferogram with path delay from 0 to D yet it would be measured in half the time; however, the image would have half the spatial resolution in one dimension. In general, such retardance compensation layers offer a means of trading off spatial and spectral resolution and imaging speed.
If a specific retardance is required during an extended period of time, for example holding the retardance at 0 for traditional non-hyperspectral imaging, the retardance of the variable retarder could be set to a desired value and maintained at that value over time. In another modality, an effective retardance could be accomplished over time by continuously varying the retardance in such manner that the average retardance over time is the desired one.
The HSI component may have many different configurations.
As shown in
The resulting time-dependent variations in intensity are picked up by the focal plane array such as 24, with the detected intensity versus path delay shown in
Some embodiments achieve retardance through high optical path delays between the two polarization components while maintaining low liquid crystal driving voltages and/or fast liquid crystal response times. As is known in the art of Fourier transform spectroscopy, high optical path delays yield high spectral resolution, and are therefore beneficial. However, a high optical path delay generally implies a greater total thickness of liquid crystal. To keep the driving voltages and response times low, a single thick liquid crystal cell can be broken up into multiple cells in series, as shown in
The liquid crystal response time for an individual cell at a given voltage scales as the square of the cell thickness, so two liquid crystal variable retarders with identical path delay, one consisting of a single cell, and one split into two cells, would have a switching time differing by a factor of 4. Conversely, if the switching time is held constant, the two cells would differ in switching voltage by a factor of 4. The liquid crystal variable retarder embodiment of
If one layer or cell of a multilayer liquid crystal variable retarder has any symmetry-breaking features, where here a layer could be a subset of a cell or could be a whole cell, these features should be alternated or opposed between layers such that the stack as a whole retains favorable symmetry properties. Such symmetry-breaking features can be considered to have a polarity, which refers generally to the notion of whether a symmetry-breaking feature is directed along or in opposition to a test direction. The polarity with which electrodes are connected to a voltage source is one such feature, wherein the notion of polarity is immediate. The liquid crystal alignment direction is another such feature, wherein the notion of polarity can be made concrete by considering the rubbing direction of the upper-most electrode of a liquid crystal cell as seen in a cross-sectional depiction of the cell in a plane that is parallel to the LC directors, such as
For a conventional antiparallel cell, all liquid crystal molecules tend to be oriented in the same direction, which corresponds to the rubbing direction of an electrode. Such a configuration has a first-order dependence of optical path delay on incident light angle as the incident angle deviates from the normal. If two antiparallel cells are stacked with opposite polarities such that their alignment directions oppose each other, then the first order dependencies of optical path delay on incident light angle are equal and opposite and hence cancel each other out. Therefore, by paying careful attention to the arrangement of polarities of symmetry-breaking features of stacks of liquid crystal cells, it is possible to maintain an incident light angle dependence of optical path delay of second order or higher, as well as other advantageous operating characteristics.
It becomes more critical to shorten the response time of the LC variable retarder if the application demands taking multiple hyperspectral images in succession, as in a hyperspectral movie. Typically, LC variable retarder cells are switched on and then passively allowed to relax. In one embodiment, an LC cell is actively switched between a configuration with a maximal optical phase delay and a configuration with a minimal optical phase delay. This active switching may be implemented in many ways. In one embodiment, each electrode of the pair of electrodes that traditionally surround the LC material has been replaced with a pair of interdigitated electrodes.
Another aspect of response times is the selection of the LC material itself. When choosing an LC material, one has to balance multiple factors such as optical birefringence, dielectric anisotropy, and rotational viscosity. An LC material with high optical birefringence would result in thinner LC cells that achieve the same optical retardance as a thicker LC cell, with a benefit in lowered response time and/or driving voltage, partially offset by the typically increased rotational viscosity of such high birefringence materials.
An LC material with a high dielectric anisotropy would produce the same response as a lower dielectric anisotropy material but from a lower drive voltage. An LC material with a lower rotational viscosity would have a faster response time than a material with a higher rotational viscosity. As will be discussed in more detail later, material-dependent properties such as optical dispersion and temperature dependence of refractive index and rotational viscosity can be calibrated out of the system performance, but the LC material may still be selected to optimize the system performance post-calibration. In another embodiment, an LC material and/or LC cell preparation may be used to give the LC a larger ‘pretilt’ angle because this decreases the LC switching time.
In addition to fast response times, high viewing angles increase the usefulness of the hyperspectral imaging component. While hyperspectral imaging systems currently exist, many of them have limited viewing angle due to the angle-dependent properties of the optical filters employed. As used here, ‘viewing angle’ refers to the level of invariance of the optical phase delay for a given wavelength and a given state of the liquid crystal variable retarder with respect to deviations of the incident light angle from the normal of the LC variable retarder.
Typically, liquid crystal displays (LCDs) are designed as switchable half-wave plates between crossed polarizers that can alternate between light transmitting and light obscuring states. The term ‘viewing angle’ as used here differs from the conventional usage as applied to typical LCDs, which refers to the angle that a specific contrast ratio is reached between the on and off states. In the embodiments here, the LC variable retarder may function as a high-order waveplate. Because a single point in an image will be formed with a cone of light rays that has a non-zero numerical aperture (NA), each ray of the cone travels at a different incident angle through the LC variable retarder. Consider the difference in optical phase delay at a given wavelength between the ray with the most phase delay and the ray with the least phase delay within the cone of light rays that form a single image pixel. As this difference in phase delay for these two rays approaches π radians, the contrast of the interferogram recorded at this image pixel decreases.
High total optical phase delay is necessary to achieve high spectral resolution at a given wavelength; however, the average variation of the optical phase delay over the incident light angles of the rays corresponding to a single image pixel must be significantly less than π radians. The variation in optical phase delay as a function of angle is proportional to the total optical phase delay, so high spectral resolution imaging is challenging because it combines the above two conflicting requirements of high total optical phase delay with minimal phase delay variation as a function of angle. Therefore, in order to successfully obtain high spectral resolution HSI data, either the imaging NA has to be decreased to decrease the range of angles corresponding to the rays that form an image pixel, or the viewing angle of the liquid crystal variable retarder must be increased. Because decreasing the imaging NA decreases the optical throughput of the system, it is critical to develop techniques that increase the viewing angle to image with high spectral resolution while maintaining high optical throughput.
One particular embodiment of an LC cell that achieves an extended viewing angle uses parallel rubbing layers, known as a pi-cell or optically compensated bend (OCB) cell. The two alignment layers internal to the liquid crystal cell may be rubbed in parallel directions, shown as 72 in
Another embodiment, the double-nematic cell, involves stacking two conventional anti-parallel cells, such as 70 in
When light rays travel through a birefringent medium, they can undergo an effect referred to as ‘beam walk-off’ in which the wave vector and the Poynting vector are no longer parallel. The embodiment of two layers with opposed symmetry provides a remedy for this, because the walk-off of the first would be corrected by the walk-off of the second. In general, symmetry-preserving arrangements of LC cells in which walk-off is cancelled between two cells or two cell halves with opposite symmetry properties could correct this walk-off. IPS LC cells would not have a walk-off issue because beam walk-off is minimal when the wave vector is perpendicular or parallel to the LC director. If walk-off is not corrected, the image may drift as a function of retardance of the LC variable retarder, creating artifacts on the edges within the image once the Fourier transform was obtained. However, it may still be possible to correct this walk-off algorithmically, neglecting dispersive effects of the liquid crystal, by image registration techniques.
Having demonstrated different embodiments of the structure of the HSI component, the discussion now turns to additional elements and the methods of operating the HSI component, as well as its calibration.
The term voltages as used above refers to a vector where each element of the vector corresponds to one particular electrode potential with response to a reference voltage, such as system ground. In some embodiments, such a vector could be a vector of AC voltages characterized by phase, amplitude and frequency applied to each electrode. By using different frequencies the retardance controller can take advantage of the non-uniform frequency dependence of some liquid crystals, such as the frequency-dependent dielectric anisotropy. The camera processor also can trigger the focal plane array 24 at the times of the retardance time series to sample the focal plane array detector 24 at 84, to produce a series of images at the retardances commanded by the processor.
The retardance extractor 27 can extract the actual retardance at each point in time from either information in the series of images from the FPA 24, a series of partial images from the FPA, selected pixel values from the FPA, single pixel values from the FPA, or from signals from the photodetector 47 or other light sensor. Retardance extraction can be performed following the method as described in Mitsuo Takeda, Hideki Ina, and Seiji Kobayashi, “Fourier-transform method of fringe-pattern analysis for computer-based topography and interferometry,” J. Opt. Soc. Am. 72, 156-160 (1982) to extract the phase delay at each point in the recorded interferograms. The system then has a series of images at known retardances, which the processor 28 then processes to produce the HSI data at 86 to send to the requesting application on the host device or on the camera. Alternatively, if the retardance controller has high enough accuracy, no need exists for the retardance extractor. This is because the acquisition of each image has been synchronized with the timing of the retardance controller and the retardance of each image within the series is sufficiently close to that commanded by the camera processor 28.
Some liquid crystal materials experience a change in sign of the dielectric anisotropy at a certain driving frequency. Driving the material below this frequency causes the molecules to align parallel or perpendicular to the driving electric field, and driving above this frequency causes the molecules to align at 90 degrees with respect to their alignment at the lower frequency. This feature can be used in an active on and active off embodiment by switching the liquid crystal material between two orientations with the choice of drive frequency.
Many characteristics of the hyperspectral imaging system may require calibration. For example, the calculated wavelength of a monochromatic source as a function of position over the FPA may not appear uniform, because the LC cells of the LC variable retarder may have a non-uniform thickness, and there is also a dependence of calculated wavelength on angle of the chief ray at each pixel position. A calibration would take this information into account so that a processed HSI dataset of a monochromatic source, such as a laser, would show a spectral peak at the same wavelength in all image pixels. For example, a laser source may have a wavelength of 532 nanometers. Directing light from the laser source through the HSI component to an FPA and then determining the peak wavelength at a certain point on the FPA by obtaining HSI data of the light from the laser source may result in a peak being detected at 540 nanometers. The application software could be programmed to adjust for this offset. Due to the smoothly varying nature of this offset as a function of position, the calibration process may be performed at a few points or pixel-binned regions in the image plane and then interpolated across the entire image plane, or it could be performed individually at all pixels.
In addition, the index of refraction/dispersion of the LC material, as well as its rotational viscosity and other material parameters, may vary as a function of temperature, and there may be some hysteresis inherent in the switching process. Therefore, the retardance controller should be calibrated and optimized to provide the correct voltages to the electrodes of the LC cells within the LC variable retarder as a function not only of time but also of imaging speed and operating temperature, etc. This calibration of the retardance controller or the above calibration of spectral offsets may be assisted by pointing the HSI component at a fluorescent light bulb or other light source that has multiple known spectral lines. This would be especially useful when optical dispersion is present and multiple spectral peaks are needed to estimate the dispersion.
Referring back to
In the calibration process, the light sources included as part of the device may be used as the calibration light source. The sources should have known spectral characteristics that allow adjustment of the various performance characteristics of the hyperspectral system based upon those characteristics, such that the calculated spectrum at each image pixel accurately reflects the known spectrum of the calibrated source with the highest possible spectral resolution.
The retardance controller is in theory capable of knowing the current state of the LC variable retarder. For example, the LC variable retarder can be initialized in a given state on power-up, or at any point during operation. One possible way to initialize the LC variable retarder is to drive it at a voltage high enough to quickly bring it into equilibrium in its minimum or maximum retardance state. In order for the retardance controller to go from input, such as retardance at specified times, to output, voltages versus time, the retardance controller can use a look-up table, physical model, heuristic algorithm, etc. as a reference. It can also take into account the starting retardance of the liquid crystal variable retarder.
An important aspect of the retardance controller is its ability to produce the correct voltage series for a given retardance series, even if the time series describing when the retardances should be achieved differs. The output of the retardance controller therefore must have a functional dependence on both the retardance series and the time series, not just the retardance series. One should note that the time series is generally ordered but the retardance series is not. If the time intervals between adjacent retardance series members is longer than the liquid crystal variable retarder's relaxation time, the system is said to be adiabatically driven and the voltage series should be relatively independent of the input time series. However, if the time intervals between adjacent retardance series members is less than the liquid crystal variable retarder's relaxation time, the system is said to be dynamically driven because the required voltage series depends strongly on the time series. In other words, the retardance controller is a dynamic retardance controller.
The calibration could also be updated during normal use as in
One should note that the HSI system may use calibration data from other sources, rather than generating the calibration data itself. While the calibration data generated in the above self-calibration process may be stored in the memory of the HSI system, the memory may also store calibration data provided with the system, available from other sources, etc. There is no limitation intended nor should any be assumed that the only calibration data available is if the system performs the above-described processes.
In some embodiments it may be advantageous to perform pixel binning, whereby the intensity values recorded at neighboring pixels in a region of the FPA are summed together either directly on the FPA or later in software. If the pixels are binned together at the FPA before they are read out, it is generally possible to increase the frame rate of the FPA. This assumes a fixed maximum communication speed between the FPA and the device that records the data from the FPA, such as the camera processor 28. Varying the number of pixels binned thus allows one to trade-off between spatial resolution and imaging speed, and since there is also a tradeoff between imaging speed and spectral resolution, pixel binning is yet another method to trade off between spectral and spatial resolution and imaging speed. In addition, pixel binning may be important for increasing the signal to noise ratio of an image, especially when signals are weak and minimal spatial resolution is needed. In an extreme limit, all pixels could be binned together, and the embodiment would function as a non-imaging Fourier spectrometer.
Pixel binning may be particularly useful for calibration of various smoothly-varying position-dependent quantities of the FPA such as the position-dependent variation in optical phase delay at a given wavelength and state of the liquid crystal variable retarder. In order to get reliable calibration information it may be necessary to achieve a high signal to noise ratio, whereas not much spatial resolution would be needed because of the smoothly varying nature of the quantities needing calibration. Calibration information across the field of view of the HSI system could be interpolated from the measured results.
The retardance controller can have the ability, in addition to taking into account different time series, to optionally account for different temperatures, because temperature affects the dynamic properties of the liquid crystal material such as dielectric anisotropy, rotational viscosity, and elastic constants, as well as affecting the birefringence. However, this is optional because if the melting point of the liquid crystal material is high enough relative to the operating temperature, the liquid crystal material will not be that sensitive to temperature. The controller can be fully calibrated in the factory and be fully open loop, and if this calibration is dependable enough, the phase reference, in this embodiment, the laser diode 20, is not needed, nor is the retardance extractor 27. However, even if the calibration is fixed in the factory, it may be helpful to have the phase reference to know the actual retardance of the acquired images.
If desired, the retardance controller can update itself between acquisitions of hyperspectral image data as described previously in reference to
At 110, the retardance controller 50 of
Other modifications and embodiments are possible. In order to decrease imaging time, for example, the system could include an optical bandpass filter, such as a Bayer filter, in front of the HSI component to restrict light hitting the FPA to a known spectral band, allowing subsampling of the interferogram without aliasing. An electronic digital or analog bandpass filter that filtered the signals recorded at each image pixel would achieve the same effect. An optical filter may also increase the spectral resolution if one is imaging a spectral feature that occurs near a sharp cutoff of a filter that has sharp cutoffs. As used here, a sharp cutoff, or transition between the passband and the stopband, is one that is sharper than the full width at half maximum of a spectral peak that would be obtained from a monochromatic source if the filter were not present. Such a filter would yield information as to whether the spectral peak occurred below or above the filter cutoff.
Other types of optical filters or optical components, such as different types of optical films, may also be employed in certain embodiments. For example, a retardance compensation device such as a film may be used to achieve a specific retardance in the “on” or “off” state of the LC variable retarder. The filter or film may also provide chromatic compensation to correct for the dispersive effects of the liquid crystal. Or, standard antireflection coatings may be used on or in the LC cells of the LC variable retarder.
The LC electrodes may consist of graphene, indium tin oxide, or other material with high conductivity and high optical transparency. Especially when considering the multilayer embodiments of the present invention, the light reflected or absorbed from the electrodes must be minimized, and it is therefore advantageous to maintain high optical transparency in the electrode layers.
Many of the embodiments discussed above have assumed the presence of the HSI component in the same system as a traditional camera, with the HSI component having a zero-retardance mode to allow the traditional camera to operate without obtrusion. However, it may be desirable to have the HSI component not in the final focal plane of the FPA nor in close proximity to it, but instead in a conjugate of the focal plane of the FPA and linked to the focal plane of the FPA by one or more sets of relay optics. This would enable an add-on module to be used with existing cell phones or cameras, whereby the add-on would be a hyperspectral component that would control or be controlled by, for example, an existing cell phone or camera, to synchronize the driving of the liquid crystal variable retarder with the acquisition of individual image frames.
Alternatively, in addition to making the device transparent to incoming light, it may also be desirable to make the polarizers completely transparent to incoming light. In this embodiment, when the camera operates as a normal, non-hyperspectral camera, the system does not lose light due to the polarizers. In one embodiment, the system employs a switchable polarizer that can switch between at least two states. One state nominally transmits only one polarization of light and absorbs, reflects or blocks the orthogonal polarization. The other state transmits both polarizations. One can make such a polarizer using anisotropic liquid crystal gels, as set out in H. Ren and S. T. Wu, “Anisotropic Liquid Crystal Gels for Switchable Polarizers and Displays”, Appl. Phys. Lett. 81, 1432-1434 (2002). These polarizers also have the desirable properties of low operating voltage, high contrast ratio, broad bandwidth, wide viewing angle, and fast response times.
Another modification that would apply to one or more of the liquid crystal cells in the liquid crystal variable retarder involves the flatness of these cells. The control of the state of the liquid crystal variable retarder across its clear aperture, that is, the portion of the liquid crystal variable retarder through which light passes to the focal plane array 24 or light sensor 47 of
Including spacers in the clear aperture provides one way to make the LC cells uniformly thick. However, the spacers displace the liquid crystal material and therefore they change the retardance variation of the liquid crystal variable retarder at the positions where they are present. This results in their appearance as artifacts in the hyperspectral image data. However, options exist to reduce the significance of these artifacts.
In one embodiment, the spacers have a high aspect ratio, such that the size or diameter of the occluded spot they produce as viewed along the imaging system's optical axis has a small size relative to the spacer height. This causes minimal disruption in the image relative to a low aspect ratio spacer.
In another embodiment, the liquid crystal cell, the liquid crystal cells, or the liquid crystal variable retarder can move away from the focal plane of the imaging optics within an HSI camera. In fact, the liquid crystal cells of the liquid crystal variable retarder can be individually placed at any location with respect to the imaging optics, as long as they are between the input polarizer and the output polarizer of the HSI system. The liquid crystal cells can be adjacent to each other or they can be separate. One or more can be in a focal plane of the imaging optics, a conjugate focal plane, a Fourier plane, between the imaging optics and the scenery to be imaged, between the imaging optics and the FPA, etc. As long as the cells have sufficiently uniform flatness, the rays that make up a given point in the image can traverse different points of the liquid crystal cells. As these points all have the same thickness within a given cell, these rays all experience the same retardance. Moving the cell away from the focal plane causes a blurring of the occluded spots caused by any spacers within the clear aperture of the cell.
In
Another option with regard to spacers and the focal plane array involves building the spacers between the pixels of the focal plane array, as shown in
Another embodiment involves the use of an LC cell that has uniform thickness but is not itself flat. For wide field of view imaging systems, the central ray of a given field point may enter the hyperspectral camera at a very oblique angle, as shown in
As discussed previously, advantages may exist in illuminating the focal plane array with a monochromatic light source or calibration light source, such as that provided by the laser diode 20 in
The illumination geometry in
In
Yet another modification involves the control of the liquid crystal variable retarder. As dynamic control of the liquid crystal material can be problematic, it may be advantageous to drive the retarder adiabatically but in a way that allows controlled changes in retardance much faster than the retarder's passive relaxation time. This can be done by significantly decreasing the relaxation or response time of the retarder through the proper choice of voltage waveforms. The response time of the liquid crystal variable retarder can be made very short if the driving voltages are high enough. Therefore it may be useful to drive the liquid crystal variable retarder with a two-component electric field.
As used here, a two-component field has a first component, or a perpendicular component, which causes the liquid crystal material to align perpendicular to the cell's layers and a second component, or a parallel component that causes the liquid crystal to align parallel with the cells' layers. These components can be generated, for example, by combinations of electrode driving potentials to create perpendicular and parallel electric fields, as in
If only a parallel (or perpendicular) electrical field were used, then the first component EA would be the field amplitude at frequency f1 (f2), and the second component EB would be the field amplitude at frequency f2 (f1). Alternatively, one can use only frequency f1 (or only frequency f2) for the electric field. In this case, component EA would be the parallel (or perpendicular) field and component EB would be the perpendicular (or parallel) field. The liquid crystal director orientation 190 will adiabatically track the angle theta (A) given by arctan(EA/EB), and the relaxation time to this angle is given by τ=(E2AE2B)) trelax, where trelax is the passive (field-off) relaxation time of the liquid crystal cell and Ec is the critical field response to create a finite distortion at the mid-point of the cell. Orientation changes of the LC director can be made arbitrarily fast by ensuring the relaxation time τ is much faster than any time between orientation changes of the liquid crystal director. This can be accomplished by making the field components EA and EB large enough, but with the correct ratio to reach the desired liquid crystal director orientation and hence the desired retardance of the liquid crystal variable retarder. More generally, one component of a two component field can apply positive torque to rotate the liquid crystal molecules within a cell of the liquid crystal variable retarder to increase the retardance of that cell, and the other component can apply negative torque to decrease the retardance. Simultaneous application of both components causes the liquid crystal molecules to come to equilibrium at an angle set by the ratio of the components, at a rate determined by the magnitude of the components. Thus, a two-component field in principle allows arbitrary control of the director of the liquid crystal molecules and hence arbitrary control of the retardance of a liquid crystal variable retarder.
Other considerations with regard to faster drive times lie in the selection of the liquid crystal material itself. Ferroelectric liquid crystals or polymer network liquid crystals have faster response times. A polymer network liquid crystal material has liquid crystal material embedded with a polymer network.
The methods and devices above may be employed by one of several applications, such as medical imaging, sorting, spectroscopy of material discovered in the field, etc. Each of these may have its own software program, typically referred to as an ‘app’ in the smartphone world. As mentioned previously, an HSI system may be integrated into many different types of host devices, such as phones, tablets, etc., as well as more traditional lab equipment like microscopes and telescopes.
One specific application that may be enabled by this system is the ability of a user to use the HSI system as a medical diagnostics device, for example for colorimetric readout of home medical diagnostic tests. The user could take one or more HSI datasets of body locations and/or medical diagnostic testing strips. The resultant HSI datasets can be fully or partially processed into diagnostic information using the system's included processing power, or they can be processed in the cloud. With the included communications link, the diagnostic information can ultimately be forwarded to a doctor or laboratory.
It will be appreciated that several of the above-disclosed and other features and functions, or alternatives thereof, may be desirably combined into many other different systems or applications. Also that various presently unforeseen or unanticipated alternatives, modifications, variations, or improvements therein may be subsequently made by those skilled in the art which are also intended to be encompassed by the following claims.
Claims
1-35. (canceled)
36. A hyperspectral imaging system, comprising:
- a processor configured to receive hyperspectral imaging parameters and to determine a series of retardances at a series of retardance times based on the hyperspectral imaging parameters;
- a hyperspectral imaging component comprising: at least one input polarizer, wherein the input polarizer receives and polarizes an incoming beam of light; a liquid crystal variable retarder arranged adjacent the input polarizer opposite the incoming beam of light to receive the polarized beam of light from the input polarizer and to change polarization of the light to produce wavelength-dependent polarized light; an output polarizer arranged to receive the wavelength-dependent polarized light and to convert polarization state information of the light into a form detectable as light intensity; and a retardance controller configured to compute a voltage series that controls the retardance of the liquid crystal variable retarder;
- a focal plane array synchronized with the retardance controller and configured to receive the light in a form detectable as light intensity as a function of retardance of the liquid crystal variable retarder and to convert the light to an electrical output signal that represents a series of images, the processor configured to perform transformations of the series of images to hyperspectral image data; and
- wherein the liquid crystal variable retarder comprises a liquid crystal cell having spacers between substrate layers of the cell, the spacers disposed within a clear aperture of the liquid crystal cell.
37. The hyperspectral imaging system of claim 36, wherein the processor resides in a host system and the hyperspectral imaging component resides in an add-on module to the host system.
38. The hyperspectral imaging system of claim 36, wherein the focal plane array resides in a host system and the hyperspectral imaging component resides in an add-on module to the host system.
39. The hyperspectral imaging system of claim 36, further comprising imaging optics arranged between the incoming beam of light and the focal plane array.
40. The hyperspectral imaging system of claim 36, further comprising at least one light source arranged adjacent to the at least one input polarizer such that light from the light source reaches the focal plane array.
41. The hyperspectral imaging system of claim 40, further comprising a photodetector arranged adjacent to the focal plane array and positioned to receive light from the at least one light source through the liquid crystal variable retarder.
42. The hyperspectral imaging system of claim 36, further comprising a waveplate between the at least one input polarizer and the output polarizer.
43. The hyperspectral imaging system of claim 36, further comprising a retardance compensation device between the at least one input polarizer and the output polarizer.
44. The hyperspectral imaging system of claim 36, further comprising a retardance extractor.
45. The hyperspectral imaging system of claim 36, further comprising at least one temperature sensor that provides temperature information of the liquid crystal variable retarder to the retardance controller, wherein the temperature sensor comprises a patterned electrode of a liquid crystal cell.
46. The hyperspectral imaging system of claim 36, further comprising a clock connected to the processor and the retardance controller.
47. The hyperspectral imaging system of claim 36, wherein at least one of the polarizers is switchable between a first state that transmits one polarization and blocks an orthogonal polarization, and a second state that transmits the one polarization and the orthogonal polarization.
48. The hyperspectral imaging system of claim 39, wherein the liquid crystal cell resides away from the focal plane array such that rays of light that pass through the imaging optics experience a same thickness of the liquid crystal cell before striking the focal plane array and are blurred in an area in which the spacers occlude the light.
49. The hyperspectral imaging system of claim 36, wherein the at least one input polarizer comprises a polarizing beam splitter.
50. The hyperspectral imaging system of claim 49, further comprising a second liquid crystal variable retarder, a second output polarizer and a second focal plane array, arranged with respect to the polarizing beam splitter, wherein image light entering the polarizing beam splitter through a first port has a first polarization direction and image light entering the polarizing beam splitter through a second port has a second polarization.
51. The hyperspectral imaging system of claim 49, further comprising a calibration light source, wherein light from the calibration light source is directed to an entrance port of a polarizing beam splitter with polarization orthogonal to that of an entrance port pointing at scenery to be imaged.
52. The hyperspectral imaging system of claim 36, wherein the liquid crystal variable retarder comprises one of a ferroelectric liquid crystal material or a polymer network liquid crystal.
53. The hyperspectral imaging system of claim 36, wherein the voltage series is configured to provide a two-component electric field, a first component of the two-component electric field causing liquid crystal material of the liquid crystal variable retarder to align substantially perpendicular to layers of the liquid crystal cell and a second component of the two-component electric field causing the liquid crystal material to align substantially parallel to the layers of the liquid crystal cell.
54. A method of producing hyperspectral image data, comprising:
- receiving a set of hyperspectral imaging parameters at a processor;
- generating, with the processor, a series of retardances at a series of retardance times corresponding to the hyperspectral imaging parameters;
- sending the series of retardances at the series of retardance times to a retardance controller;
- generating, with the retardance controller, a series of voltages at a series of voltage times to be applied to a liquid crystal variable retarder;
- applying the series of voltages at the series of voltage times to a liquid crystal variable retarder;
- capturing a series of images through the liquid crystal variable retarder with a focal plane array; and
- using the processor to generate hyperspectral image data from the series of images.
55. The method of claim 54, wherein applying the series of voltages to the liquid crystal variable retarder comprises applying a two-component field.
Type: Application
Filed: Aug 20, 2020
Publication Date: Dec 3, 2020
Inventors: Alex Hegyi (San Francisco, CA), Joerg Martini (San Francisco, CA)
Application Number: 16/998,870