HYPERSPECTRAL IMAGE CONSTRUCTION OF BIOLOGICAL TISSUE FOR BLOOD HEMOGLOBIN ANALYSIS USING A SMARTPHONE

Abstract

A bloodless system for numerically generated hyperspectral imaging data for measuring biochemical compositions is disclosed which includes an optical imaging device adapted to acquire an RGB image from an area of interest, a processor adapted to receive a hyperspectral dataset representing an a priori hyperspectral data of the area of interest of a population to which the subject belongs, receive RGB response for each one of RGB channels of the optical imaging device, pair the corresponding RGB data with the hyperspectral data, obtain a transformation matrix adapted to convert a subject-specific RGB image dataset into a subject-specific hyperspectral dataset for the optical imaging device, receive a subject- specific RGB dataset, generate a subject- specific hyperspectral dataset using the transformation matrix, and compute a blood hemoglobin level of the subject from the generated subject-specific hyperspectral dataset.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

The present patent application is related to and claims the priority benefit of U.S. Provisional Patent Application Serial No. 62/945,816, filed Dec. 09, 2019, titled "HYPERSPECTRAL IMAGE CONSTRUCTION OF BIOLOGICAL TISSUE FOR BLOOD HEMOGLOBIN ANALYSIS USING A SMARTPHONE"; and U.S. Provisional Patent Application Serial No. 62/945,808 filed Dec. 09, 2019, titled "VIRTUAL HYPERSPECTRAL IMAGING OF BIOLOGICAL TISSUE FOR BLOOD HEMOGLOBIN ANALYSIS", the contents of each of which are hereby incorporated by reference in its entirety into the present disclosure.

STATEMENT REGARDING GOVERNMENT FUNDING

This invention was made with government support under R21TW010620 awarded by the National Institutes of Health and 7200AA18CA00019 awarded by the US Agency for International Development. The government has certain rights in the invention.

TECHNICAL FIELD

The present disclosure generally relates to generating a hyperspectral imaging dataset, recovering hyperspectral information from RGB values, analyzing blood, and in particular, to a system and method of analyzing biological tissue for blood hemoglobin analysis.

BACKGROUND

This section introduces aspects that may help facilitate a better understanding of the disclosure. Accordingly, these statements are to be read in this light and are not to be understood as admissions about what is or is not prior art.

Blood hemoglobin (Hgb) tests are routinely ordered as an initial screening of the amount of red blood cells (hemoglobin) in the blood as part of a general health test for a subject. Blood Hgb tests are extensively performed for a variety of patient care needs, such as anemia detection as a cause of other underlying diseases, hemorrhage detection after traumatic injury, assessment of hematologic disorders, and for transfusion initiation. There are several biological assays for measuring Hgb content in grams per deciliter (i.e. g dL^-1) from blood drawn via traditional needle-based methods. Portable point-of-care hematology analyzers using blood draws (e.g. Abbott i-STAT and HemoCue) are also commercially available. However, all these tests require expensive and environment-sensitive analytical cartridges with short shelf lives, as well as unaffordable for both resource-limited and homecare settings. In addition, repeated blood Hgb measurements using these invasive tests can cause iatrogenic complications such as blood loss.

Unlike measuring oxygen saturation with pulse oximetry, noninvasive measurements of a total Hgb concentration in the blood are not straightforward. A few noninvasive Hgb testing devices have recently become available that are currently undergoing clinical studies for immediate reading and continuous monitoring of blood Hgb levels in different clinical settings. Aside from the relatively high cost associated with operating and maintaining the equipment, the medical community agrees that the broad limits of agreement between these devices and central laboratory tests pose a significant challenge in making clinical decision, thus generating skepticism in clinical adaptation. Several smartphone-based anemia detection technologies have also made progress, however, most of these mobile applications are intended for initial screening or risk stratification of severe anemia and are not developed for measuring exact Hgb content in the unit of g dL^-1.

Therefore, there is an unmet need for a novel approach that can provide continuous and on-demand Hgb measurements that can be relied for accuracy without the complications associated with expensive laboratory equipment.

SUMMARY

A bloodless system for numerically generated hyperspectral imaging data for measuring biochemical compositions is disclosed. The system includes an optical imaging device adapted to acquire an RGB image from an area of interest, thereby generating a subject-specific RGB dataset representing the area. The system also includes a processor. The processor is adapted to receive a hyperspectral dataset representing an a priori hyperspectral data of the area of interest of a population to which the subject belongs. The processor is further adapted to receive RGB response for each one of RGB channels of the optical imaging device. Additionally, the processor is adapted to pair the corresponding RGB data with the hyperspectral data. Furthermore, the processor is adapted to obtain a transformation matrix adapted to convert a subject-specific RGB image dataset into a subject-specific hyperspectral dataset for the optical imaging device. Also the processor is adapted to receive a subject-specific RGB dataset, generate a subject-specific hyperspectral dataset using the transformation matrix, and compute a blood hemoglobin level of the subject from the generated subject-specific hyperspectral dataset.

According to one embodiment of the present disclosure, in the system the paired pixels from the RGB image is associated with a 3×1 RGB value matrix.

According to one embodiment of the present disclosure, in the system the paired pixels from the hyperspectral dataset is associated with an N×1 spectrum, where N represents discretized spectra between a lower bound and an upper bound.

According to one embodiment of the present disclosure, in the system the lower and upper bounds are 400 nm and 800 nm, respectively.

According to one embodiment of the present disclosure, in the system the transformation matrix is a form of inverse of the RGB response matrix of the RGB sensor that converts an RGB to a spectrum.

According to one embodiment of the present disclosure, in the system the inverse of the transformation matrix is determined numerically by using the paired RGB and hyperspectral data of the population.

According to one embodiment of the present disclosure, in the system the biochemical compositions include blood hemoglobin.

According to one embodiment of the present disclosure, in the system the area of interest includes the inner surface of a subject’s inner eyelid.

According to one embodiment of the present disclosure, in the system the biochemical compositions are determined using spectral analysis.

According to one embodiment of the present disclosure, in the system the spectral analysis includes a partial least square regression statistical modeling technique to first build a model from a training set of a first hyperspectral dataset vs. the biochemical compositions and then apply the model to a second dataset from the generated hyperspectral image dataset.

A method for a bloodless numerically generated hyperspectral imaging data for measuring biochemical compositions is also disclosed. The method includes obtaining an RGB image from an area of interest, thereby generating a subject-specific RGB dataset representing the area of interest. The method also includes receiving a hyperspectral dataset representing an a priori hyperspectral data of the area of interest for a population to which the subject belongs and receiving an RGB response for each one of RGB channels of the optical imaging device. The method also includes pairing the corresponding RGB data with the hyperspectral data, and obtaining a transformation matrix adapted to convert a subject-specific RGB image dataset into a subject-specific hyperspectral dataset for the optical imaging device. Furthermore, the method includes generating a subject-specific hyperspectral dataset using the transformation matrix; and computing a blood hemoglobin level of the subject from the generated subject-specific hyperspectral dataset.

According to one embodiment of the present disclosure, in the method the paired pixels from the RGB image is associated with a 3×1 RGB value matrix.

According to one embodiment of the present disclosure, in the method the paired pixels from the hyperspectral dataset is associated with an N×1 spectrum, where N represents discretized spectra between a lower bound and an upper bound.

According to one embodiment of the present disclosure, in the method the lower and upper bounds are 400 nm and 800 nm, respectively.

According to one embodiment of the present disclosure, in the method the transformation matrix is a form of inverse of the RGB response matrix of the RGB sensor that converts an RGB to a spectrum.

According to one embodiment of the present disclosure, in the method the inverse of the transformation matrix is determined numerically by using the paired RGB and hyperspectral data of the population.

According to one embodiment of the present disclosure, in the method the biochemical compositions include blood hemoglobin.

According to one embodiment of the present disclosure, in the method the area of interest includes the inner surface of a subject’s inner eyelid.

According to one embodiment of the present disclosure, in the method the biochemical compositions are determined using spectral analysis.

According to one embodiment of the present disclosure, in the method the spectral analysis includes a partial least square regression statistical modeling technique to first build a model from a training set of a first hyperspectral dataset vs. the biochemical compositions and then apply the model to a second dataset from the generated hyperspectral image dataset.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1a is a simplified block diagram depicting the major blocks of a system of the present disclosure.

FIG. 1b is a combination of an algorithm of the present disclosure, imaging of the inner eyelid, and spectroscopic quantification of blood hemoglobin (Hgb) content which combination offers a bloodless spectrometer-free hematology analyzer using a smartphone.

FIG. 2 is a block diagram which provides steps of using image data in order to numerically generate a hyperspectral image in order to estimate a blood Hgb level of a subject.

FIG. 3 is a histogram which summarizes the blood Hgb values of a total of 153 individuals that were used for spectroscopic and blood Hgb measurements using the algorithm of the present disclosure.

FIG. 4 provides blood hemoglobin vs. wavelength diagrams along with 95% confidence diagrams showing comparisons between original hyperspectral dataset (acquired by an image-guided hyperspectral system) and the those generated based on the algorithm of the present disclosure (i.e., constructed hyperspectral datasets for both training and testing groups.

FIG. 5 is a collection of graphs of a linear correlation between the computed blood Hgb content and the laboratory blood Hgb levels and differences in blood hemoglobin in g dL-1 for one subset of the population of individuals (138) used as training data as well as a second population of individuals (15) used as testing data.

FIGS. 6a and 6b provide receiver operating characteristic (ROC) curves of mHematology performance for anemia assessment with a cutoff < 12 g dL-1 for females and a cutoff < 13 g dL-1 for males, respectively.

FIGS. 7a and 7b are diagrams of blood Hgb prediction using RGB information based on the algorithm of the present disclosure vs. blood Hgb found by other means as well as a plot showing Bland-Altman analysis of the data.

DETAILED DESCRIPTION

For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the embodiments illustrated in the drawings, and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of this disclosure is thereby intended.

In the present disclosure, the term "about" can allow for a degree of variability in a value or range, for example, within 10%, within 5%, or within 1% of a stated value or of a stated limit of a range.

In the present disclosure, the term "substantially" can allow for a degree of variability in a value or range, for example, within 90%, within 95%, or within 99% of a stated value or of a stated limit of a range.

For noninvasive blood hemoglobin (Hgb) measurements, it is important to rely on an appropriate anatomical sensing site where the underlying microvasculature is exposed on the skin surface without being affected by confounding factors of skin pigmentation and light absorption of molecules (e.g. melanin) in tissue. Commonly used clinical examination sites of pallor or microcirculation, such as the conjunctiva, the nailbed, the palm, and the sublingual region, provide a clue for an examination site selection. Specially, the palpebral conjunctiva (i.e. inner eyelid) can serve as an ideal site for peripheral access, because the microvasculature is easily visible and melanocytes are absent. The easy accessibility of the inner eyelid allows for reflectance spectroscopy and digital photography to be tested for anemia assessments.

While for blood Hgb quantifications, spectroscopic analyses of light absorption of Hgb in reflection spectra can be used to measure Hgb content in tissue, such methods rely heavily on complex and costly optical instrumentation such as spectrometers, imaging spectrographs, mechanical filter wheels, or liquid crystal tunable filters which can also result in significantly slow data acquisition, hampering clinical translation. While in the sister patent application referred to in the CROSS-REFERENCE TO RELATED APPLICATIONS section of the present disclosure to which the present disclosure claims priority, a method and system is disclosed whereby a simplified hyperspectral imaging device capable of generating a linescan dataset, such imaging apparatus remains unattainable in certain areas of the world, particularly in developing countries. Towards this end, the method and system of the present disclosure, provides a mathematical solution to construct hyperspectral with high spectral resolution or multispectral with several spectral measurements data from RGB images taken using a conventional camera (i.e. three-color sensors). This data-driven approach lays the groundwork for computational spectroscopy that overcomes the aforementioned hardware limitations.

Therefore, according to the present disclosure, we introduce a system and method referred to herein as the Virtual Hyperspectral Image Construction (VHIC) for a noninvasive blood Hgb measurements, with results that are relatively comparable to clinical laboratory blood Hgb tests.

Referring to FIG. 1a, a simplified block diagram is shown depicting the major blocks of a VHIC system 10 of the present disclosure. The system 10 includes a dataset 12 which includes a priori information representative of hyperspectral dataset of the eyelids of a population of interest. This database can also be acquired by a conventional hyperspectral imager or by numerical simulations (modeling or Monte Carlo simulations). Referring to FIG. 1a, the VHIC system 10 also includes a color (RGB) imaging apparatus 14 capable of generating a red-green-blue (RGB) image of an area (or region) of interest (ROI). The hyperspectral data and RGB data are combined by a processing system (not shown) but partially represented as a summer 16 which produces a matrix of intensity as a function of the position (x, y) without using a conventional hyperspectral imaging system. Using the a priori hyperspectral dataset and an RGB dataset from, a transformation (or extrapolation) algorithm is used to construct a hyperspectral image, all of which is represented by the summer 16 by a fixed design linear regression with polynomial features to build a construction matrix to generate hyperspectral data from the RGB image (new dataset). The aforementioned transformation algorithm is then applied to generate a hyperspectral image dataset for a partial portion or all of the ROI by the processing system (not shown), as represented by the block 18.

Referring to FIG. 1b, a combination of VHIC, imaging of the inner eyelid, and spectroscopic quantification of blood Hgb content are shown which combination offers a bloodless spectrometer-free hematology analyzer using a smartphone - (referring to herein also as mHematology). In VHIC, reflection hyperspectral data (or multispectral data) in the visible range are mathematically constructed from an RGB image of the inner eyelid acquired using a smartphone camera. In other words, the VHIC methods of the present disclosure virtually transforms a smartphone camera into a hyperspectral imager without any accessory attachment. The inner eyelid (i.e. palpebral conjunctiva) is used as an accessible sensing site for noninvasive blood Hgb quantification. An RGB image of the inner eyelid is conveniently captured using the camera of a smartphone. The subject simply pulls down on the eyelid to expose the conjunctiva (inner eyelid) and the user takes a photograph of the inner eyelid, by herself or by the assistance of another person. The method of the present disclosure then collects red (R), green (G), and blue (B) color information from the eyelid image and applies the methods associated with VHIC to a mathematical construct reflection hyperspectra in the visible wavelength range. The hyperspectral data of the inner eyelid is sensitive to changes in Hgb content in the blood. The constructed hyperspectral data of the acquired eyelid image is then processed to accurately and precisely estimate the amount of total blood Hgb content. The result displays the blood Hgb count in the unit of g dL^-1, which are found to be comparable to laboratory Hgb tests.

A beta version of mHematology application is developed for data acquisition in low-end smartphones of SAMSUNG GALAXY J3 to build a robust mobile platform for all smartphones regardless of RGB image qualities. On the main application screen, the mHematology application displays a circle and arc to serve as guidance for locating the eyeball and the inner eyelid at a consistent distance and position within the image. To remove the background room light, the application automatically acquires two RGB photographs by controlling the built-in flashlight (i.e. white-light LED) to turn on and off. To compensate for the system response, two RGB images of a reflectance standard are taken. Similarly, the application automatically takes two RGB images with flash on and flash off for the individual's exposed eyelid.

Our data-driven computational spectroscopy provides the following advantages: the inner eyelid is used as a peripheral sensing site (vs. fingertip or fingernail) with the aforementioned advantages, hyperspectral image data construction vs. mere RGB images, spectroscopic analysis of Hgb (vs. empirical approach), and built-in camera in a smartphone (vs. costly accessory attachment). The two-step algorithm for blood Hgb estimation according to the present disclosure includes a first step which is to apply VHIC to the eyelid portion of the RGB image. The methodology then uses fixed design linear regression with polynomial features to build a construction matrix for the hyperspectral data from the RGB image.

Referring to FIG. 2, a block diagram is shown which provides steps of using image data in order to numerically generate a hyperspectral image in order to estimate a blood Hgb level of a subject. First, the VHIC algorithm uses a priori representative hyperspectral dataset of the eyelids of a population of interest. In addition, the VHIC uses the information on the RGB response functions (i.e., spectral responsivity functions of the image sensor for each RGB channel) of the camera to be used, which can be directly obtained from the image sensor manufacturer. An example of an RGB image sensor (e.g., SONY ICX 625) is shown in a panel of FIG. 2 in the form of a graph of relative intensity as a function of wavelength in nm, in which the spectral response functions of the image sensor for each of the RGB channels is provided. Then, the RGB responses are applied to the hyperspectral dataset (a priori hyperspectral representation of the population of interest) to generate a corresponding RGB dataset of the eyelids of the population of interest that would be used to compare to the image that would be obtained by the same camera. By pairing the hyperspectral data and the RGB data of the population of interest, a transformation (extrapolation) matrix is obtained to convert subject-specific RGB image data into subject-specific hyperspectral data. The transformation matrix can be fine-tuned for the specific image sensor of the camera to be used. This is a one-time hyperspectral-to-RGB transformation dataset for the population of interest which can be held in memory. Second, after the VHIC refinement, an RGB image of the inner eyelid of a patient of interest from the population of interest that is taken by the camera is fed into the VHIC algorithm. Third, by applying the transformation matrix to the subject-specific RGB dataset, the VHIC then generates the subject-specific hyperspectral data. Using the generated subject-specific hyperspectral dataset, the blood Hgb content can then be computed. Specifically, to compute blood Hgb content in g dL^-1, a spectral analysis of Hgb is performed as discussed in the sister patent application listed in the CROSS-REFERENCE TO RELATED APPLICATIONS section of the present disclosure to which the present disclosure claims priority. The constructed hyperspectral reflection data of the eyelid is analyzed using a partial least squares regression (PLSR) model to predict a blood Hgb value, which can be validated by the laboratory blood Hgb tests (which is viewed as the gold standard)

In order to make comparison with clinical data, reference is made to FIG. 3 which summarizes the blood Hgb values of a total of 153 individuals that were used for spectroscopic and VHIC blood Hgb measurements (Table 1). In order to mimic the a prior hyperspectral dataset for a population of interest, we randomly selected 138 individuals (78 females and 60 males). This dataset is referred to as a training dataset. In order to test the algorithm and thus mimic actual patients to be tested, we use the rest of 15 individuals (12 females and 3 males) not included in the testing dataset. This independent new dataset is referred as a testing dataset. The average Hgb level of the training dataset is 12.65 g dL^-1 with a standard deviation (SD) of 3.11 g dL^-1 and the average age is 37.78 years with SD of 16.38 years. The average Hgb level of the testing dataset is 11.06 g dL^-1 with SD of 3.62 g dL^-1. The average age is 39.13 years with SD of 17.30 years. Overall, the study covers a wide range of Hgb values from 3.3 to 19.2 g dL^-1. We conducted a clinical study within the facilities overseen by the accepted authorities. We enrolled patients who were referred for complete blood count (CBC) tests. For all individuals enrolled in the study, we collected hyperspectral data and RGB images from the palpebral conjunctiva (i.e. inner eyelid) using an image-guided hyperspectral line-scanning system and a mobile application in low-end Android phones (SAMSUNG GALAXY J3). As the ‘gold standard’ clinical laboratory measurements, blood Hgb levels were measured in an accredited clinical laboratory using a commercial hematology analyzer (BECKMAN COULTER AcT 5diff auto, BECKMAN COULTER, INC.) with blood draws.

Our spectroscopic and VHIC blood Hgb measurements are not affected by variations in the illumination and detection of the imaging systems as well as the background ambient room light as follows: The spectral intensity I_m(λ) reflected from the inner eyelid in a given location of (x, y) is expressed as a function of the wavelength λ:

$I_m\left(\lambda \right)=L\left(\lambda \right)C\left(\lambda \right)D\left(\lambda \right)r\left(\lambda \right)$

where L(λ) is the spectral shape of the illumination light source, C(λ) is the spectral response of all optical components in the imaging system (e.g. lenses and diffraction grating), D(λ) is the spectral response of the detector (e.g. mono sensor in the image-guided hyperspectral line-scanning system or three-color RGB sensor embedded in the smartphone), and r(λ) is the true spectral intensity reflected from the inner eyelid. First, to compensate for the system response (i.e. L(λ)C(λ)D(λ)), we use the reference reflectance standards that have a reflectivity of 99% in the visible range. I_m(λ) is normalized by the reflectance I_reference(λ) of the diffuse reflectance standard in which r_reference(λ) = 0.99 in the visible range

$r\left(\lambda \right)=\frac{I_m\left(\lambda \right)}{I_{reference}\left(\lambda \right)}$

Second, to remove the ambient stray and background light I_background(λ), two measurements are acquired with the external light source (i.e. built-in flashlight LED of the smartphones) on and off. The measurements are repeated without the sample while the illumination is kept on. Finally, r(λ) is calculated by subtracting I_background(λ) from each measurement such that:

$r\left(\lambda \right)=\frac{I_m\left(\lambda \right)-I_{background}\left(\lambda \right)}{I_{reference}\left(\lambda \right)-I_{background}\left(\lambda \right)}$

This systematic and rigorous data acquisition procedure serves as the foundation for developing a reliable and universal blood Hgb computation algorithm without being affected by the ambient light and the different systems (e.g. smartphones). It should be noted that the built-in data acquisition step to factor out the contributions of room light conditions and different smartphone models provide a unique advantage to generate this reliable blood Hgb calculation.

VHIC in mHematology is the key concept to achieve spectrometer-free, yet hyperspectral, quantification of blood Hgb content. VHIC allows for the mathematical reconstruction of the full spectral information from an RGB image taken by a conventional camera (i.e. three-color information from R, G, and B channels). The mathematical relationship between the full spectrum and the RGB intensity is described as

$x=Sr+e$

where x is a vector corresponding to the reflection intensity in each R, G, and B channel, S is a matrix of the RGB spectral responses of the three-color sensor, r is a vector of the spectral intensity reflected from the inner eyelid, and e is a vector of the system noise. In our case, the hyperspectral construction from the RGB signal is an inverse problem such that the number of actual measurements (i.e. three-color information) is less than the dimensionality of the full spectrum with λ = λ₁, λ₂, ..., λ_N. We took advantage of fixed-design linear regression with polynomial features to reliably construct the full spectral information r(λ₁, λ₂, ..., λ_N) from the RGB signals x(R, G, B) of the three-color RGB sensor embedded in the smartphone, as shown in FIG. 4, wherein a comparison is provided between the original hyperspectral dataset (acquired by an image-guided hyperspectral system) and the VHIC-constructed hyperspectral datasets (by SAMSUNG GALAXY J3) for both training and testing groups. The differences (hyperspectral vs. SAMSUNG GALAXY J3 VHIC) are plotted in solid color shading with 95% confidence intervals at each wavelength. The overall differences in SAMSUNG GALAXY J3 are small, supporting the high-fidelity of VHIC. The differences in the wavelength range between 450 and 575 nm are generally higher, because the distinct Hgb absorption is present in this range. First, the measured RGB intensity is described explicitly:

$x_{3\times 1}=S_{3\times N}{r}_{N\times 1}+e_{3\times 1}$

where x is a 3 × 1 vector corresponding to the reflection intensity in each R, G, and B channel, S is a 3 × N matrix of the RGB spectral responses of the 3-color sensor (i.e. built-in camera of SAMSUNG GALAXY J3), r is an N × 1 vector that has the spectral reflection intensity, and e is a 3 × 1 vector of the system noise with zero mean. In our case, r(λ = λ₁, λ₂, ..., λ_N) is discretized from 450 nm to 679 nm with a spectral interval of 1 nm. We take advantage of fixed-design linear regression to reconstruct hyperspectral data from RGB images. We paired the hyperspectral reflection dataset (acquired by the image-guided hyperspectral line-scanning system) and the RGB dataset (acquired by the RGB camera). It should be noted that the RGB dataset can also be generated by applying the RGB spectral responses to the hyperspectral dataset. X_3×m and R_N×m, are formed by adding x_3×1 and r_N×1 from m different measurements. The relationship in Equation (5-1) is described as:

$X_{3\times m}=S_{3\times N}{R}_{N\times m}$

which can be expressed as:

$R_{N\times m}=T_{N\times 3}{X}_{3\times m}$

where the transformation (or extrapolation) matrix T_N×3 = [S_3×N]^-1. If Equation (5-3) is solved for the unknown T_N×3, then T_N×3 can be used to transform the RGB dataset into the hyperspectral reflection dataset. Each three-color sensor model in different cameras has unique RGB spectral responses with spectral overlaps among the R, G, and B channels (also known as the sensitivity function of the camera of SAMSUNG GALAXY J3). To effectively incorporate the RGB spectral response of the camera, we expanded

$X_{3\times m}$

to

${\widehat{X}}_{p\times m}$

for maximizing the accuracy of the hyperspectral reconstruction such that:

$R_{N\times m}={\widehat{T}}_{N\times p}{\widehat{X}}_{p\times m}$

here

${\widehat{X}}_{p\times m}$

can be expressed explicitly such that:

$\begin{array}{l}{\widehat{X}}_{p\times m}=\\ {\left[\begin{array}{ccccccccccccccccc}{R}_1& G_1& B_1& \cdots & R_1^i& G_1^i& B_1^i& R_1{G}_1& G_1{B}_1& B_1{R}_1& \dots & {\left(R_1{G}_1\right)}^j& {\left(G_1{B}_1\right)}^j& {\left(B_1{R}_1\right)}^j& R_1{G}_1{B}_1& \cdots & {\left(R_1{G}_1{B}_1\right)}^j\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ R_m& G_m& B_m& \cdots & R_m^i& G_m^i& B_m^i& R_m{G}_m& G_m{B}_m& B_m{R}_m& ⋮& {\left(R_m{G}_m\right)}^j& {\left(G_m{B}_m\right)}^r& {\left(B_m{R}_m\right)}^j& R_m{G}_m{B}_m& \cdots & {\left(R_m{G}_m{B}_m\right)}^j\end{array}\right]}^{\text{T}}\end{array}$

where the exact powers of i and j of the single and cross terms are uniquely determined for a specific three-color sensor model, by checking the error between the reconstructed hyperspectral data and the original data.

The inverse of the expanded transformation matrix

$\widehat{T}$

Equation (5-4) can be considered to be the minimum-norm-residual solution to

$R=\widehat{T}\widehat{X}$

.Typically, this inverse problem is to solve a least-squares problem. We used QR decomposition, in particular the QR solver. After QR factorization is applied

$\widehat{X}$

,

$\widehat{T}$

to is estimated by minimizing the sum of the squares of the elements of

$R-\widehat{T}\widehat{X}$

and is selected such that the number of nonzero entries in

$\widehat{T}$

is minimized. Overall, the computation of the transformation (extrapolation) matrix establishes VHIC, eliminating a need of bulky dispersion hardware components (e.g. spectrometer, spectrograph, mechanical filter wheel, or liquid crystal tunable filter).

We now describe the partial least square regression (PLSR). We built a model for computing blood Hgb content from the hyperspectral reflection data of the inner eyelid. Analytical model-based Hgb prediction methods are often used, because Hgb has distinct spectral signatures (e.g. Soret and Q bands) in the visible range. However, such model-based approaches often require a priori information on all possible light absorbers in tissue for reliable Hgb quantification. Thus, we made use of PLSR, which has been extensively used to model relationships among measured variables (i.e. predictors) and response variables (i.e. outcomes) in a variety of biological and medical applications. Because PLSR transforms high-dimensional measured variables onto a reduced space of latent variables, it is highly beneficial to examine the significance of individual measured variables by eliminating insignificant variables. While PLSR is based on the extraction of principal components, it incorporates variations of both predictor and outcome variables simultaneously, enhancing the prediction performance. Similar to principal component analysis, it is critical to determine an optimal number of components in PLSR. The determination of an optimal number of principal components in ten-fold cross-validation of PLSR is performed. In particular, as the number of partial least squares (PLS) components increases, the percentage variance in the true Hgb values (outcome variable) increases, while the mean squared prediction error has minimal values for 18 components. These numbers of PLS components contribute to appropriate representation of variations in the spectroscopic and laboratory blood Hgb values simultaneously, thus making its prediction errors lower. As a result, 18 PLS components are selected and used in the Hgb prediction model. Although the use of PLSR often avoids overfitting when the number of predictors is larger than the sample size, it is also important to evaluate the ability for predicting Hgb levels from a completely new dataset after the model is established properly. As described above, we defined the two datasets for training and testing the blood Hgb model without reutilization of data from the same individuals.

Based on the aforementioned information, a hyperspectral/imaging data processing and statistical analysis is now provided. For data processing and algorithm development, we computed the hyperspectral and RGB data and developed the blood Hgb prediction model and the VHIC algorithm using MATLAB (MATLAB R2018b, The MathWorks, Inc.). For statistical analyses, we evaluated multiple linear regression, linear correlations, and intra-class correlations using STATA (STATA 14.2, STATACORP LLC). We conducted Bland-Altman analyses to compare the blood Hgb measurements as non-parametric methods. The bias is defined by the mean of the differences between the hyperspectral (or VHIC) and central laboratory blood Hgb measurements (d = y^VHI - y^central):

$\text{Bias =}\overline{d}=\frac{1}{n}{\Sigma }_{k=1}^n{d}_k.$

The 95% limits of agreement (LOA) is defined by a 95% prediction interval of the standard deviation:

$\text{LOA =}\overline{d}\pm 1.96\sqrt{\frac{1}{n-1}{\displaystyle \sum {}_{k=1}^n}=\left(d_k-\overline{d}\right)}$

Patient characteristics
Disorder                        Number of patients
Cancer                          45
HIV                             46
Tuberculosis                    8
Sickle cell disease             19
Acute kidney failure            1
Heart failure                   4
Malaria                         1
Anemia                          3
Immune thrombocytopenic purpura 1
No major disease                25
Dataset                         Average Hgb (g dL-1)
Training dataset (n = 138)      12.65 (SD = 3.11)
Testing dataset (n = 15)        11.06 (SD = 3.62)

The hyperspectral data reconstructed from the smartphone RGB images of the inner eyelids reliably estimate the actual blood Hgb levels. We evaluated SAMSUNG GALAXY J3 (see FIG. 5) for computing blood Hgb content using PLSR with 18 principal components, using the training dataset of 138 individuals. Referring to FIG. 5, the linear correlation analysis returns an R² value of 0.93 between the computed blood Hgb content and the laboratory blood Hgb levels for the training dataset. The Bland-Altman analysis shows that six out of 138 fall outside LOA of [- 1.57, 1.59 g dL^-1] with bias of 0.01 g dL^-1. To rigorously validate the Hgb prediction ability, the testing dataset of 15 individuals was applied to the identical PLSR model. Once again, the testing sample size was sufficient to cover a relatively large physiological range of Hgb levels from 4.3 to 15.3 g dL^-1. The Bland-Altman analysis shows 0 % (0 out of 17) outside LOA with bias of 0.04 g dL^-1.

Although mHematology is not limited to anemia assessments, when anemia is defined as Hgb < 12 g dL^-1 for females and Hgb < 13 g dL^-1 for males, the receiver operating characteristic (ROC) curves of SAMSUNG GALAXY J3 report the comparable performance with the image-guided hyperspectral line-scanning system (see FIG. 6). FIGS. 6a and 6b provide receiver operating characteristic (ROC) curves of mHematology performance for anemia assessment with a cutoff < 12 g dL^-1 for females and a cutoff < 13 g dL^-1 for males, respectively. Referring to FIG. 6a, the areas under the ROC curves are 0.99 and 0.97 for the training and testing datasets. Referring to FIG. 6b, the areas under the ROC curves are 0.98 and 1.00 for the training and testing datasets. These results validate the translational potential of spectroscopic and VHI blood Hgb measurements into highly reliable mobile anemia detection at least, while continuous and real-time monitoring of blood Hgb measurements has several other clinical applications. It should be noted that the overall performance of the VHI Hgb computation is drastically enhanced, compared with the simple RGB information (See FIGS. 7a and 7b). Referring to FIG. 7a, blood Hgb prediction using RGB information based on VHIC, according to the present disclosure is provided. The multiple linear regression analyses of 153 subjects show poor correlation coefficients in SAMSUNG GALAXY J3, seen in FIG. 7a. These results support the idea that mere RGB data without VHIC do not provide sufficient information to reliably assess blood Hgb levels. Statistical data using multiple linear regression of simple RGB information acquired using the camera of SAMSUNG GALAXY J3 is provided in Table 2.

Multiple linear regression of simple RGB information without VHI, acquired using the camera of SAMSUNG GALAXY J3
Source	SS	df	MS	Number of obs	= 153
				F(3, > F	= 40.38
Model	690.829718	3	230.276573
Residual	849.758666	149	5.7030783	R-squared Adj R-squared Root MSE	= 0.4484 = 0.4373 = 2.3881
Total	1540.58838	152	10.1354499
hgb	Coef.	Std. Err.	t	P>|t| [95% Conf. Interval]
r	22.31038	3.008851	7.41	0.000	16.36485	28.25591
g	-38.94604	4.708287	-8.27	0.000	-48.24967	-29.6424
b	18.88886	5.01539	3.77	0.000	8.978383	28.79933
_cons	3.007299	2.860896	1.05	0.295	-2.645869	8.660467

Using all of 153 individuals, the multiple linear regression analyses of the actual blood Hgb levels (i.e. outcome variable) against the eyelid RGB signals (i.e. predictor variables) return underperforming R² values of 0.45 in Galaxy J3 (see Tables 2). In other words, simple conjunctival redness scoring or pallor examination using mere RGB data may not provide sufficient information to reliably assess blood Hgb levels. Theoretically, mHematology can be used by an individual user as an ‘eyelid selfie.’ Overall, the reported results validate the potential of VHIC for translating RGB images into computational spectroscopy in a smartphone - mHematology that can be used for noninvasive, continuous and real-time blood Hgb measurements, which are comparable to clinical laboratory blood Hgb tests.

Those having ordinary skill in the art will recognize that numerous modifications can be made to the specific implementations described above. The implementations should not be limited to the particular limitations described. Other implementations may be possible.

Claims

1. A bloodless system for numerically generated hyperspectral imaging data for measuring biochemical compositions, comprising:

an optical imaging device adapted to acquire an RGB image from an area of interest, thereby generating a subject-specific RGB dataset representing the area; and

a processor adapted to: receive a hyperspectral dataset representing an a priori hyperspectral data of the area of interest of a population to which the subject belongs, receive RGB response for each one of RGB channels of the optical imaging device, pair the corresponding RGB data with the hyperspectral data, obtain a transformation matrix adapted to convert the subject-specific RGB image dataset into a subject-specific hyperspectral dataset for the optical imaging device, receive a subject-specific RGB dataset, generate a subject-specific hyperspectral dataset using the transformation matrix, and compute a blood hemoglobin level of the subject from the generated subject-specific hyperspectral dataset.

2. The system of claim 1, wherein the paired pixels from the RGB image is associated with a 3×1 RGB value matrix.

3. The system of claim 2, wherein the paired pixels from the hyperspectral dataset is associated with an N×1 spectrum, where N represents discretized spectra between a lower bound and an upper bound.

4. The system of claim 3, wherein the lower and upper bounds are 400 nm and 800 nm, respectively.

5. The system of claim 1, wherein the transformation matrix is a form of inverse of the RGB response matrix of the RGB sensor that converts an RGB to a spectrum.

6. The system of claim 5, wherein the inverse of the transformation matrix is determined numerically by using the paired RGB and hyperspectral data of the population.

7. The system of claim 1, wherein the biochemical compositions include blood hemoglobin.

8. The system of claim 1, wherein the area of interest includes the inner surface of a subject's inner eyelid.

9. The system of claim 1, wherein the biochemical compositions are determined using spectral analysis.

10. The system of claim 9, wherein the spectral analysis includes a partial least square regression statistical modeling technique to first build a model from a training set of a first hyperspectral dataset vs. the biochemical composition and then apply the model to a second dataset from the generated hyperspectral image dataset.

11. A method for a bloodless numerically generated hyperspectral imaging data for measuring biochemical compositions, comprising:

obtaining an RGB image from an area of interest, thereby generating a subject-specific RGB dataset representing the area of interest;

receiving a hyperspectral dataset representing an a priori hyperspectral data of the area of interest for a population to which the subject belongs;

receiving an RGB response for each one of RGB channels of the optical imaging device, pairing the corresponding RGB data with the hyperspectral data;

obtaining a transformation matrix adapted to convert the subject-specific RGB image dataset into a subject-specific hyperspectral dataset for the optical imaging device;

generating a subject-specific hyperspectral dataset using the transformation matrix; and

computing a blood hemoglobin level of the subject from the generated subject-specific hyperspectral dataset.

12. The method of claim 1, wherein the paired pixels from the RGB image is associated with a 3×1 RGB value matrix.

13. The method of claim 12, wherein the paired pixels from the hyperspectral dataset is associated with an N×1 spectrum, where N represents discretized spectra between a lower bound and an upper bound.

14. The method of claim 13, wherein the lower and upper bounds are 400 nm and 800 nm, respectively.

15. The method of claim 11, wherein the transformation matrix is a form of inverse of the RGB response matrix of the RGB sensor that converts an RGB to a spectrum.

16. The method of claim 15, wherein the inverse of the transformation matrix is determined numerically by using the paired RGB and hyperspectral data of the population.

17. The method of claim 11, wherein the biochemical compositions include blood hemoglobin.

18. The method of claim 11, wherein the area of interest includes the inner surface of a subject's inner eyelid.

19. The method of claim 11, wherein the biochemical compositions are determined using spectral analysis.

20. The method of claim 19, wherein the spectral analysis includes a partial least square regression statistical modeling technique to first build a model from a training set of a first hyperspectral dataset vs. the biochemical composition and then apply the model to a second dataset from the generated hyperspectral image dataset.

21. A method for a bloodless numerically generated hyperspectral imaging data for measuring biochemical compositions, comprising:

obtaining an RGB image from an area of interest, thereby generating a subject-specific RGB dataset representing the area of interest;

receiving a hyperspectral dataset representing an a priori hyperspectral data of the area of interest for a population to which the subject belongs;

receiving an RGB response for each one of RGB channels of the optical imaging device, pairing the corresponding RGB data with the hyperspectral data;

converting the subject-specific RGB image dataset into a subject-specific hyperspectral dataset for the optical imaging device; and

computing a blood hemoglobin level of the subject from the generated subject-specific hyperspectral dataset.