DISTRIBUTED MICROWAVE IMAGE PROCESSING SYSTEM AND METHOD
A distributed imaging system uses non-ionizing radiation in the form of microwaves to image the body. This non-ionizing radiation is safer for a patient than traditional x-rays. The majority of image processing takes place in a centralized computing environment which receives microwave image data from many remote data acquisition sites (such as imaging centers, radiology groups, and/or doctor's offices). The centralized computing environment is specially configured to receive microwave image data from many sites and produce microwave images based on the received microwave image data. The microwave images may then be sent back to the acquisition site and/or to other sites for viewing and evaluation. This relieves the image data acquisition site of the burden and expense of having specialized high speed computer equipment that is necessary to produce microwave images. Because the computing environment collects and stores information (data and results) from many data acquisition sites in one central location, the prior results from all of the acquisition sites can be accessed and used to enhance processing and diagnosis.
Latest ELLUMEN, INC. Patents:
- Directable antenna system and method for improved communications quality
- Accurate signal compensations for UWB radar imaging in dispersive medium
- Phase confocal method for near-field microwave imaging
- DIRECTABLE ANTENNA SYSTEM AND METHOD FOR IMPROVED COMMUNICATIONS QUALITY
- ACCURATE SIGNAL COMPENSATIONS FOR UWB RADAR IMAGING IN DISPERSIVE MEDIUM
This application is a divisional of U.S. application Ser. No. 13/798,428, filed Mar. 13, 2013, the entire contents of which is incorporated herein by reference.
BACKGROUND OF THE INVENTIONThe present invention relates generally to the field of image processing, and more specifically to a microwave image processing system.
Microwave imaging is a field of research that attempts to solve the inverse scattering problem. When radio frequency (RF) energy moves through air and impinges an object, scattering from the object occurs as the RF energy hits the surface and moves through the object. The idea of the inverse scattering problem is to measure this scattering field and, combined with other information about the object, determine an ‘image’ of the object that created the measured scattering field.
Microwaves are non-ionizing RF energy. Radiation that has enough energy to move atoms in a molecule around or cause them to vibrate, but not enough to remove electrons, is referred to as “non-ionizing radiation.” Examples of this kind of radiation are sound waves, visible light, and microwaves. Ionizing radiation, on the other hand, is high-frequency radiation that has enough energy to remove an electron from (ionize) an atom or molecule. Ionizing frequencies have been shown to have cancer causing effects. Ionizing radiation has enough energy to damage the DNA in cells, which in turn may lead to cancer. Gamma rays, x-rays, some high-energy UV rays, and some sub-atomic particles such as alpha particles and protons are forms of ionizing radiation. The use of ionizing frequencies also has increased costs associated with their production and requires specialized environmental protections.
Efforts in microwave image processing involve researching and developing hardware and software solutions to the inverse scattering problem. The goal of this research is to develop methodologies, algorithms, hardware, systems, and techniques that allow users to completely image the human body in sufficient detail as to render a timely and accurate medical diagnosis by trained medical professionals.
SUMMARY OF THE INVENTIONThrough the course of research it was discovered that the following problems must be solved:
Modeling of RF scattering problems on the scale of the human body is a very computationally intensive task. This modeling is called solving the forward scattering problem;
Solving the inverse scattering problem requires us to not only solve the forward scattering problem but then use additional computationally intensive algorithms to complete the solution for solving the inverse problem (i.e., create an image);
As a result, the overall solution to MWI (microwave imaging) is a computationally heavy and expensive endeavor that requires specialized computing techniques, hardware, and infrastructure that are not readily available.
What is needed in the field is a hardware and software platform designed specifically for processing microwave images. The ideal system would take advantage of parallel processing and optimized data storage and retrieval techniques in order to speed reconstruction of microwave images. The ideal system would also have a reconstruction algorithm specifically designed for reconstruction of microwave images. The ideal reconstruction software package would further include a learning algorithm that is able to learn from prior reconstructions and use the prior data to speed current reconstruction. The ideal system would further allow for viewing, diagnosis and reporting by medical professionals. Further, the ideal image processing system would provide for communications with research institutions and insurance companies.
A distributed imaging system is described below for imaging parts of a human body, or the entire human body, (or an animal body or part thereof) wherein non-ionizing radiation is used to image the body so that the imaging process is safer for patients than traditional x-rays. The majority of image processing takes place at a site specifically adapted for MWI, for example, a cloud based computing environment, thereby relieving the image data acquisition site and the image viewing location of the processing burden of image reconstruction. The distributed imaging system comprises an image data acquisition site, a computing environment, and a viewing location. The computing environment is remote from the acquisition site and viewing locations. The acquisition site has an image data capture device capable of capturing data for images of all or part of the human body. The capture device uses non-ionizing radiation, for example microwaves, to collect image data of the body thereby providing safer imaging for a patient requiring repeated imaging. The acquisition site also has a computing device with networking capabilities for transmitting the captured data as microwave image (MWI) data to the, for example, cloud based computing environment.
The computing environment receives the MWI data, processes the MWI data to reconstruct images, and makes the images available for viewing. The computing environment comprises at least two processors, at least two high-speed memory buses, and a plurality of memories that are connected to the processors by the high-speed memory buses. The computing environment includes: a raw image database that receives the MWI data from the acquisition site; an image reconstruction algorithm and MWI learning algorithm for processing the MWI data and reconstructing images; a reconstruction database for storing reconstructed images that are available for viewing; and, a MWI viewing application that allows for viewing of the reconstructed images.
The image viewing location includes a computing device that is able to access the computing environment over a network, and a display that allows a user to view selected images via a MWI viewing application. The image viewing location can be used by a doctor to view images, prepare a diagnosis, and submit a report on the patient. The viewing location can also be used by the patient to view images and reports. The viewing location can further be used by scientists doing research, and by insurance companies filing inquiries.
At the acquisition site, patient identification information is attached to the MWI data prior to transmission to the computing environment. At the computing environment, the MWI learning algorithm is able to provide the image reconstruction algorithm with prior data that is from prior reconstructions in order to speed reconstruction of the image. The computing environment further comprises at least one processing queue that receives data from the raw image database and provides the data to the processors. The at least two processors employ parallel computing techniques so that the processors operate in parallel on the MWI data. Further, the plurality of memories are specially configured for storage of MWI data so as to optimize the parallel operations of the processors.
The computing environment further comprises: a MWI patient reporting application that allows the user to prepare and submit reports on patients; a patient history database that stores and provides access to patient histories; and, a reporting database that stores and provides access to patient reports that have been submitted.
The computing environment further comprises: a patient diagnostic learning algorithm that provides the system with automated diagnostics of reconstructed images; a MWI data collaboration application that accepts and responds to insurance inquiries; and, a MWI DICOM (digital imaging and communications in medicine) communications application that allows researchers and scientists to access and communicate with the system.
In one preferred embodiment, the image data capture device is a full body scanner, and the non-ionizing radiation used by the image data capture device is in the range of 200-4000 MHz. Further, each of the at least two processors are preferably multi-core processors.
Embodiments will now be described in more detail with reference to the accompanying drawings, given only by way of example, in which:
Use of a centralized database of prior raw data from many acquisition sites and calculated images allows for the selection of a better “seed” (a better starting point for the iterative reconstruction calculation) which substantially reduces computation time.
MWI processing can be cost affordable because specialized and non-standard computing resources at scanner locations (acquisition sites) are not required;
Image viewing is more cost affordable because users do not have to implement software or specialized computers at their location;
Insurance reimbursements are cheaper and allows insurance companies to access data for medically reimbursed claims; and,
Scientists and researchers have a central location to access all aspects of the MWI process for medical research purposes.
The acquisition site 101 can be in a hospital, a radiology group, a doctor's office, a medical imaging facility, or other site with a microwave image data capture device. A computer with network capabilities is also located at the acquisition site 101. After raw image data has been captured, the raw microwave image data is transmitted to the computing environment 102 for reconstruction of the image. Upon completion of reconstruction, the images are stored in a reconstruction database within the computing environment 102.
The microwave image processing for microwave imaging can make use of a conventional cloud environment augmented with components specific to MWI, including:
Platform Specific Parts:
-
- RAW Scanner Database
- RAW Scanner Processing Queue
- Patient History Database
- MWI Reconstruction Algorithms
- MWI Learning Algorithms
- Patient Diagnostic Learning Algorithms
- Reporting Database
Application Specific Parts:
-
- MWI Data Collaboration
- MWI DICOM Communication
- MWI Viewing Application
- MWI Patient Reporting Application
Infrastructure Specific Parts:
-
- MWI Computing Platforms for MWI reconstruction
- Specialized Storage for MWI
- Internal Specialized Networking for MWI
A primary goal of the present system 100 is to allow users to completely image the human body in sufficient detail as to render a timely and accurate medical diagnosis by trained medical professionals. A doctor or other medical professional can access the computing environment 102 from viewing location 103, in order to view reconstructed images, diagnose the patient, and submit a report on the patient. The patient can access the computing environment 102 from viewing location 104, in order to view reconstructed images, review the patient's history, and provide updates to the patient's personal information. Scientists performing research can access the computing environment 102 from viewing location 105. The present system 100 preferably includes a Digital Imaging and Communications in Medicine (DICOM) communications application to communicate with researchers in an industry standard format. Insurance companies can access and communicate with the system via viewing location 106 that includes a Data Collaboration application that provides for communications in insurance industry standards.
In most embodiments, the computing environment 102 is located remote from acquisition sites 101, viewing locations 103-105, and insurance company 106. In many embodiments, the environment 102 is located many miles away (for example more than 10 or more than 100 miles away) from sites 101, locations 103-105 and/or company 106. However, in certain applications of the invention, “remote” as used herein can mean in a different room or in a different building.
The environment 102 contains a centralized database 412 of prior raw microwave data along with the resulting calculated images and other information previously calculated for the acquisition sites 101. This centralized database is a significant feature because it includes raw data (and calculated images and other information) from a plurality of acquisition sites 101 (for example, 10 or more sites or 100 or more sites) and thus the processing for an individual site 101 is able to take advantage of prior calculations done for all of the sites in the distributed system 100. Since calculation of a microwave image is an iterative process which requires a seed (an educated guess as to the electrical properties of the subject patient) as a starting point, the use of prior results from calculations done for all of sites 101 results in a better seed, and thus a substantial reduction in computing.
The centralized database may also collect the images and resulting diagnosis from all of the sites in the distributed system, which results in a more informed automated diagnostic algorithm.
Background on microwave imaging is set forth in the following texts: Matteo Pastorino, “Microwave Imaging,” WILEY, 2010; Jaleel Akhtar, “Microwave Imaging: Reconstruction of One Dimensional Permittivity Profiles,” VDM Verlag, 2008; and Bindu Gopinathan Nair, “Active Microwave Imaging for Mammography: Microwave Medical Imaging,” Lap Lambert Academic Publishing, 2012. The entire contents of all three of these texts is incorporated herein by reference for the hardware, software, components, techniques, algorithms, data, and mathematical techniques described therein related to microwave and other types of non-ionizing radiation imaging.
Software module 410 provides the programs, algorithms, applications and other software required by the computing environment 102. Further details of the software module 410 are provided in conjunction with
Network Interface module 411 provides the necessary components and software modules for interfacing with the Internet, other networks, the acquisition sites 101, the multiple viewing locations 103-105, and with the insurance company 106.
Other configurations of the computing environment are possible. A main feature is that the environment be built for computational speed, preferably using parallel computing components including high speed RAM and disk drives. The aggregate computation performance of the computational environment is preferably greater than 10 Teraflops for example, 15 Teraflops, 25 Teraflops or more. In an alternate embodiment, a head node acts as a controller for compute jobs in the high-speed computational environment. The job of the head node also includes task scheduling, resource allocation, and other relevant tasks to support computing in a parallel environment. A typical CPU will have multiple processors having at least 40 total cores, 256 GB RAM, and 2 Terabytes of Data Storage. The computing environment has multiple CPUs of this capability. Data Communication between the major systems utilizes high-speed fiber optic communications running typically greater than 40 GB/s. Data Communications within a major system parallel data path run typically greater than 4.80 GT/s (Giga-transfers per second). Further, a powerful supplemental computational platform is constructed from external graphical processing units (GPU). The computing environment includes 4 to 16 or more GPUs that are merged into the overall computing environment, and are accessible as a general processing node. Finally, a high-speed data bus interconnecting GPUs is provided for shared memory/processor access, and a high-speed data port for communicating results to the outside world is provided.
The computing devices at the viewing locations 103-105 and at the insurance company 106 can be laptop or desktop computers, and the computers can be connected to the network via a wired or wireless connection.
The Input to reconstruction and learning algorithms includes:
-
- 1. RF scattering information measured from an antenna apparatus where the scattering object is either in a coupled or non-coupled media.
- 2. Three dimensional surface map of the scattering object obtained using optical or non-optical methods.
- 3. Positional information of RF emitters related to the collected scattering information.
- 4. Geometric parameters of the collection device (this is done in order to accommodate collection devices that are of different sizes and configurations).
- 5. RF and electrical characteristics of the RF emitters.
- 6. RF and electrical characteristics of the RF receivers.
- 7. Collection device calibration data.
- 8. Collection site specific data (location, phone numbers, POCs, etc.)
- 9. Scattering Object specific information:
- a. Object Classification
- i. Human
- ii. Animal
- iii. Phantom
- iv. Other
- b. For Patients (Human or Animal):
- i. Demographic information, sex, weight and appropriate patient identifiers used for identity management.
- ii. Any presenting symptoms, reasons for examination and any other pertinent previous health information.
- iii. Insurance/payment information.
- c. Phantom ID for Calibration Phantoms.
- d. Other Data for Other Objects.
- 10. Requested study output—study type, orthogonal views, anatomy of interest, area of interest.
Input data is encoded according to the DICOM standard. Provisions are also made to handle other types of data encoding for data exchange such as HL7, XML, etc.
In step 1102, a database of prior RF scans is searched. This search involves looking for prior RF scans that are not just similar in electrical characteristics but also are similar in other characteristics including, patient sex, age, weight and location of the scanned data. The database of prior RF scans is essentially a database of the input data from step 1101. The input data also contains the prior RF scanned data including raw measured data. However, associating all the input data with the RF scans and saving in a database, allows searching and relating data upon other characteristics as mentioned above.
In step 1103, it is determined if there is a possible match that can be used. There is a computational component in this block that does further feature extraction and comparisons in order to ensure like data is being appropriately compared.
In step 1104, if a close match is found, the prior reconstructed data is retrieved from the database of dielectric images. Once this data is retrieved it is made available as a ‘seed’ into the reconstruction process, described further below.
As discussed above, this is a substantial feature because the iterative reconstruction processing takes substantially less time if a better seed (or starting point) is used. The database of dielectric images is a database of successfully reconstructed dielectric images. Associated with this data is the input data from step 1101.
In step 1105, the RAW dielectric images are reconstructed using MWI Reconstruction and Learning Algorithms. Detailed steps of image reconstruction are discussed in greater detail below in conjunction with
In step 1109, Hounsfield encoding is performed. Hounsfield encoding is a step taken to relate the dielectric values of the image(s) reconstructed to an appropriate Hounsfield value. Hounsfield values are typically used in CT studies and are a unit of measure that represents the different density levels of tissues and/or other substances. In step 1110, using the Hounsfield encoded data; a DICOM formatted imaging study is created. The rational for Hounsfield encoding and putting into a DICOM format is to allow for the exchange of information created using MW techniques with current diagnostic tools and techniques. Additionally, the medical community is familiar with diagnosing Hounsfield imaging studies and hence physicians will require little if any retraining to use the output of the present MWI process. In step 1111, the DICOM Hounsfield imaging study is stored into a database of Hounsfield encoded imaging studies, which is a database of successfully reconstructed dielectric imaging studies encoded to Hounsfield units. Associated with the stored data is the input data from step 1101, less the RAW RF Scan data.
The present distributed microwave image processing has numerous advantages over traditional systems that effectively put it in a new class of image processing. The computational difficulties associated with microwave imaging had previously made it impossible to use this technology effectively for clinical purposes. However CBMWIP enables the use of MWIP for clinical purposes. The particular composition and configuration of CBMWIP accounts for the necessary business processes that are required to make use of MWI data and provide that capability in an economically feasible package. The advantages provided by the present system to the imaging community include:
CBMWIP uses cloud technology to distribute the computationally intensive Image Reconstruction Algorithm and Learning Algorithm;
The Image Reconstruction Algorithm provides reconstruction of raw data into higher resolution images than previously possible;
The Learning Algorithm speeds the image reconstruction;
Allows for the decoupling of the microwave scanning device from the MWI reconstruction process;
Allows for the decoupling of image viewing of reconstructed MWIs from user's computer resources;
Allows for installation of scanning devices in less environmentally stringent conditions since specialized computer resources are removed from the scanner location;
Provides a centralized database of patient image history from scanning devices that are used to aid the diagnostic process;
Allows for clinicians to access a patient's reconstructed microwave images and history for diagnostic purposes;
Allows for clinicians to store diagnostic reports;
Allows for patients to access their reconstructed microwave images and history for medical purposes;
Allows for other imaging systems in clinical locations to access the reconstructed MWI for diagnostic or reference purposes using the DICOM standard; and,
Allows for learning algorithms to use stored diagnostic reports to aid automated diagnostic capabilities.
The invention is not limited to the embodiments described above. Instead, many other variations of the invention are within the scope and spirit of the invention. The invention is therefore defined with reference to the following claims.
Claims
1. A system for producing images of at least part of a body, the system comprising:
- a centralized computing environment configured to receive microwave data from a plurality of remote data acquisition sites, wherein the centralized computing environment is specially configured to receive the microwave data and reconstruct at least one image based on the microwave data.
2. A system as set forth in claim 1, wherein the plurality of remote acquisition sites comprise:
- at least one microwave transmitter and at least one microwave receiver, configured to transmit a microwave to a body and to receive a microwave from the body and to generate the microwave data; and
- a communications device configured to transmit the microwave data to the centralized computing environment.
3. A system as set forth in claim 1, wherein the centralized computing environment comprises:
- a communications device configured to transmit a reconstructed microwave image to a remote viewing location.
4. A system as set forth in claim 1, wherein the centralized computing environment comprises:
- at least two processors, at least two high-speed memory buses, and a plurality of memories that are connected to the processors by the high-speed memory buses.
5. A system as set forth in claim 1, wherein the centralized computing environment comprises:
- a centralized database configured to store past microwave data from the plurality of remote data acquisition sites; and
- a seed generation program that compares the past microwave data with present microwave data in order to generate a seed for reconstruction.
6. A system as set forth in claim 1, wherein the centralized computing environment comprises:
- a centralized database configured to store past diagnoses associated with past image reconstructions; and
- a learning program that uses the past diagnoses to suggest a diagnosis for a present microwave image.
7. A system as set forth in claim 1, wherein the centralized computing environment comprises at least one processing queue that receives microwave data from the plurality of remote data acquisition sites and provides the microwave data to at least two processors such that processors operate in parallel on the microwave data.
8. A system as set forth in claim 1, wherein the centralized computing environment comprises:
- a patient history database that stores and provides access to patient histories.
9. A system as set forth in claim 1, wherein the centralized computing environment comprises:
- a data collaboration application that accepts and responds to insurance inquiries.
10. A system as set forth in claim 1, wherein the centralized computing environment comprises:
- a DICOM communications application that allows researchers and scientists to access the system.
11. A system as set forth in claim 1, wherein the centralized computing environment comprises:
- multi-core processors.
12. A system as set forth in claim 1, wherein the centralized computing environment comprises:
- a program to relate dielectric values of the reconstructed image to Hounsfield values.
13. A method for producing images of at least part of a body, the method comprising:
- receiving microwave data from a plurality of remote data acquisition sites in a centralized computing environment specially configured to receive the microwave data and reconstruct at least one image based on the microwave data to generate a reconstructed image.
14. A method as set forth in claim 13, wherein the method comprises:
- storing past microwave data from the plurality of remote data acquisition sites; and
- comparing the past microwave data with present microwave data and generating seed information for reconstruction.
15. A method as set forth in claim 13, wherein the method comprises:
- storing past diagnoses associated with past image reconstructions in a centralized database; and
- using past diagnoses to suggest a diagnosis for a present microwave image.
16. A method as set forth in claim 13, wherein the method comprises:
- generating microwave images in a DICOM compatible format.
17. A method as set forth in claim 13, wherein the method comprises:
- relating dielectric values of a reconstructed image to Hounsfield values.
Type: Application
Filed: Jun 9, 2016
Publication Date: Oct 6, 2016
Applicant: ELLUMEN, INC. (Arlington, VA)
Inventors: William J. McCollough (Earslyville, VA), Todd R. McCollough (Barrington, IL)
Application Number: 15/177,511