COMPUTER-AIDED DETECTION OF FOLDS IN MEDICAL IMAGERY OF THE COLON
The application discloses computer-based apparatus and methods for analysis of images of the colon to assist in the detection of colonic polyps. The apparatus and methods include the detection, classification and display of candidate colonic folds.
The application discloses computer-based apparatus and methods for analysis of images of the colon to assist in the inspection of the colon.
BACKGROUNDColon cancer is the second leading cause of cancer death among men and women in the United States. The identification of suspicious polyps in the colonic lumen may be a critical first step in detecting the early signs of colon cancer. Many colon cancers can be prevented if precursor colonic polyps are detected and removed.
Computed tomographic (CT) and magnetic resonance (MR) colonography, two new “virtual” techniques for imaging the colonic lumen, have emerged as alternatives to the invasive optical colonoscopy procedure, which has traditionally been considered the gold standard for viewing the colon. CT imaging systems, for example, may acquire a series of cross-sectional images (i.e., slices) of the abdomen using scanners and x-rays. Computer software may be used to construct additional imagery from the slices, such as a three-dimensional (3-D) volume of the abdominal region. A physician may inspect the imagery for indicators of colonic polyps.
The human colon has many folds that complicate the physician's inspection procedure. While most folds are considered healthy tissue, polyp-like anomalies may form either on or near folds and should be carefully examined by a physician. As a result, a physician may frequently change the viewing angle while inspecting the colon, which may undesirably increase the physician's overall interpretation time. Even still, physicians may fail to detect polyps due to folds, which may be attributed in part to the long interpretation times required to inspect a colon, and to human error associated with such inspection, such as error resulting from fatigue.
Researchers have begun exploring automatic, computer-implemented approaches for assisting the inspecting physician who may miss polyps due to folds. Several notable approaches will now be discussed in brief detail.
In “Colon Straightening Based on an Elastic Mechanics Model,” Biomedical Imaging: Nano to Macro, 2004. IEEE International Symposium on, Publication Date: 15-18 Apr. 2004, page(s): 292-295, Vol. 1, Zhang et al. “flatten” the folds of a colon surface, which may provide a form of fold subtraction. While interesting in theory, physicians may not accept the distorted colon for purposes of inspection and diagnosis, as artifacts may be introduced by the algorithm. Furthermore, any processing to correlate the results of the flattened and original colon may be extremely sensitive to the algorithm used. Thus, a solution that does not distort the colon imagery may be more desired by the physician.
In U.S. Pat. No. 7,286,693, “Medical viewing system and image processing method for visualization of folded anatomical portions of object surface,” Makram-Ebeid et al. detect folded objects in the colon that may have a “hidden portion,” such as an area that may be hidden because it appears between the surface and a fold of the colon. Hidden portions of a colon have a high likelihood of being missed by an inspecting physician. The detected folded objects are then displayed in various ways to capture the attention of the inspecting physician. Measurements regarding both the folded objects and their hidden portions are also displayed as output. While Makram-Ebeid's approach identifies folded portions of a colon that require careful inspection, the approach is limited to the detection of only those folds that have a hidden portion. There may be many folds in a colon that do not have a hidden portion but that may still be of interest to the physician. For example, folds adjacent or near to a polyp-like anomaly may be of particular interest. Thus, a means for identifying folds of a colon, regardless of whether folds have a “hidden portion” or not, is still desired. Furthermore, while Makram-Ebeid's approach calls attention to specific folded portions that may be of interest, the physician may still be required to change the viewing angle of the colon to properly inspect the hidden portion of the fold. A solution that reduces or eliminates the need for the physician to change the viewing angle around colonic folds would be desirable.
Two automated methods for detecting colonic folds (including those folds without a “hidden portion”) can be seen in the prior art. In “Tissue Classification Based on 3D Local Intensity Structures for Volume Rendering,” IEEE Transactions on Visualization and Computer Graphics, April-June 2000, Vol. 6:2, pp. 160-180, Sato et al. teach a sheet structure enhancement filter method for detecting folds. In “Haustral fold analysis may aid detection of flat colorectal polyps,” IEICE Tech. Rep., vol. 108, no. 131, MI2008-31, pp. 59-64, July 2008, Oda et al. improve on Sato's method by using a ridge structure enhancement (RSE) filter method for detecting folds. Curvature-based fold detection methods such as these may have inherent limitations due to tortuous colons, adequacy of colonic distention, and the complexity of fold composition (e.g., shapes and sizes). In clinical practice, insufflation may be performed with highly varying accuracy and thus, fold distention may also be highly variable. Furthermore, in clinical practice, a wide range of colon and fold compositions may be encountered. Thus, there is a need for an alternative, automated method of identifying folds that is not dependent on adequate colonic distention and is applicable to a wider range of colon and fold compositions.
It is therefore an object of this disclosure to automatically compute and output colonic fold information in various ways that may improve a physician's ability to inspect colon imagery.
It is another object of this disclosure to depict colonic folds in various ways that may reduce the time it takes a physician to inspect areas around colonic folds.
It is yet another object of this disclosure to detect colonic folds using a method that is not dependent on consistently adequate colonic distention and is applicable to a wider range of colon and fold compositions.
SUMMARYDisclosed are computer-implemented methods of presenting colonic folds in a colon under study to a user.
The methods may comprise receiving, through at least one input device, digital imagery representing at least a portion of a colon; using at least some of said digital imagery, detecting, in at least one processor, at least one candidate colonic fold in said at least a portion of a colon; classifying, in at least one processor, at least one of said candidate colonic folds as a colonic fold; and outputting, through at least one output device, information identifying said at least one candidate colonic fold which was classified as a colonic fold.
Detecting at least one candidate colonic fold may comprise performing a colonic wall segmentation step; and based upon the colonic wall segmentation, performing a candidate fold segmentation step, wherein a colonic wall segmentation may include soft tissue objects protruding from said wall into the lumen of said colon. Performing the colonic wall segmentation step may comprise performing at least one of an active contour method, a level set method, and a CT value and CT gradient method. Performing the colonic wall segmentation step may comprise performing a colon lumen segmentation step; and based upon the colonic lumen segmentation, performing a colon wall identification step. Performing the colonic lumen segmentation step may comprise segmenting a representation of air of said colon; and segmenting a representation of fluid of said colon. Performing the colonic wall identification step may comprise performing at least one of a local convex hull operation and a morphological closing operation. Performing the candidate fold segmentation step may comprises performing an erosion of the colonic wall; and based on the colonic wall erosion, performing a thresholding operation on the eroded colon wall. Performing an erosion of the colonic wall may comprise performing at least one of a morphological erosion, an active contour, or a distance transform operation. Performing an erosion of the colonic wall may comprise performing a first operation on said colon wall to identify a body of said at least one candidate colonic fold; and performing a second operation on said colon wall to identify a base of said at least one candidate colonic fold.
Classifying at least one of said candidate colonic folds as a colonic fold may comprise performing at least one of a distance feature extraction step and a non-distance feature extraction step on the candidate colonic fold; and based upon the at least one of the distance feature extraction step and the non-distance feature extraction step performed, performing a classification step. Performing a distance feature extraction step may comprise computing at least one distance measurement from a common voxel point to voxel points along a boundary where said candidate colonic fold meets said colon wall. Performing a non-distance feature extraction step may comprise computing at least one of a volume feature, a feature describing the amount the candidate colonic fold touches the colonic wall, a shape index feature, a curvature feature, and a texture feature. Performing a classification step may comprises computing a discriminant score from at least one of a distance feature measurement extracted and a non-distance feature measurement extracted; and classifying said at least one candidate colonic fold based on said discriminant score computed. The classification may be a binary decision as to whether the candidate colonic fold is a colonic fold. The classification may be a probability as to whether the candidate colonic fold is a colonic fold.
Outputting may comprise displaying digital imagery representing at least a portion of the colon on at least one output device; and specially depicting said at least one candidate colonic fold which was classified as a colonic fold in said at least a portion of the colon displayed. Outputting may further comprise, in said special depiction of said at least one candidate colonic fold which was classified as a colonic fold, displaying the said at least one candidate colonic fold which was classified as a colonic fold at least partially transparently. At least a portion of the digital imagery representing at least a portion of a colon may derive from a non-invasive imaging method. The non-invasive imaging method may be selected form the set composed of CT scanning and MRI imaging.
Also disclosed is a computer-readable medium having computer-readable instructions stored thereon which, as a result of being executed in a computer system having at least one processor, at least one output device and at least one input device, instruct the computer system to perform the above methods.
Also disclosed is a computer system for presenting colonic folds in a colon under study to a user, comprising at least one processor, at least one input device and at least one output device, so configured that the computer system is operable to perform the above methods.
The methods may comprise receiving, through at least one input device, digital imagery representing at least a portion of a colon; using at least some of said digital imagery, detecting, in at least one processor, at least a portion of a colonic wall in said at least a portion of a colon; segmenting, in at least one processor, at least one candidate colonic fold from said at least a portion of a colonic wall; and outputting, through at least one output device, information identifying said at least one candidate colonic fold which was segmented from said at least a portion of a colonic wall.
Detecting at least a portion of a colonic wall in said at least a portion of a colon may comprise performing at least one of an active contour method, a level set method, and a CT value and CT gradient method. Detecting at least a portion of a colonic wall in said at least a portion of a colon may comprise performing a colon lumen segmentation step; and based upon the colonic lumen segmentation, performing a colon wall identification step. Performing the colonic lumen segmentation step may comprise segmenting a representation of air of said colon; and segmenting a representation of fluid of said colon. Performing the colonic wall identification step may comprise performing at least one of a local convex hull operation and a morphological closing operation.
Segmenting at least one candidate colonic fold from said at least a portion of a colonic wall may comprise performing an erosion of the colonic wall; and based on the colonic wall erosion, performing a thresholding operation on the eroded colon wall. Performing an erosion of the colonic wall may comprise performing at least one of a morphological erosion, an active contour, or a distance transform operation. Performing an erosion of the colonic wall may comprises performing a first operation on said colon wall to identify a body of said at least one candidate colonic fold; and performing a second operation on said colon wall to identify a base of said at least one candidate colonic fold.
The method may further comprise classifying, in at least one processor, at least one of said candidate colonic folds segmented from said at least a portion of a colonic wall as a colonic fold. Classifying at least one of said candidate colonic folds as a colonic fold may comprise performing at least one of a distance feature extraction step and a non-distance feature extraction step on the candidate colonic fold; and based upon the at least one of the distance feature extraction step and the non-distance feature extraction step performed, performing a classification step.
Outputting may comprise displaying digital imagery representing at least a portion of the colon on at least one output device; and specially depicting said at least one candidate colonic fold which was classified as a colonic fold in said at least a portion of the colon displayed.
Also disclosed is a computer-generated user interface for presenting a graphical representation of a colon, the user interface comprising a depiction of the colon; wherein regions of the colon segmented as colonic folds are displayed at least partially transparent.
In the following detailed description of embodiments, reference is made to the accompanying drawings that form a part hereof, and in which are shown, by way of illustration and not by way of limitation, specific embodiments in which the methods and systems disclosed herein may be practiced. It is to be understood that other embodiments may be utilized and that logical, mechanical, and electrical changes may be made without departing from the scope of the methods and systems disclosed herein.
This disclosure is directed to a system for and method of automatically detecting and outputting the folds of an anatomical colon.
System 100 includes an image acquisition unit 110 for performing a medical imaging procedure of a patient's colon and an image viewing station 120 for processing and displaying colon imagery to a physician or other user of the system. Image acquisition unit 110 may connect to and communicate with image viewing station 120 via any type of communication interface, including but not limited to, physical interfaces, network interfaces, software interfaces, and the like. The communication may be by means of a physical connection, or may be wireless, optical or of any other means. It will be understood by a person of skill in the art that image acquisition unit 110 and image viewing station 120 may be deployed as parts of a single system or, alternatively, as parts of multiple, independent systems, and that any such deployment may be utilized in conjunction with embodiments of the methods disclosed herein. If image acquisition unit 110 is connected to image viewing station 120 by means of a network or other direct computer connection, the network interface or other connection means may be the input device for image viewing station 120 to receive imagery for processing by the methods and systems disclosed herein. Alternatively, image viewing station 120 may receive images for processing indirectly from image acquisition unit 110, as by means of transportable storage devices (not shown in
Image acquisition unit 110 is representative of a system that can acquire imagery of a patient's abdominal region using non-invasive imaging procedures (e.g. a virtual colonography imaging procedure). Such a system may use computed tomography (CT), magnetic resonance imaging (MRI), or another suitable method for creating images of a patient's abdominal and colonic regions as will be known to a person of skill in the art. Examples of vendors that provide CT and MRI scanners include the General Electric Company of Waukesha, Wis. (GE); Siemens AG of Erlangen, Germany (Siemens); and Koninklijke Philips Electronics of Amsterdam, Netherlands.
Image viewing station 120 is representative of a system that can analyze the medical imagery for anomalies such as folds and polyps and output both the medical imagery and the results of its analysis. Image viewing station 120 may further comprise a processor unit 122, a memory unit 124, an input interface 126, an output interface 128, and program code 130 containing instructions that can be read and executed by the station. Input interface 126 may connect processor unit 122 to an input device such as a keyboard 136, a mouse 138, and/or another suitable device as will be known to a person of skill in the art, including for example and not by way of limitation a voice-activated system. Thus, input interface 126 may allow a user to communicate commands to the processor. One such exemplary command is the execution of program code 130 tangibly embodying the automated fold detection steps disclosed herein. Output interface 128 may further be connected to processor unit 122 and an output device such as a graphical user interface (GUI) 140. Thus, output interface 128 may allow image viewing station 120 to transmit data from the processor to the output device, one such exemplary transmission including medical imagery and anomalies for display to a user on GUI 140.
Memory unit 124 may include conventional semiconductor random access memory (RAM) 142 or other forms of memory known in the art; and one or more computer readable-storage mediums 144, such as a hard drive, floppy drive, read/write CD-ROM, tape drive, flash drive, optical drive, etc. Stored in program code 130 may be an image reconstruction unit 146 for constructing additional imagery from the images acquired by image acquisition unit 110; and a computer-aided detection (CAD) processing unit 148 for automatically detecting anomalies representing folds and, in certain embodiments, anomalies representing polyps of a colon, in accordance with the methods disclosed herein.
It is further noted that while image reconstruction unit 146 and CAD processing unit 148 are depicted as being components within image viewing station 120, one skilled in the art will appreciate that such components may be deployed as parts of separate computers, computer processors, or computer systems. For example, image reconstruction unit 146 may be deployed as part of a virtual colonography review workstation system (e.g., V3D-Colon™ from Viatronix, Inc. of Stony Brook, N.Y.).
In colon acquisition step 210, medical image data representing a colon, or at least a portion of a colon, may be received in a memory such as memory unit 124. In certain embodiments, the medical image data may be a plurality of cross-sectional, two-dimensional (2-D) images of a patient's abdomen. Such imagery may be generated by performing an abdominal scan procedure on a patient using image acquisition unit 110 or other suitable imaging system. In certain other embodiments, the medical image data may be a three-dimensional (3-D) volumetric image or “volume” of the patient's abdomen. A suitable volumetric image may be constructed from the acquired cross-sectional images using computer software. For example, cross-sectional images generated using image acquisition unit 110 may be transferred to image viewing station 120, whereby image reconstruction unit 146 may construct a 3-D volume of the abdominal region by performing a filtered backprojection algorithm on the cross-sectional images as is known in the art. The volumetric image may be comprised of a series of slices. By way of a non-limiting example, each slice image in the volume may be constructed at 512×512 pixels and a spatial resolution of 0.75 millimeters×0.75 millimeters, and the medical image volume may be comprised of a total of 300-600 slices with a spatial resolution of 1 millimeter.
In certain embodiments, multiple volume images of all or portions of the same colon may be obtained at colon acquisition step 210. The multiple volumes may be acquired by imaging a patient's colon at different angles. For example, in clinical practice today, it is common to image the patient in the prone and the supine positions. In other embodiments, the multiple volumes may be acquired by imaging a patient's colon at different times. For example, a patient's colon may be imaged at one point in time and then reimaged at a later point in time, such as five or ten years later.
Fold identification step 220 is then performed on all or a portion of the acquired colon imagery to automatically identify folds that may be of interest to a physician. An example of numerous folds 310 of a colon can be seen in
The processing steps performed in candidate fold detection step 222 take advantage of the fact that folds typically will protrude into and/or cross the colonic lumen while other soft tissue objects such as polyps, stool, and normal colon wall perimeter will not, or at least will not to the same extent.
One skilled in the art will appreciate that there are numerous possibilities for performing colonic wall segmentation step 410. By way of several non-limiting examples, a CT value (i.e., intensity) and CT gradient method as described in U.S. Pat. No. 7,379,572, entitled “Method for computer-aided detection of three-dimensional lesions,” a level set method as described in “Detection of Colon Wall Outer Boundary and Segmentation of the Colon Wall Based on Level Set Methods,” Van Uitert et al., Engineering in Medicine and Biology Society, 2006. EMBS '06. 28th Annual International Conference of the IEEE, Publication Date: Aug. 30, 2006-Sep. 3, 2006, page(s): 3017-3020; or an active contour method may be employed.
The colonic lumen typically consists of air and fluid. As is known in the art, the image units of colonic air will typically exhibit relatively low intensity values (e.g., less than or equal to −800 Hounsfield Units in CT imagery) when compared with the image units of other objects, such as tagged colonic fluid and the colonic wall. In contrast, the image units of tagged colonic fluid will typically exhibit relatively high intensity values (e.g., 300 Hounsfield Units and greater in CT imagery) when compared with adjacent objects such as colonic air and the colonic wall. One means for identifying and segmenting the colonic lumen at step 510 is described in U.S. Pat. No. 6,246,784, “Method for segmenting medical images and detecting surface anomalies in anatomical structures,” which is incorporated herein by reference. In this patent, a region growing technique is described for identifying and segmenting the air and fluid regions of a colon.
Returning to
First, a mask representing an estimate of the colonic lumen is received. In
Alternatively, colonic wall identification step 520 may also be configured to identify and segment the colonic wall and soft tissue objects from the colonic lumen using morphological closing operation(s). One skilled in the art will appreciate that morphological closing operations represent an alternative means to the convex hull operations described hereinabove.
Returning again to
In embodiments where a convex hull algorithm is performed to segment the colonic wall at step 410, candidate fold segmentation step 420 may be configured to perform a morphological closing operation either before or after the convex hull operation as a means to smooth the colonic wall, or reclassify the image units of soft tissue objects to colonic wall that may be inadvertently classified as interstitial tissue. An example of such an artifact is shown in
Again referencing
In describing the structuring element sizes of the various morphological operations described hereinabove, one skilled in the art will appreciate that the exact structuring element size may be changed empirically, depending on numerous factors associated with the imagery in which the system and methods described herein are performed. For example, the structuring element size may be changed depending on the sharpness or resolution of the image data acquired, as a larger structuring element may be required given lower resolution image data and vice versa. In particular, CT and MR typically acquire colon imagery at different resolutions and may therefore require different structuring element sizes to adequately realize the system and methods described herein.
From a segmented representation of the colonic wall that protrudes into the colonic lumen, one means for then segmenting a representation of each individual candidate fold object from other image units of non-tissue in the colonic lumen is to perform a simple thresholding operation. Folds are soft tissue structures and exhibit an intensity range that is suitably different from other image units of colonic air, tagged colonic fluid, and other tagged objects such as stool. In embodiments where the intensity thresholding operation is performed on CT imagery, contiguous image elements having intensities within the range of −650 and 300 Hounsfield Units may be identified as candidate fold objects. A histogram analysis of the image data may be required and performed to obtain suitable parameters for an intensity thresholding operation on non-normalized imagery, such as MR imagery. A filtering step in which objects less than a certain size are removed (e.g., 15 cubic millimeters in volume) may also be performed to eliminate non-fold objects from consideration. This eliminates small objects formed possibly from the curvature of the colonic perimeter or portions of sessile polyps that are of non-interest. One skilled in the art will appreciate that the fold objects themselves are not complicated and thus, do not further require a segmentation operation; however, any suitable segmentation algorithm such as, but not limited to, an active contour or a deformable model segmentation algorithm could be performed on each individual candidate fold object obtained after performing the thresholding operation described hereinabove to further refine the exact pixels or voxels of the candidate fold object.
Again referencing
When comparing folds against false positives that protrude into the colonic lumen and thus may also have been selected in step 222, folds will typically span a greater distance across and often connect opposing regions of the colonic lumen while false positives will typically not. For example, referring back to exemplary fold 320 of
Thus, again referencing
While distance feature extraction step 1010 alone may provide suitable measurements for effectively classifying folds from false positives, a non-distance feature extraction step 1020 may also be performed either separately, or in joint combination with distance feature extraction step 1010, to compute a likelihood or probability that characterizes whether each object is a fold or non-fold. For example, features that describe the total volume (e.g., total number of pixels or voxels) of the candidate fold object or the amount of the candidate fold object that touches the colonic wall (e.g., total number and/or percentage of pixels or voxels) may be computed. Typically, a fold, particularly those in a well-distended colon, will be both larger and wider than other tissue objects (e.g., a small portion of a sessile polyp or part of a pedunculated polyp that may be folded over). Other features describing the shape index, curvature, and/or texture of the candidate fold object may be computed at step 1020 and used for classification.
Classification step 1030 is then performed on the extracted feature values resulting from steps 1010 and/or 1020 to assign each candidate fold object to either a fold or a non-fold class, or to assign a classifier probability of being a fold versus a non-fold, as is known in the art. In certain embodiments, a rules-based or probabilistic classifier such as a Naïve Bayes classifier may be constructed and used at step 1030. As is known in the art, a Naïve Bayes classifier assumes independence between each feature value computed. An initial probability statistic set at zero is increased or decreased by comparing the value of each feature metric against prior learning of the classifier. For example, feature metrics describing a large distance between opposing regions of a candidate fold object and/or a large volume of a candidate fold object may substantially increase the probability statistic. Such probability statistic rules may be derived through supervised or unsupervised learning of the examples of each feature metric value exhibited by samples of folds and samples of false positives, or may be established in other ways. The probability statistic computed by the classification algorithm is then compared against a classification threshold. The threshold may be determined and set empirically by applying the aforementioned feature metric and probability statistic computations to exemplar folds and fold-like false positives as part of a training process and choosing an operating point that classifies folds with a suitable sensitivity at an acceptable false positive rate. In certain embodiments, the classifier may be constructed to output a two-class decision. For example, if the probability exceeds the threshold, the classifier may be constructed to classify the object as in a “fold” class. Otherwise, the classifier may be constructed to classify the object as in a “false positive” class. False positives may then be rejected from further consideration as potential folds.
While in one embodiment a Naïve Bayes rule-based classifier may be used in performing classification step 1030, there are numerous other statistical classification algorithms that may also be suitable. Examples include, but are not limited to, other types of linear classifiers, quadratic classifiers, neural networks, Bayesian networks, support vector machines (SVMs), decision trees, k-nearest neighbors, or other classifiers known in the art of pattern recognition. (See Pattern Classification, Duda et al., John Wiley & Sons, New York, October 2000). One skilled in the art would understand that the features described hereinabove could be quantized into grammatical space and then classified using syntactical classification algorithms.
The classification steps described in reference to
Again referencing
As is visually depicted in
One suitable means for depicting candidate fold objects with the appearance of semi-transparency is alpha compositing. In alpha compositing, in addition to storing a color or grayscale value for each image unit of a candidate fold object in memory unit 124, an additional alpha parameter (i.e., an alpha value) may be set that specifies the amount of semi-transparency in which a candidate fold object should be rendered and displayed on GUI 140. In certain embodiments, any or all fold objects detected in accordance with the methods described hereinabove may be rendered with semi-transparency by first setting an alpha parameter value anywhere greater than 0 and less than 1, where 1 is completely opaque and 0 is completely transparent. In certain other embodiments which are described hereinbelow, only those fold objects that have a probability of obscuring a polyp-like anomaly may be made semi-transparent, so as to permit a physician to not have his vision obscured by folds in proximity to a polyp-like anomaly.
There are numerous means described in the prior art for displaying colon imagery (e.g., CT or MR imagery of an abdominal region) in ways that are suitable for a physician to inspect a colon on an output device such as GUI 140. Any such means may be suitable for rendering and displaying both the colon imagery and the fold objects detected at fold identification step 220 hereinabove including, but not limited to: U.S. Pat. Nos. 5,782,762, 5,920,319, 6,083,162, 6,272,366, 6,366,800, 6,694,163, 6,909,913, and 7,149,564 to Vining et al.; U.S. Pat. Nos. 5,891,030 and 6,928,314 to Johnson et al. For example, the system and methods described herein may be particularly useful for physicians reviewing virtual endoscopic or “fly-through” views of the colon.
Another means for improving a physician's ability to inspect a colon may be derived by combining the automatic fold detection methods described hereinabove with an automated polyp detection method, the latter of which is well-known in the prior art.
In polyp identification step 226, measures of curvature, shape index, sphericity, or combinations thereof may be used as a means to identify the image elements (e.g., the pixels or the voxels) known to exhibit the general characteristics of polyps. Such measures are well-known in the art. One suitable means or “polyp detection algorithm” can be seen in U.S. Pat. No. 7,236,620, “Computer-aided detection methods in volumetric imagery,” which is incorporated herein by reference. In this patent, polyp-like anomalies are identified using spherical summation means. The overall number of false positives that may be detected using such “polyp detection algorithms” may be substantially reduced by further processing each detected polyp-like anomaly using a classification method. Suitable algorithms for classifying polyps from normal tissue (i.e., false positives) include, but are not limited to, those described in references such as: “Computer-assisted detection of colonic polyps with CT colonography using neural networks and binary classification trees,” Medical Physics, Volume 30, Issue 1, pp. 52-60 (January 2003) by Jerebko et al.; “Multiple Neural Network Classification Scheme for Detection of Colonic Polyps in CT Colonography Data Sets,” Academic Radiology, Volume 10, Issue 2, Pages 154-160 by Jerebko et al.; “Support vector machines committee classification method for computer-aided polyp detection in CT colonography,” Academic Radiology, Volume 12, Issue 4, Pages 479-486, by Jerebko et al.; U.S. Pat. No. 7,260,250 to Summers et al.; U.S. Pat. No. 7,440,601 to Summers et al.; U.S. application Ser. No. 12/179,787 to Collins et al; and U.S. application Ser. No. [insert], “Computer-Assisted Analysis Of Colonic Polyps By Morphology In Medical Images” to Van Uitert et al.
Thus, given that each candidate fold object detected and each polyp-like anomaly detected is represented by image units having a particular location in the colon imagery, various polyp-fold location comparisons may then be computed at fold-polyp analysis step 228.
In one example of a simple yet useful analysis of polyp-like anomaly detections and fold-like detections, a computation may be made that determines whether a polyp-like anomaly overlaps or is adjacent to at least one candidate fold object. For example, a binary mask representing polyp-like anomalies detected and/or classified in the colon as suspicious may be logically ANDed with a binary mask representing candidate fold objects detected using any or all of the methods described hereinabove. Candidate fold objects in contact with (i.e., overlapping or bordering) a polyp-like anomaly can be labeled as belonging to a first class, while polyp-like anomalies that are not in contact with a candidate fold object can be labeled as belonging to a second class. Candidate fold objects in contact with or adjacent to a polyp-like anomaly may be specially depicted using the semi-transparency technique described above, so as to allow the physician to “see through” the candidate fold object to the anomaly near or adjacent to the fold. This solves the problem that the polyp-like anomaly might otherwise be obstructed by the fold and thus, the chance that the polyp-like anomaly is missed by the physician would therefore be reduced.
In a further example of a useful comparison between polyp-like anomalies and fold-like objects, a distance map, which may be readily available in embodiments where distance features are computed to classify candidate fold objects at step 1010, may further be used as a means to determine the likelihood that a polyp-like anomaly that is not on a candidate fold object may be obscured by a nearby candidate fold object during inspection viewing. For example, using a distance map, distance measurements may be computed from a common image element reference point at which a polyp-like anomaly touches the colon wall to the point at which the nearest candidate fold object touches the colon wall. The distance measurements may further be classified with other important features (e.g., height of the polyp-like anomaly, height of the candidate fold object) to derive a probability or likelihood of obscuration. Generally speaking, a polyp-like anomaly that is located within a small distance from a candidate fold object and is small in comparison to the candidate fold object is more likely to be obscured and thus, may warrant special depiction at this colon location to assist the physician in inspection. This would help ensure that areas of a colon in which a polyp-like object may be in contact with or proximate to a candidate fold object are carefully reviewed. Previously, no such assistance was provided to assist an inspecting physician. For example, the candidate fold object may be displayed with semi-transparency, as previously described. An indicator may be displayed in the colon to direct the radiologist to review the location of the polyp-like anomaly. Alternatively, the candidate fold object may be electronically “subtracted” from the colon (i.e., the image units of the fold may be made completely transparent) so as to leave a region that may appear as colonic air. Areas of imagery adjacent to the subtracted objects may be smoothed using a Gaussian filter or other suitable technique to minimize artifacts. In such embodiments where the candidate folds are electronically subtracted, to avoid subtracting a fold having a polyp-like anomaly of interest to the physician, ideally, polyp identification step 226 may be performed at or near 100% sensitivity.
Any of the aforementioned special depiction techniques or variables in which to turn on/off the special depiction technique may further be implemented and stored as an “option” in memory unit 124 of image viewing station 120. Each “option” and/or variable may further be presented graphically to a user via GUI 140 and may be selected or changed via an input interface 126 such as keyboard 136, mouse 138, and/or other suitable device. The option may be presented, for example, as a slider bar control (as for example to control degree of transparency), on/off toggle, etc. and the options may be specified or changed either prior to, during, or after the depiction of fold objects detected in accordance with the methods disclosed herein.
In addition to specially depicting fold objects detected, any information computed during the fold detection process may also be presented visually to the physician to aid the inspection of the colon. For example, a reference pattern, a reference color, or a reference label may be presented on or near (i.e., proximate to) each candidate fold object so as to provide the physician with reference landmarks. Such landmarks may be particularly useful in embodiments where the physician reviews multiple images of the same colon, such as the prone and the supine views of a colon, and needs to visually correlate objects or locations in multiple views. The corresponding sets of fold landmarks may also be uniquely depicted. For example, fold landmark with reference number #1 may be colored with a blue mark in each image; fold landmark with reference #2 may be colored with a yellow mark in each image, etc. Other computed information that may be presented includes the feature metric values computed during statistical classification as described hereinabove, which may be useful for a physician in evaluating the suspiciousness of a structure; or the individual probability statistics computed for each feature value metric during statistical classification as described hereinabove; which may be useful for a physician in understanding how and why an automated, computer-implemented decision was made to specially depict certain candidate fold objects in the colon.
It is noted that terms like “preferably,” “commonly,” and “typically” are not utilized herein to limit the scope of this disclosure or to imply that certain features are critical, essential, or even important to the structure or function of the methods and systems disclosed herein. Rather, these terms are merely intended to highlight alternative or additional features that may or may not be utilized in a particular embodiment.
Having described the methods and systems disclosed herein in detail and by reference to specific embodiments thereof, it will be apparent that modifications and variations are possible without departing from the scope of this disclosure. More specifically, although some aspects of this disclosure may be identified herein as preferred or particularly advantageous, it is contemplated that the methods and systems disclosed herein are not necessarily limited to these preferred aspects.
Claims
1. A computer-implemented method of presenting colonic folds in a colon under study to a user comprising:
- a) receiving, through at least one input device, digital imagery representing at least a portion of a colon;
- b) using at least some of said digital imagery, detecting, in at least one processor, at least one candidate colonic fold in said at least a portion of a colon;
- c) classifying, in at least one processor, at least one of said candidate colonic folds as a colonic fold; and
- d) outputting, through at least one output device, information identifying said at least one candidate colonic fold which was classified as a colonic fold.
2. The method of claim 1, wherein detecting at least one candidate colonic fold comprises:
- b1. performing a colonic wall segmentation step; and
- b2. based upon the colonic wall segmentation, performing a candidate fold segmentation step,
- wherein a colonic wall segmentation includes soft tissue objects protruding from said wall into the lumen of said colon.
3. The method of claim 2, wherein performing the colonic wall segmentation step comprises performing at least one of an active contour method, a level set method, and a CT value and CT gradient method.
4. The method of claim 2, wherein performing the colonic wall segmentation step comprises:
- b1a. performing a colon lumen segmentation step; and
- b1b. based upon the colonic lumen segmentation, performing a colon wall identification step.
5. The method of claim 4, wherein performing the colonic lumen segmentation step comprises:
- b1a1. segmenting a representation of air of said colon; and
- b1a2. segmenting a representation of fluid of said colon.
6. The method of claim 4, wherein performing the colonic wall identification step comprises performing at least one of a local convex hull operation and a morphological closing operation.
7. The method of claim 2, wherein performing the candidate fold segmentation step comprises:
- b2a. performing an erosion of the colonic wall; and
- b2b. based on the colonic wall erosion, performing a thresholding operation on the eroded colon wall.
8. The method of claim 7, wherein performing an erosion of the colonic wall comprises performing at least one of a morphological erosion, an active contour, or a distance transform operation.
9. The method of claim 7, wherein performing an erosion of the colonic wall comprises:
- b2a1. performing a first operation on said colon wall to identify a body of said at least one candidate colonic fold; and
- b2a2. performing a second operation on said colon wall to identify a base of said at least one candidate colonic fold.
10. The method of claim 1, wherein classifying at least one of said candidate colonic folds as a colonic fold comprises
- c1. performing at least one of a distance feature extraction step and a non-distance feature extraction step on the candidate colonic fold; and
- c2. based upon the at least one of the distance feature extraction step and the non-distance feature extraction step performed, performing a classification step.
11. The method of claim 10 wherein performing a distance feature extraction step comprises computing at least one distance measurement from a common voxel point to voxel points along a boundary where said candidate colonic fold meets said colon wall.
12. The method of claim 10 wherein performing a non-distance feature extraction step comprises computing at least one of a volume feature, a feature describing the amount the candidate colonic fold touches the colonic wall, a shape index feature, a curvature feature, and a texture feature.
13. The method of claim 10, wherein performing a classification step comprises:
- c2a. computing a discriminant score from at least one of a distance feature measurement extracted and a non-distance feature measurement extracted; and
- c2b. classifying said at least one candidate colonic fold based on said discriminant score computed.
14. The method of claim 10, wherein the classification is a binary decision as to whether the candidate colonic fold is a colonic fold.
15. The method of claim 10, wherein the classification is a probability as to whether the candidate colonic fold is a colonic fold.
16. The method of claim 1, wherein said outputting comprises:
- d1. displaying digital imagery representing at least a portion of the colon on at least one output device; and
- d2. specially depicting said at least one candidate colonic fold which was classified as a colonic fold in said at least a portion of the colon displayed.
17. The method of claim 16 further comprising: in said special depiction of said at least one candidate colonic fold which was classified as a colonic fold, displaying the said at least one candidate colonic fold which was classified as a colonic fold at least partially transparently.
18. The method of claim 16, wherein at least a portion of the digital imagery representing at least a portion of a colon derives from a non-invasive imaging method.
19. The method of claim 18, wherein the non-invasive imaging method is selected form the set composed of CT scanning and MRI imaging.
20. A computer-readable medium having computer-readable instructions stored thereon which, as a result of being executed in a computer system having at least one processor, at least one output device and at least one input device, instructs the computer system to perform a method of presenting colonic folds in a colon under study to a user, comprising:
- a) receiving, through at least one input device, digital imagery representing at least a portion of a colon;
- b) using at least some of said digital imagery, detecting, in at least one processor, at least one candidate colonic fold in said at least a portion of a colon;
- c) classifying, in at least one processor, at least one of said candidate colonic folds as a colonic fold; and
- d) outputting, through at least one output device, information identifying said at least one candidate colonic fold which was classified as a colonic fold.
21. The computer-readable medium of claim 20, wherein detecting at least one candidate colonic fold comprises:
- b1. performing a colonic wall segmentation step; and
- b2. based upon the colonic wall segmentation, performing a candidate fold segmentation step,
- wherein a colonic wall segmentation includes soft tissue objects protruding from said wall into the lumen of said colon.
22. The computer-readable medium of claim 21, wherein performing the colonic wall segmentation step comprises performing at least one of an active contour method, a level set method, and a CT value and CT gradient method.
23. The computer-readable medium of claim 21, wherein performing the colonic wall segmentation step comprises:
- b1a. performing a colon lumen segmentation step; and
- b1b. based upon the colonic lumen segmentation, performing a colon wall identification step.
24. The computer-readable medium of claim 23, wherein performing the colonic lumen segmentation step comprises:
- b1a1. segmenting a representation of air of said colon; and
- b1a2. segmenting a representation of fluid of said colon.
25. The computer-readable medium of claim 23, wherein performing the colonic wall identification step comprises performing at least one of a local convex hull operation and a morphological closing operation.
26. The computer-readable medium of claim 21, wherein performing the candidate fold segmentation step comprises:
- b2a. performing an erosion of the colonic wall; and
- b2b. based on the colonic wall erosion, performing a thresholding operation on the eroded colon wall.
27. The computer-readable medium of claim 26, wherein performing an erosion of the colonic wall comprises performing at least one of a morphological erosion, an active contour, or a distance transform operation.
28. The computer-readable medium of claim 26, wherein performing an erosion of the colonic wall comprises:
- b2a1. performing a first operation on said colon wall to identify a body of said at least one candidate colonic fold; and
- b2a2. performing a second operation on said colon wall to identify a base of said at least one candidate colonic fold.
29. The computer-readable medium of claim 20, wherein classifying at least one of said candidate colonic folds as a colonic fold comprises
- c1. performing at least one of a distance feature extraction step and a non-distance feature extraction step on the candidate colonic fold; and
- c2. based upon the at least one of the distance feature extraction step and the non-distance feature extraction step performed, performing a classification step.
30. The computer-readable medium of claim 29 wherein performing a distance feature extraction step comprises computing at least one distance measurement from a common voxel point to voxel points along a boundary where said candidate colonic fold meets said colon wall.
31. The computer-readable medium of claim 29 wherein performing a non-distance feature extraction step comprises computing at least one of a volume feature, a feature describing the amount the candidate colonic fold touches the colonic wall, a shape index feature, a curvature feature, and a texture feature.
32. The computer-readable medium of claim 29, wherein performing a classification step comprises:
- c2a. computing a discriminant score from at least one of a distance feature measurement extracted and a non-distance feature measurement extracted; and
- c2b. classifying said at least one candidate colonic fold based on said discriminant score computed.
33. The computer-readable medium of claim 29, wherein the classification is a binary decision as to whether the candidate colonic fold is a colonic fold.
34. The computer-readable medium of claim 29, wherein the classification is a probability as to whether the candidate colonic fold is a colonic fold.
35. The computer-readable medium of claim 20, wherein said outputting comprises:
- d1. displaying digital imagery representing at least a portion of the colon on at least one output device; and
- d2. specially depicting said at least one candidate colonic fold which was classified as a colonic fold in said at least a portion of the colon displayed.
36. The computer-readable medium of claim 35 further comprising computer-readable instructions stored thereon which, as a result of being executed in the computer system, instructs the computer system to, in said special depiction of said at least one candidate colonic fold which was classified as a colonic fold, display the said at least one candidate colonic fold which was classified as a colonic fold at least partially transparently.
37. The computer-readable medium of claim 35, wherein at least a portion of the digital imagery representing at least a portion of a colon derives from a non-invasive imaging method.
38. The computer-readable medium of claim 37, wherein the non-invasive imaging method is selected form the set composed of CT scanning and MRI imaging.
39. A system for presenting colonic folds in a colon under study to a user, comprising a computer system with at least one processor, at least one input device and at least one output device, so configured that the system is operable to:
- a) receive, through at least one input device, digital imagery representing at least a portion of a colon;
- b) using at least some of said digital imagery, detect, in at least one processor, at least one candidate colonic fold in said at least a portion of a colon;
- c) classify, in at least one processor, at least one of said candidate colonic folds as a colonic fold; and
- d) output, through at least one output device, information identifying said at least one candidate colonic fold which was classified as a colonic fold.
40. The system of claim 39, wherein detecting at least one candidate colonic fold comprises:
- b1. performing a colonic wall segmentation step; and
- b2. based upon the colonic wall segmentation, performing a candidate fold segmentation step,
- wherein a colonic wall segmentation includes soft tissue objects protruding from said wall into the lumen of said colon.
41. The system of claim 40, wherein performing the colonic wall segmentation step comprises performing at least one of an active contour method, a level set method, and a CT value and CT gradient method.
42. The system of claim 40, wherein performing the colonic wall segmentation step comprises:
- b1a. performing a colon lumen segmentation step; and
- b1b. based upon the colonic lumen segmentation, performing a colon wall identification step.
43. The system of claim 42, wherein performing the colonic lumen segmentation step comprises:
- b1a1. segmenting a representation of air of said colon; and
- b1a2. segmenting a representation of fluid of said colon.
44. The system of claim 42, wherein performing the colonic wall identification step comprises performing at least one of a local convex hull operation and a morphological closing operation.
45. The system of claim 40, wherein performing the candidate fold segmentation step comprises:
- b2a. performing an erosion of the colonic wall; and
- b2b. based on the colonic wall erosion, performing a thresholding operation on the eroded colon wall.
46. The system of claim 45, wherein performing an erosion of the colonic wall comprises performing at least one of a morphological erosion, an active contour, or a distance transform operation.
47. The system of claim 45, wherein performing an erosion of the colonic wall comprises:
- b2a1. performing a first operation on said colon wall to identify a body of said at least one candidate colonic fold; and
- b2a2. performing a second operation on said colon wall to identify a base of said at least one candidate colonic fold.
48. The system of claim 39, wherein classifying at least one of said candidate colonic folds as a colonic fold comprises
- c1. performing at least one of a distance feature extraction step and a non-distance feature extraction step on the candidate colonic fold; and
- c2. based upon the at least one of the distance feature extraction step and the non-distance feature extraction step performed, performing a classification step.
49. The system of claim 48 wherein performing a distance feature extraction step comprises computing at least one distance measurement from a common voxel point to voxel points along a boundary where said candidate colonic fold meets said colon wall.
50. The system of claim 48 wherein performing a non-distance feature extraction step comprises computing at least one of a volume feature, a feature describing the amount the candidate colonic fold touches the colonic wall, a shape index feature, a curvature feature, and a texture feature.
51. The system of claim 48, wherein performing a classification step comprises:
- c2a. computing a discriminant score from at least one of a distance feature measurement extracted and a non-distance feature measurement extracted; and
- c2b. classifying said at least one candidate colonic fold based on said discriminant score computed.
52. The system of claim 48, wherein the classification is a binary decision as to whether the candidate colonic fold is a colonic fold.
53. The system of claim 48, wherein the classification is a probability as to whether the candidate colonic fold is a colonic fold.
54. The system of claim 39, wherein said outputting comprises:
- d1. displaying digital imagery representing at least a portion of the colon on at least one output device; and
- d2. specially depicting said at least one candidate colonic fold which was classified as a colonic fold in said at least a portion of the colon displayed.
55. The system of claim 54 wherein the system further is operable, in said special depiction of said at least one candidate colonic fold which was classified as a colonic fold, to display the said at least one candidate colonic fold which was classified as a colonic fold at least partially transparently.
56. The system of claim 54, wherein at least a portion of the digital imagery representing at least a portion of a colon derives from a non-invasive imaging method.
57. The system of claim 56, wherein the non-invasive imaging method is selected form the set composed of CT scanning and MRI imaging.
58. A computer-implemented method of presenting colonic folds in a colon under study to a user comprising:
- a) receiving, through at least one input device, digital imagery representing at least a portion of a colon;
- b) using at least some of said digital imagery, detecting, in at least one processor, at least a portion of a colonic wall in said at least a portion of a colon;
- c) segmenting, in at least one processor, at least one candidate colonic fold from said at least a portion of a colonic wall; and
- d) outputting, through at least one output device, information identifying said at least one candidate colonic fold which was segmented from said at least a portion of a colonic wall.
59. The method of claim 58 wherein detecting at least a portion of a colonic wall in said at least a portion of a colon comprises performing at least one of an active contour method, a level set method, and a CT value and CT gradient method.
60. The method of claim 58, wherein detecting at least a portion of a colonic wall in said at least a portion of a colon comprises:
- b1a. performing a colon lumen segmentation step; and
- b1b. based upon the colonic lumen segmentation, performing a colon wall identification step.
61. The method of claim 60 wherein performing the colonic lumen segmentation step comprises:
- b1a1. segmenting a representation of air of said colon; and
- b1a2. segmenting a representation of fluid of said colon.
62. The method of claim 60, wherein performing the colonic wall identification step comprises performing at least one of a local convex hull operation and a morphological closing operation.
63. The method of claim 58, wherein segmenting at least one candidate colonic fold from said at least a portion of a colonic wall comprises:
- b2a. performing an erosion of the colonic wall; and
- b2b. based on the colonic wall erosion, performing a thresholding operation on the eroded colon wall.
64. The method of claim 63, wherein performing an erosion of the colonic wall comprises performing at least one of a morphological erosion, an active contour, or a distance transform operation.
65. The method of claim 63, wherein performing an erosion of the colonic wall comprises:
- b2a1. performing a first operation on said colon wall to identify a body of said at least one candidate colonic fold; and
- b2a2. performing a second operation on said colon wall to identify a base of said at least one candidate colonic fold.
66. The method of claim 58 further comprising: classifying, in at least one processor, at least one of said candidate colonic folds segmented from said at least a portion of a colonic wall as a colonic fold.
67. The method of claim 66, wherein classifying at least one of said candidate colonic folds as a colonic fold comprises
- c1. performing at least one of a distance feature extraction step and a non-distance feature extraction step on the candidate colonic fold; and
- c2. based upon the at least one of the distance feature extraction step and the non-distance feature extraction step performed, performing a classification step.
68. The method of claim 67, wherein said outputting comprises:
- d1. displaying digital imagery representing at least a portion of the colon on at least one output device; and
- d2. specially depicting said at least one candidate colonic fold which was classified as a colonic fold in said at least a portion of the colon displayed.
69. A computer-generated user interface for presenting a graphical representation of a colon, the user interface comprising a depiction of the colon; wherein regions of the colon segmented as colonic folds are displayed at least partially transparent.
Type: Application
Filed: Jan 29, 2009
Publication Date: Jul 29, 2010
Inventors: Ryan McGinnis (London, OH), Kevin Woods (Beavercreek, OH), Senthil Periaswamy (Beavercreek, OH), Robert L. Van Uitert (Hollis, NH)
Application Number: 12/362,111
International Classification: G06K 9/00 (20060101);