Method and System for Detection of Contrast Injection Fluoroscopic Image Sequences
A method and system for detecting a spatial and temporal location of a contrast injection in a fluoroscopic image sequence is disclosed. Training volumes generated by stacking a sequence of 2D fluoroscopic images in time order are annotated with ground truth contrast injection points. A heart rate is globally estimated for each training volume, and local frequency and phase is estimated in a neighborhood of the ground truth contrast injection point for each training volume. Frequency and phase invariant features are extracted from each training volume based on the heart rate, local frequency and phase, and a detector is trained based on the training volumes and the features extracted for each training volume. The detector can be used to detect the spatial and temporal location of a contrast injection in a fluoroscopic image sequence.
Latest Siemens Aktiengesellschaft Patents:
- Method for Determining the Integrity of a Data Processing Operation, Device, and Data Processing Installations
- Composite of Heat Sink and Electrical and/or Electronic Component
- PERMANENT MAGNET SYNCHRONOUS MOTOR
- Encapsulated MEMS Switching Element, Device and Production Method
- Rotor Lamination, Method for Producing a Rotor Lamination and Electric Machine
This application is a divisional of U.S. application Ser. No. 12/231,770, filed Sep. 5, 2008 and claims the benefit of U.S. Provisional Application No. 60/974,100, filed Sep. 21, 2007, the disclosure of which is herein incorporated by reference.
BACKGROUND OF THE INVENTIONThe present invention relates to detection of contrast injection in fluoroscopic image sequences, and more particularly to detection of where and when a contrast agent is injected in a fluoroscopic image sequence.
Coronary angiography is a procedure that is recommended preoperatively for patients who are suffering from or at risk for coronary artery disease. Angiography is a medical imaging technique in which X-ray images are used to visualize internal blood filled structures, such as arteries, veins, and the heart chambers. Since blood has the same radiodensity as the surrounding tissues, these blood filled structures cannot be differentiated from the surrounding tissue using conventional radiology. In angiography, a catheter is inserted into a blood vessel, typically in the groin or the arm. The catheter is guided and positioned either in the heart or in arteries near the heart, and a contrast agent is added to the blood via the catheter to make the blood vessels in the heart visible via X-ray. As the contrast agent travels down the branches of the coronary artery, the vessel branches become visible in the X-ray (fluoroscopic) image. The X-ray images are taken over a period of time, which results in a sequence of fluoroscopic images.
The moment when the contrast is injected provides important temporal information for the automatic analysis of vessels. This temporal information can be used to trigger the starting of automatic vessel detection. For example, this temporal information can be used in implementing Digital Subtraction Angiography (DSA), which detects vessels by subtracting a pre-contrast image or “mask image” from later fluoroscopic images once the contrast agent has been introduced. Furthermore, the spatial location of the contrast injection point can be used as a starting point for vessel detection and tracking. Accordingly, it is desirable to detect the time and location of a contrast injection in a fluoroscopic image sequence.
BRIEF SUMMARY OF THE INVENTIONThe present invention provides a method and system for detecting the temporal and spatial location of a contrast injection in a sequence of fluoroscopic images. Embodiments of the present invention detect the contrast injection in a fluoroscopic image sequence as a 3-dimensional detection problem, with two spatial dimensions and one time dimension.
In one embodiment of the present invention, training volumes are received. Each training volumes is generated by stacking a sequence of 2D fluoroscopic images in time order and has two spatial dimensions and one temporal dimension. Each training volume is annotated with a ground truth contrast injection point. A heart rate is globally estimated for each training volume, and local frequency and phase is estimated in a neighborhood of the ground truth contrast injection point for each training volume. Frequency and phase invariant features are extracted from each training volume based on the heart rate, local frequency and phase. A detector, for detecting a spatial and temporal location of a contrast injection in a fluoroscopic image sequence, is trained based on the training volumes and the features extracted for each training volume. The detector can be trained using a probabilistic boosting tree (PBT).
In another embodiment of the present invention, a fluoroscopic image sequence is received, and a 3D volume is generated by stacking the fluoroscopic image sequence. The 3D volume has two spatial dimensions and one temporal dimension, and can be generated by stacking the 2D fluoroscopic images in time order and interpolating the stacked 2D fluoroscopic images to generate a continuous 3D volume. The spatial and temporal location of the contrast injection in the fluoroscopic image sequence is then detected by processing the 3D volume using a trained contrast injection detector. The trained contrast injection detector can be trained using a PBT based on training examples and frequency and phase invariant features extracted from the training examples.
These and other advantages of the invention will be apparent to those of ordinary skill in the art by reference to the following detailed description and the accompanying drawings.
The present invention relates to detection of a contrast injection time and location in a fluoroscopic image sequence. Embodiments of the present invention are described herein to give a visual understanding of the contrast injection detection method. A digital image is often composed of digital representations of one or more objects (or shapes). The digital representation of an object is often described herein in terms of identifying and manipulating the objects. Such manipulations are virtual manipulations accomplished in the memory or other circuitry/hardware of a computer system. Accordingly, is to be understood that embodiments of the present invention may be performed within a computer system using data stored within the computer system.
A sequence of fluoroscopic images contains multiple 2D X-ray images obtained in real time.
Embodiments of the present invention utilize the facts that vessel motion is mainly because of the heart beating and the heart beating is periodic. As shown in image 204 of
Returning to
Returning to
At step 308, frequency and phase invariant features are extracted from each sequence (training volume) using the estimated heart rate, local frequency, and phase. For classification of a candidate location (x,y,t) in a volume, a sub-window is aligned in time with the start of the estimated vessel period, such that the start of the sub-window is expressed as t_start=phase+floor((t−phase)/period)*period. Floor (x) is the largest integer which is not greater than x, and the period can be estimated as T=1024/f. The sub-window is extended in time for 2 periods to t_end=t_start+2*period. Thus, for a given amplitude a, the sub-window is from (x,y−a,t_start) to (x,y+a,t_end). For example, a can be fixed at 20 pixels.
Features are then defined relative to the sub-window and parameterized by a height and a shift as fractions of the amplitude and period. This provides invariance to differing phases and local frequencies in the same volume in different sub-windows, as well as different phases and frequencies (global heart rates and local frequencies) in different sequences. The height and shift can be discretized into 10 and 20 values, respectively. At each (height, shift) pair, features are generated based on intensity, gradient, difference in intensity one period ahead, and difference in intensity half a period ahead with inverted amplitude. At each value of shift, mean intensity features are generated for all heights (i.e., mean of the current column is the sub-window), and the difference in location of the maximum value in the previous and next shifts. Features are also generated that are global to the whole sub-window based on differences in intensity values in frames previous to the candidate location and in frames after the candidate location, as well as a feature based on the correlation between pixels for two consecutive heart beat periods. This means that more features are generated around the candidate location.
At step 310, a detector is trained based on the training volumes and the features extracted from the training volumes. As described above, each of the training volumes is annotated with the location of a ground truth contrast injection point. These ground truth locations are used as positive training examples, and other locations in the volumes are used as negative training examples.
Once a detector is trained based on the training volumes and the extracted features, the detector can be used to detect the spatial and temporal location of a contrast injection in fluoroscopic image sequences.
At step 804, a 3D volume is generated from the fluoroscopic image sequence by stacking the 2D fluoroscopic images in the sequence. The fluoroscopic images are stacked in time order, and the discrete images are interpolated based on a sampling rate to generate a continuous 3D volume.
At step 806, the trained detector is used to detect the spatial and temporal location of the contrast injection point in the fluoroscopic image sequence. The detector is trained using the method of
The spatial and temporal location of a contrast injection point can be used in automated image processing methods, such as vessel extraction or segmentation methods. For example, automated vessel segmentation methods, such as coronary digital subtraction angiography (DSA), may return erroneous results when trying to segment images in a fluoroscopic image sequence prior to the contrast injection. Such automated vessel segmentation methods can restrict segmentation to after the detected contrast injection point in a fluoroscopic image sequence. The spatial and temporal location of the contrast injection point can also be used in coronary DSA to determine a pre-contrast image, as well as for virtual contrast to determine the model cycle.
The above-described methods for contrast injection detection can be implemented on a computer using well-known computer processors, memory units, storage devices, computer software, and other components. A high-level block diagram of such a computer is illustrated in
The foregoing Detailed Description is to be understood as being in every respect illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is not to be determined from the Detailed Description, but rather from the claims as interpreted according to the full breadth permitted by the patent laws. It is to be understood that the embodiments shown and described herein are only illustrative of the principles of the present invention and that various modifications may be implemented by those skilled in the art without departing from the scope and spirit of the invention. Those skilled in the art could implement various other feature combinations without departing from the scope and spirit of the invention.
Claims
1. A method for detecting a spatial and temporal location of a contrast injection in a fluoroscopic image sequence, comprising:
- receiving a fluoroscopic image sequence;
- generating a 3D volume by stacking the fluoroscopic image sequence; and
- detecting a spatial and temporal location of the contrast injection in the fluoroscopic image sequence by processing the 3D volume using a trained contrast injection detector.
2. The method of claim 1, wherein said step of generating a 3D volume comprises:
- stacking a plurality of 2D fluoroscopic images in the fluoroscopic image sequence in time order; and
- interpolating the stacked 2D fluoroscopic images to generate a continuous 3D volume having two spatial dimensions and one time dimension.
3. The method of claim 1, wherein the trained contrast injection detector is trained using a probabilistic boosting tree (PBT) based on training examples and frequency and phase invariant features extracted from the training examples.
4. The method of claim 1, wherein the 3D volume has two spatial dimensions and one temporal dimension and said step of detecting a spatial and temporal location of the contrast injection in the fluoroscopic image sequence comprises:
- detecting a contrast injection point in the 3D volume using the contrast injection detector, wherein coordinates of the detected contrast injection point give the spatial and temporal location of the contrast injection.
5. An apparatus for detecting a spatial and temporal location of a contrast injection in a fluoroscopic image sequence, comprising:
- means for receiving a fluoroscopic image sequence;
- means for generating a 3D volume by stacking the fluoroscopic image sequence; and
- means for detecting a spatial and temporal location of the contrast injection in the fluoroscopic image sequence by processing the 3D volume using a trained contrast injection detector.
6. The apparatus of claim 5, wherein said means for generating a 3D volume comprises:
- means for stacking a plurality of 2D fluoroscopic images in the fluoroscopic image sequence in time order; and
- means for interpolating the stacked 2D fluoroscopic images to generate a continuous 3D volume having two spatial dimensions and one time dimension.
7. The apparatus of claim 5, wherein the trained contrast injection detector is trained using a probabilistic boosting tree (PBT) based on training examples and frequency and phase invariant features extracted from the training examples.
8. A computer readable medium encoded with computer executable instructions for detecting a spatial and temporal location of a contrast injection in a fluoroscopic image sequence, the computer executable instructions defining steps comprising:
- receiving a fluoroscopic image sequence;
- generating a 3D volume by stacking the fluoroscopic image sequence; and
- detecting a spatial and temporal location of the contrast injection in the fluoroscopic image sequence by processing the 3D volume using a trained contrast injection detector.
9. The computer readable medium of claim 8, wherein the computer executable instructions defining the step of generating a 3D volume comprise computer executable instructions defining the steps of:
- stacking a plurality of 2D fluoroscopic images in the fluoroscopic image sequence in time order; and
- interpolating the stacked 2D fluoroscopic images to generate a continuous 3D volume having two spatial dimensions and one time dimension.
10. The computer readable medium of claim 8, wherein the trained contrast injection detector is trained using a probabilistic boosting tree (PBT) based on training examples and frequency and phase invariant features extracted from the training examples.
11. The computer readable medium of claim 8, wherein the 3D volume has two spatial dimensions and one temporal dimension and the computer executable instructions defining the step of detecting a spatial and temporal location of the contrast injection in the fluoroscopic image sequence comprise computer executable instructions defining the step of:
- detecting a contrast injection point in the 3D volume using the contrast injection detector, wherein coordinates of the detected contrast injection point give the spatial and temporal location of the contrast injection.
Type: Application
Filed: Apr 25, 2012
Publication Date: Oct 11, 2012
Applicant: Siemens Aktiengesellschaft (Munich)
Inventors: Benjamin J. Sapp (Philadelphia, PA), Wei Zhang (Falls Church, VA), Bogdan Georgescu (Plainsboro, NJ), Simone Prummer (Forchheim), Dorin Comaniciu (Princeton Junction, NJ)
Application Number: 13/455,619
International Classification: G06K 9/00 (20060101);