STEREOSCOPIC ENDOSCOPE APPARATUS
To provide a stereoscopic endoscope apparatus capable of reducing the visual interference caused when a treatment instrument approaches an imaging unit. Provided is a stereoscopic endoscope apparatus that includes a plurality of imaging units and a channel having an operable treatment instrument inserted therethrough and stereoscopically displays an image obtained from the plurality of imaging units, the stereoscopic endoscope including: a front end detecting unit that detects a front end of the treatment instrument within the image acquired by the imaging units; an area determining unit that determines an area to be displayed as a two-dimensional image; and a two-dimensional image processing unit that two-dimensionally images the area to be displayed as a two-dimensional image, in which the area to be displayed as a two-dimensional image is an area where the treatment instrument is seen.
Latest Canon Patents:
- MEDICAL INFORMATION PROCESSING DEVICE, MEDICAL INFORMATION PROCESSING METHOD, AND STORAGE MEDIUM
- MEDICAL LEARNING APPARATUS, MEDICAL LEARNING METHOD, AND MEDICAL INFORMATION PROCESSING SYSTEM
- MEDICAL INFORMATION PROCESSING APPARATUS, MEDICAL INFORMATION PROCESSING SYSTEM, AND NON-TRANSITORY COMPUTER READABLE MEDIUM
- AUTOMATIC ANALYZING APPARATUS
- MEDICAL IMAGE PROCESSING APPARATUS, METHOD OF MEDICAL IMAGE PROCESSING, AND NONVOLATILE COMPUTER READABLE STORAGE MEDIUM STORING THEREIN MEDICAL IMAGE PROCESSING PROGRAM
1. Field of the Invention
The present invention relates to a stereoscopic endoscope apparatus, and particularly, to an image process for satisfactorily observing a subject at a comparatively close distance using a plurality of imaging units installed in a front end of the stereoscopic endoscope apparatus.
2. Description of the Related Art
As illustrated in
In such a stereoscopic endoscope, since the convergence angle which is determined by the stereo cameras is too large with respect to the treatment instrument that closely approaches the stereo cameras, stereoscopic displaying cannot be easily performed. For example, with regard to the images which are acquired by the stereo cameras 101R and 101L in a state where the treatment instrument is delivered from the channel 102, the image of the stereo camera 101L is acquired as illustrated in
In this state, the treatment instruments within the images acquired from the respective cameras overlap each other in a deviated state as illustrated in
In order to solve this problem, Japanese Patent Application Laid-Open No. 2004-65804 discloses a method of two-dimensionally displaying a predetermined area or a method of generating a mask image and overlapping the mask image with an original image.
SUMMARY OF THE INVENTIONHowever, in the related art which is disclosed in Japanese Patent Application Laid-Open No. 2004-65804, an area which is not easily displayed as a stereoscopic image is obtained in advance and only a single-eye image for that area is displayed or the area is overlapped with the mask image. For this reason, there is a problem that other areas are also displayed as a two-dimensional image or displayed with a mask, even when the treatment instrument is slightly delivered from the channel. The invention is made in view of the above-described problems, and it is an object of the invention to provide a structure capable of ensuring a stereoscopically displayed area by detecting a front end of a treatment instrument and with minimized amount of processes.
In order to attain the above-described object, the apparatus according to an aspect of the invention includes the following configuration. That is, a first aspect of the invention relates to a stereoscopic endoscope apparatus capable of operating a treatment instrument through a channel and stereoscopically displaying an image acquired by a plurality of imaging units, the stereoscopic endoscope apparatus including: a front end detecting unit that detects a front end of the treatment instrument in at least one arbitrary image from the plurality of images acquired by the plurality of imaging units; an area determining unit that determines an area which is distant from the front end of the treatment instrument by a predetermined distance and is to be displayed as a two-dimensional image; and a two-dimensional image processing unit that two-dimensionally displays the area in an image used for the stereoscopic display in the plurality of images acquired by the plurality of imaging units.
Further, a second aspect of the invention relates to a stereoscopic endoscope apparatus capable of operating a treatment instrument through a channel and stereoscopically displaying an image acquired by a plurality of imaging units, the stereoscopic endoscope apparatus including: a front end detecting unit that detects a front end of the treatment instrument in at least one arbitrary image from the plurality of images acquired by the plurality of imaging units; a deformation area determining unit that determines an area where an image is deformed within an image distant from the front end of the treatment instrument by a predetermined distance; and a deformation unit that deforms the deformation area determined in the images used for the stereoscopic display in the plurality of images acquired by the plurality of imaging units in a shape in which the disparity of the stereoscopic image decreases.
According to the invention, since the visual interference may be reduced by processing only an area which is essentially required in a stereoscopic observation area, a more easily operable stereoscopic endoscope can be provided.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Preferred embodiments of the present invention will now be described in detail in accordance with the accompanying drawings.
Hereinafter, exemplary embodiments of a processing apparatus and a processing method according to the invention will be described in detail according to the accompanying drawings. However, the scope of the invention is not limited to the example illustrated in the drawings.
First EmbodimentA stereoscopic endoscope apparatus according to the embodiment reduces the visual interference by detecting a front end of a treatment instrument which is operable through a channel and displaying a portion of the treatment instrument other than a predetermined area of the front end as a two-dimensional image.
Next, the process of the front end detecting unit will be described by referring to
For simple description, the camera is assumed as a pinhole camera, and the thickness of the treatment instrument is disregarded. Further, the distance up to the image plane from the camera is denoted by f, the pixel pitch of the imaging plane is denoted by p, and the position of the front end on the image plane when the treatment instrument 24 is delivered by L (the distance of the image plane from the center 28) is denoted by y. At this time, the following relation is obtained.
In the same way,
In these parameters, p and f may be obtained in advance for each imaging condition. yd and xd1 become constant values by the stereoscopic endoscope to be used. Therefore, n and m may be obtained as a function of the length L in which the treatment instrument 24 is delivered. Consequently, when the template of the image of the treatment instrument is created in advance and the template matching is performed on the trace of (m, n) depicted by (Expression 1) and (Expression 2), the front end position can be easily detected.
This process is not limited only to the image of the imaging unit 101R, and for example, when the same detection is also performed on the image of 101L and the combination thereof is performed, the precision may be further improved.
The process of the image processing unit 13 will be described by referring to
In the image processing unit 13, the area 31 which is present within a predetermined distance in the front end of the treatment instrument and which is obtained by the area determining unit is specified. Further, the area 32 which is distant from the front end of the treatment instrument by a predetermined distance is specified. Note that, however, the area 32 may be determined as an area where only the treatment instrument is present.
The length of the predetermined distance used when specifying the area 31 is not particularly limited, and may be arbitrarily determined in consideration of a degree in which the visual interference is reduced. Although there is no particular limitation, Ny/4 pixels or less is desirable, where Ny denotes an image size (a number of pixels) in the longitudinal direction in
When the area of the treatment instrument seen in the image is smaller than the predetermined distance from the front end thereof, the area is not displayed as a two-dimensional image.
That is, a corresponding area 122 within the image L is copied to the area where the treatment instrument is seen in the area 32 of the image R of
Alternatively, the treatment instrument may be made to be invisible by covering the image of the area 32 with a mask or the like or interpolating an image around the area 32 instead of the process of replacing the area 32. In this case, the images of the areas 121 and 122 in
The image is displayed in the display unit 14. As the display unit 14, a conventional stereoscopic display device may be used.
As described above, it is possible to provide a stereoscopic endoscope image capable of reducing the visual interference by detecting the front end of the treatment instrument and the area which is seen in the image and replacing the area of the treatment instrument within the image other than a predetermined area of the front end with the area at the same position of the other image.
The front end of the treatment instrument may be detected by a method of using an image processing or by a method of installing a sensor at the front end thereof. In the case of using the image processing, a front end of a moving subject may be determined as the front end of the treatment instrument based on a change in the image, or the shape of the front end of the treatment instrument may be searched.
Second EmbodimentIn embodiment, the same effect is aimed by deforming the image, instead of replacing the area of a part of the treatment instrument seen in the image with the area at the same position of the other image in the first embodiment.
Referring to
Due to the above-described process, although a space different from the real state is perceived in the stereoscopic space, the visual interference which prohibits the stereoscopic image may instead be reduced. Since this portion is not an area where the treatment instrument is operated, as described above, an amount of deformation can be allowed in the area.
The front end of the treatment instrument within the image may be obtained by the method of the first embodiment. Alternatively, since an area closer to the imaging unit 101 (the area where the image is deformed) corresponds to an area which includes a place where the treatment instrument is first seen in the image, the area can be set to an area which includes the front end of the treatment instrument within the image and an arbitrary range from the front end in a direction toward a place where the treatment instrument is first seen in the image. The deformation of the rectangular area mapping shape is not limited to the above trapezoid provided that the deformation reduces the disparity.
Other EmbodimentsThe first embodiment and the second embodiment can both be realized by a computer.
Referring to
Referring to
As other methods, a method may be exemplified in which a low-pass filter, for example, a Gaussian filter, is applied to an area which is distant by a predetermined distance from the front end of the treatment instrument within the image. In this method, since blur occurs in an area where the filter is applied, the visual interference may be reduced.
With regard to the method of detecting the front end of the treatment instrument, a matching method is supposed in consideration of the case where the treatment instrument deforms or freely moves instead of the template matching on the trace of the treatment instrument described by referring to
As described above, in the stereoscopic endoscope image, since a part of the area where the treatment instrument is seen is switched between the left and right images, visual interference can be reduced. Further, since the position where the treatment instrument is present may be checked by marking a part of the area of the front end of the treatment instrument as it is at all times, it is possible to prevent the body tissue from being damaged depending on a state where the treatment instrument is not visible or is not easily visible. Furthermore, when the treatment instrument seeing method is changed by partly deforming a space which is perceived as a stereoscopic display instead of the two-dimensional image, also, the visual interference may be reduced. Further, even in a method of adopting the image process in which the center and the periphery of the image are weighted differently, the visual interference may be reduced.
In the above-described embodiments, the invention has been described by exemplifying the endoscope with two imaging units. However, the stereoscopic observation may be performed in a manner such that a plurality of (two or more) imaging units are provided and the process of the invention is performed on any one of the plurality of images obtained in the respective imaging units. At this time, the process of the invention does not need to be performed on all images acquired in the plurality of imaging units, and the area where the two-dimensional image is displayed or the area where the image is deformed may be determined in only at least one arbitrary image from the obtained images.
Further, the imaging unit may include an optical system. One imaging optical system may include two or more imaging units. For example, a configuration with two or more imaging elements through one lens may be adopted.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2011-095206, filed Apr. 21, 2011, which is hereby incorporated by reference herein in its entirety.
Claims
1. A stereoscopic endoscope apparatus that includes a plurality of imaging units and a channel having an operable treatment instrument inserted therethrough and stereoscopically displays an image obtained from the plurality of imaging units, the stereoscopic endoscope apparatus comprising:
- a front end detecting unit configured to detect a front end of the treatment instrument within at least one arbitrary image from the plurality of images acquired by the imaging units;
- an area determining unit configured to determine an area to be displayed as a two-dimensional image; and
- a two-dimensional image processing unit configured to two-dimensionally image the area to be displayed as a two-dimensional image,
- wherein the area to be displayed as a two-dimensional image is an area where the treatment instrument is seen.
2. The stereoscopic endoscope apparatus according to claim 1,
- wherein the area to be displayed as a two-dimensional image does not include the front end of the treatment instrument.
3. The stereoscopic endoscope apparatus according to claim 1,
- wherein an area within a predetermined distance from the front end of the treatment instrument is not two-dimensionally imaged.
4. A stereoscopic endoscope apparatus that includes a plurality of imaging units and a channel having an operable treatment instrument inserted therethrough and stereoscopically displays an image obtained from the plurality of imaging units, the stereoscopic endoscope apparatus comprising:
- a front end detecting unit configured to detect a front end of the treatment instrument within at least one arbitrary image from the plurality of images acquired by the imaging units;
- a deformation area determining unit configured to determine an image deformation area within the image; and
- an image deformation processing unit configured to deform an image within the deformation area, wherein the image deformation processing unit is an image processing unit configured to perform an image deformation for reducing a disparity of an image acquired by the plurality of images.
5. The stereoscopic endoscope apparatus according to claim 4,
- wherein the deformation area is an area where the front end of the treatment instrument is not included and the treatment instrument is seen.
6. The stereoscopic endoscope apparatus according to claim 4,
- wherein a shape of the deformed deformation area is a trapezoid.
7. An image processing method in a stereoscopic endoscope system that includes a plurality of imaging units and a treatment instrument and stereoscopically displays an image obtained from the plurality of imaging units, the image processing method comprising:
- capturing an image;
- detecting a front end of the treatment instrument within at least one arbitrary image from the plurality of images acquired by the imaging units;
- determining an area where the front end of the treatment instrument is not included and the treatment instrument is seen; and
- performing an image process of displaying the area of the image used for the stereoscopic display in the plurality of images as a two-dimensional image.
8. An image processing method in a stereoscopic endoscope system that includes a plurality of imaging units and a treatment instrument and stereoscopically displays an image obtained from the plurality of imaging units, the image processing method comprising:
- capturing an image;
- detecting a front end of the treatment instrument within at least one arbitrary image from the plurality of images acquired by the imaging units;
- determining a deformation area that is an area distant from the front end of the treatment instrument by a predetermined distance and in which the treatment instrument is seen and the image is deformed; and
- deforming the deformation area in a predetermined shape.
Type: Application
Filed: Mar 6, 2012
Publication Date: Oct 25, 2012
Applicant: CANON KABUSHIKI KAISHA (Tokyo)
Inventor: Akihiro Katayama (Zama-shi)
Application Number: 13/412,701
International Classification: A61B 1/04 (20060101); A61B 1/018 (20060101);