Abstract: A method for automatic registration of landmarks in 3D and 2D images of an organ comprises: using a first set of coordinates of identified first, second and third landmarks of the organ, derived from a 3D surface representation of the organ, and a second set of coordinates of the landmarks, derived from 2D laparoscopic images of the organ, to register the three landmarks as identified in the 3D surface representation with the three landmarks as identified in the 2D images. The third landmark comprises a plurality of points defining a path between two points characterizing the first and second landmarks The identification of the first, second and third landmarks in the 3D representation and the 2D images, the derivation of the first and second sets of coordinates, and the registration, based on the derived first and second sets of coordinates, are performed automatically.

Abstract: A method for automatic registration of landmarks in 3D and 2D images of an organ comprises: using a first set of coordinates of identified first, second and third landmarks of the organ, derived from a 3D surface representation of the organ, and a second set of coordinates of the landmarks, derived from 2D laparoscopic images of the organ, to register the three landmarks as identified in the 3D surface representation with the three landmarks as identified in the 2D images. The third landmark comprises a plurality of points defining a path between two points characterizing the first and second landmarks The identification of the first, second and third landmarks in the 3D representation and the 2D images, the derivation of the first and second sets of coordinates, and the registration, based on the derived first and second sets of coordinates, are performed automatically.

Abstract: Various aspects of a system and a method to provide assistance in a surgery in presence of tissue deformation are disclosed herein. In accordance with an embodiment, the system includes an electronic device that receives one or more tissue material properties of a plurality of surface structures of an anatomical portion. One or more boundary conditions associated with the anatomical portion may also be received. Surface displacement of the anatomical portion may be determined by matching a first surface of the anatomical portion before deformation with a corresponding second surface of the anatomical portion after the deformation. The volume displacement field of the anatomical portion may be computed based on the determined surface displacement, the received one or more tissue material properties, and the received one or more boundary conditions.

Abstract: An image registration system, and a method of operation of an image registration system thereof, including: an imaging unit for obtaining a pre-operation non-invasive imaging volume and for obtaining an intra-operation non-invasive imaging volume; and a processing unit including: a rigid registration module for generating a rigid registered volume based on the pre-operation non-invasive imaging volume, a region of interest module for isolating a region of interest from the intra-operation non-invasive imaging volume, a point generation module, coupled to the region of interest module, for determining feature points of the region of interest, an optimization module, coupled to the point generation module, for matching the feature points with corresponding points of the rigid registered volume for generating a matched point cloud, and an interpolation module, coupled to the optimization module, for generating a non-rigid registered volume based on the matched point cloud for display on a display interface.

Abstract: An apparatus and method for localization of an internal organ of a subject to provide assistance during surgery, includes capture of a sequence of video frames of one or more internal organs in a body of a subject by insertion of an image capture device in a body of the subject. The apparatus further includes circuitry that computes appearance data of an internal organ of interest in a test video frame selected from the captured sequence of video frames. Orientation data of the internal organ of interest is determined within the body of the subject based on a known orientation of the image capture device and a maximum correlation of the test video frame with a specified anatomical template for the internal organ of interest. Supplemental information is generated to localize at least a portion of the internal organ of interest of the subject in the surgery.

Abstract: A surgical assistive apparatus and method to provide assistance during surgery, includes an organ boundary localization circuit that selects a test video frame from a captured sequence of video frames and derives a first global region boundary of an internal organ of interest in the selected test video frame by integration of an appearance likelihood result, a global segmentation result, and a global edge detection result associated with the internal organ of interest. A plurality of local region boundaries are determined for a plurality of local sub regions of the internal organ of interest and a second global region boundary is generated for the internal organ of interest based on the determined plurality of local region boundaries and the first global region boundary. The internal organ of interest is localized based on the generated second global region boundary.

Abstract: A surgical assistive device, which handles co-localization of an internal organ of interest and a neighboring organ based on images obtained during surgery, selects a test video frame from a sequence of video frames and generates first localization likelihood values and second localization likelihood values for the internal organ of interest and the neighboring organ, respectively. The surgical assistive device determines extent of anatomical adjacency boundary between the internal organ of interest and the neighboring organ, based on the first localization likelihood values and the second localization likelihood values. The surgical assistive device segments the test video frame into a plurality of regions and generates a first region based on integration of a first set of regions from the plurality of regions. The surgical assistive device further localizes the internal organ of interest based on first region, in accordance with the extent of anatomical adjacency boundary.

Abstract: Various aspects of a surgical assistive device and method to provide assistance in neurosurgery are disclosed herein. The surgical assistive device includes circuitry that determines initial positional shift in the anatomical portion of a subject caused by an incision into a protective portion. A first depth value is estimated based on the plurality of stereo images that are captured from different specified positions of the surgical image capture device to capture views of the anatomical surface of the anatomical portion through the incision. A residual shift in a position of a surgical region of interest is determined with respect to the estimated first depth value. Surgical guidance information is generated to indicate a correct depth of the surgical region of interest in a surgical cavity, based on the determined residual shift. Display of the generated surgical guidance information is controlled on a display screen coupled to the surgical assistive device.

Abstract: Various aspects of an image-processing system and method to reconstruct a three-dimensional (3D) anatomical surface of an anatomical portion are disclosed herein. The system includes an image-processing device configured to receive a plurality of stereo images of the anatomical portion. A first set of key points with a point density greater than a threshold value is identified. A second set of key points is determined based on filtration of one or more outlier key points from the identified first set of key points. A 3D anatomical surface of the anatomical portion is reconstructed based on disparity of one or more matched key points in the determined second set of key points and a smoothing operation performed on the disparity.

Abstract: Various aspects of a system and a method to process multimodal images are disclosed herein. In accordance with an embodiment, the system includes an image-processing device that generates a structured point cloud, which represents edge points of an anatomical portion. The structured point cloud is generated based on shrink-wrapping of an unstructured point cloud to a boundary of the anatomical portion. Diffusion filtering is performed to dilate edge points that correspond to the structured point cloud to mutually connect the edge points on the structured point cloud. A mask is created for the anatomical portion based on the diffusion filtering.

Abstract: Various aspects of an image-processing system and method to reconstruct a three-dimensional (3D) anatomical surface of an anatomical portion are disclosed herein. The system includes an image-processing device configured to receive a plurality of stereo images of the anatomical portion. A first set of key points with a point density greater than a threshold value is identified. A second set of key points is determined based on filtration of one or more outlier key points from the identified first set of key points. A 3D anatomical surface of the anatomical portion is reconstructed based on disparity of one or more matched key points in the determined second set of key points and a smoothing operation performed on the disparity.

Abstract: Various aspects of a system and a method to provide assistance in a surgery in presence of tissue deformation are disclosed herein. In accordance with an embodiment, the system includes an electronic device that receives one or more tissue material properties of a plurality of surface structures of an anatomical portion. One or more boundary conditions associated with the anatomical portion may also be received. Surface displacement of the anatomical portion may be determined by matching a first surface of the anatomical portion before deformation with a corresponding second surface of the anatomical portion after the deformation. The volume displacement field of the anatomical portion may be computed based on the determined surface displacement, the received one or more tissue material properties, and the received one or more boundary conditions.

Abstract: Various aspects of a system and a method to process multimodal images are disclosed herein. In accordance with an embodiment, the system includes an image-processing device that generates a structured point cloud, which represents edge points of an anatomical portion. The structured point cloud is generated based on shrink-wrapping of an unstructured point cloud to a boundary of the anatomical portion. Diffusion filtering is performed to dilate edge points that correspond to the structured point cloud to mutually connect the edge points on the structured point cloud. A mask is created for the anatomical portion based on the diffusion filtering.

Abstract: Various aspects of a method and a system for video data processing are disclosed herein. The method includes receiving video data of a human subject. Each of a sequence of frames of the received video data is associated with a plurality of spectral components. A first peak and a second peak from a plurality of peaks corresponding to the plurality of spectral components are determined. Amplitudes of the determined first peak and the second peak exceed a first threshold value. A peak separation between the determined first peak and the determined second peak is determined. The determined peak separation is within a predetermined range. Based on the determined peak separation, a heart rate of the human subject is determined.

Abstract: An image registration system, and a method of operation of an image registration system thereof, including: an imaging unit for obtaining a pre-operation non-invasive imaging volume and for obtaining an intra-operation non-invasive imaging volume; and a processing unit including: a rigid registration module for generating a rigid registered volume based on the pre-operation non-invasive imaging volume, a region of interest module for isolating a region of interest from the intra-operation non-invasive imaging volume, a point generation module, coupled to the region of interest module, for determining feature points of the region of interest, an optimization module, coupled to the point generation module, for matching the feature points with corresponding points of the rigid registered volume for generating a matched point cloud, and an interpolation module, coupled to the optimization module, for generating a non-rigid registered volume based on the matched point cloud for display on a display interface.

Abstract: A gesture recognition system using a skin-color based method combined with motion information to achieve real-time segmentation. A Kalman filter is used to track the centroid of the hand. The palm center, palm bottom, as well as the largest distance from the palm center to the contour from extracted hand mask are computed. The computed distance to a threshold is then compared to decide if the current posture is “open” or “closed.” In a preferred embodiment, the transition between the “open” and “closed” posture to decide if the current gesture is in “select” or “grab” state.

Abstract: A tool localization system and method of operation thereof including: a camera for obtaining an image frame; and a processing unit connected to the camera, the processing unit including: a classification module for detecting a surgical tool in the image frame, a motion vector module, coupled to the classification module, for modeling motion of the surgical tool based on the image frame and at least one prior image frame, a mask generation module, coupled to the motion vector module, for generating a tool mask, based on the surgical tool detected and the motion of the surgical tool, for covering the surgical tool in the image frame, and an exposure module, coupled to the mask generation module, for processing the image frame without the areas covered by the tool mask for display on a display interface.

Abstract: A system for performing an image categorization procedure includes an image manager with a keypoint generator, a support region filter, an orientation filter, and a matching module. The keypoint generator computes initial descriptors for keypoints in a test image. The support region filter and the orientation filter perform respective filtering procedures upon the initial descriptors to produce filtered descriptors. The matching module compares the filtered descriptors to one or more database image sets for categorizing said test image. A processor of an electronic device typically controls the image manager to effectively perform the image categorization procedure.

Abstract: Certain aspects of an apparatus and method for method and apparatus for tissue region identification may include segmenting the image into a plurality of regions, filtering out regions in the plurality of regions which are curvilinear, and isolating a target area where the tissue sample is identified as the plurality of regions not filtered.

Abstract: A system and method for effectively performing an integrated segmentation procedure comprises an image segmenter that includes a texture modeler, a contrast modeler, and a model integrator. The texture modeler creates a texture model based upon an original image. Similarly, the contrast modeler creates a contrast model based upon the original image. The model integrator then performs a model integration procedure to create a final segmented image by integrating the texture model and the contrast model according to a calculated texture model metric. A processor of an electronic device typically controls the image segmenter to perform the integrated segmentation procedure.