Abstract: Systems, methods, and devices for registering a 3D image of a patient with magnetic coordinates are disclosed. A tri-axial sensor (TAS) may be added to the 3D camera. The location and orientation of the TAS sensor may be determined based on the known magnetic fields that are applied by a magnetic field transmitter. The camera coordinate system may then be transferred to the magnetic coordinate system. After completing the registration of the 3D image with a CT image and the 3D image with the magnetic coordinates, the CT image may then be registered with the magnetic coordinates.

Abstract: Physicians performing invasive procedures utilize instruments inserted into a human body to perform the procedures. Such procedures typically involve actions to be performed on specific targeted anatomical structures. During the procedure, nearby anatomical structures unrelated to the procedure should generally be avoided. A system and techniques are provided herein for monitoring the position of such unrelated nearby anatomical structures relative to one or more surgical instruments. The system emits a warning to a human operator such as a surgeon if one of the instruments is too close to a monitored anatomical structure.

Abstract: Physicians performing invasive procedures require accuracy and precision when working with surgical tools. Surgical procedures are increasingly becoming minimally invasive, with physicians operating using cameras to view the surgery site and directing their tools through oculars or video displays. Ideally, the physician should be able to perform the invasive procedure while simultaneously observing both the real-time image of the patient and additional data critical for his medical decisions about the manipulation of the surgical tool and the next surgical step. The augmented reality navigation system of the present disclosure provides tool location visibility for invasive procedures through the use of location sensors included on a camera and/or on the tools used during a procedure. A location tracking system determines and monitors the locations of the tools and camera based on the characteristics of signals detected by the sensors and displays informational overlays on images obtained with a camera.

Abstract: A method and system for automatic landmark registration and registration using trajectory information and shape sensing during an endoscopic procedure, such as bronchoscopy, are described herein. A segmentation centerline of airways of a lung may be generated based on a pre-operative computed tomography (CT) image of the lung. Landmarks may be automatically detected on the segmentation centerline corresponding to bifurcations in the airways of the lung. A location data point cloud of locations of a catheter through the airways of the lung during navigation may be generated. A bounding volume of the airways of the lung may be generated and a bounding volume centerline may be detected. Landmarks may be detected on the bounding volume centerline for the same bifurcations. Then, the two sets of landmarks may be mapped as part of registration. The trajectory information with shape sensing may be used to provide non-rigid or fine registration.

Abstract: A method and system for automatic landmark registration and registration using trajectory information and shape sensing during an endoscopic procedure, such as bronchoscopy, are described herein. A segmentation centerline of airways of a lung may be generated based on a pre-operative computed tomography (CT) image of the lung. Landmarks may be automatically detected on the segmentation centerline corresponding to bifurcations in the airways of the lung. A location data point cloud of locations of a catheter through the airways of the lung during navigation may be generated. A bounding volume of the airways of the lung may be generated and a bounding volume centerline may be detected. Landmarks may be detected on the bounding volume centerline for the same bifurcations. Then, the two sets of landmarks may be mapped as part of registration. The trajectory information with shape sensing may be used to provide non-rigid or fine registration.

Abstract: Physicians performing invasive procedures require accuracy and precision when working with surgical tools. Surgical procedures are increasingly becoming minimally invasive, with physicians operating using cameras to view the surgery site and directing their tools through oculars or video displays. Ideally, the physician should be able to perform the invasive procedure while simultaneously observing both the real-time image of the patient and additional data critical for his medical decisions about the manipulation of the surgical tool and the next surgical step. The augmented reality navigation system of the present disclosure provides tool location visibility for invasive procedures through the use of location sensors included on a camera and/or on the tools used during a procedure. A location tracking system determines and monitors the locations of the tools and camera based on the characteristics of signals detected by the sensors and displays informational overlays on images obtained with a camera.

Abstract: Physicians performing invasive procedures utilize instruments inserted into a human body to perform the procedures. Such procedures typically involve actions to be performed on specific targeted anatomical structures. During the procedure, nearby anatomical structures unrelated to the procedure should generally be avoided. A system and techniques are provided herein for monitoring the position of such unrelated nearby anatomical structures relative to one or more surgical instruments. The system emits a warning to a human operator such as a surgeon if one of the instruments is too close to a monitored anatomical structure.

Abstract: A method and system for automatic landmark registration and registration using trajectory information and shape sensing during an endoscopic procedure, such as bronchoscopy, are described herein. A segmentation centerline of airways of a lung may be generated based on a pre-operative computed tomography (CT) image of the lung. Landmarks may be automatically detected on the segmentation centerline corresponding to bifurcations in the airways of the lung. A location data point cloud of locations of a catheter through the airways of the lung during navigation may be generated. A bounding volume of the airways of the lung may be generated and a bounding volume centerline may be detected. Landmarks may be detected on the bounding volume centerline for the same bifurcations. Then, the two sets of landmarks may be mapped as part of registration. The trajectory information with shape sensing may be used to provide non-rigid or fine registration.