Abstract: A system and method fort determining the relative positions of robotic manipulators makes use of a camera positioned in an operating room so as to capture images or portions of first and second robotic manipulators that are positioned outside a patient body cavity. The camera captures images of first and second robotic manipulators, the images are analyzed in real time to determine a relative distance between the first robotic manipulator and the second robotic manipulator. The system determines whether the relative distance is below a predetermined threshold, and, if it is, initiates a collision avoidance measure.

Abstract: A method and system for controlling at least one robotically controlled surgical tool via vocal activation include detecting a vocal command generated by a surgeon, converting said at least one surgeon in said surgical setting via said voice sensor. The vocal command is converted to operative instructions associated with a robotically controlled surgical tool to generate instructions according to a predetermined set of rules including at least one of a no fly zone rule and collision prevention rule.

Abstract: Calibration of parameters of an endoscopic or laparoscopic camera is conducted while the camera is in use capturing images of a surgical procedure at a surgical site. The surgical procedure is performed using a surgical instrument marked with a fiducial pattern. A processor receives image data from images captured by the camera and uses machine vision to detect points of the pattern, and carries out an optimization to determine the 3D pose of the surgical instrument relative to the camera and the camera parameters. Once calibration parameters are determined, 3D measurements between measurement points in the body cavity may be estimated.

Abstract: A method of calibrating a camera used for robotic surgery makes use of a target pattern positioned on a robotic manipulator. The camera to be calibrated is moved by the robotic manipulator to move the camera to a plurality of positions and orientations, ideally in closed loop fashion to maintain the target pattern within the camera's image plane. The camera captures images of the target pattern while in the plurality of positions and orientations. A processor receives the captured images and analyzes them to determine calibration parameters for the camera.

Abstract: A system and method apply computer vision to images captured of a surgical site in order to measure the distance between surgical instruments at the surgical site. Images of the site are captured and analyzed to estimate a width in pixels of a first surgical instrument positioned at the treatment site. The depth coordinate Z of the tip of the first surgical instrument is determined by comparing the estimated tool width to the known physical width of the first surgical instrument, and the corresponding X and Y coordinates are estimated based on the Z coordinate. The coordinates of the tip of a second instrument are determined in the same manner, and the distance between the tips is then calculated, and the distance value is displayed as an overlay on the image display.

Abstract: A method and system for controlling at least one robotically controlled surgical tool via vocal activation include detecting a vocal command generated by a surgeon, converting said at least one surgeon in said surgical setting via said voice sensor. The vocal command is converted to operative instructions associated with a robotically controlled surgical tool to generate instructions according to a predetermined set of rules including at least one of a no fly zone rule and collision prevention rule.

Abstract: A system for determining relative positions of subsystems of a robot-assisted surgical system includes a subsystem component including a plurality of manipulator arms and a surgeon console. Each subsystem component includes at least one of an optical tracker and a light emitter. Image data from the optical trackers is analyzed to determine the relative positions of the subsystems.

Abstract: The following invention is a vocally activated control system for controlling an apparatus in a surgical setting, the system comprises: a. a voice sensor configured to detect vocal commands generated by surgeons during surgery; b. a signal transmitter connected to the voice sensor, the transmitter is configured to convert a vocal command into a transmittable signal and transmit it; c. a processor connected to a signal transmitter configured to receive a transmittable vocal signal, the processor is configured to convert a vocal signal to a predetermined set of operative instructions associated with the apparatus, the predetermined set of operative instructions comprising at least one instruction; and d. control means connected to the processor and apparatus; the control means is configured to receive a predetermined set of operative instructions and to cause the apparatus to operate accordingly; Said voice sensor and said transmitter are integrated within a wearable element.

Abstract: A system and method allow creation of surgical annotations during surgery. A digital image of a treatment site in a region of patient anatomy is captured using a camera and is displayed on an image display. The system displays an overlay marking an anatomic region of interest in the displayed image. During the course of the surgery, panning of the image results in the anatomic region of interest being outside the displayed field of view. In response to user input, the displayed image is subsequently panned such that the anatomic region of interest and the displayed overlay return to the displayed field of view.

Abstract: In a system and method for determining guide points or a guide path for display on an endoscopic display, image data corresponding to a surgical treatment site is captured using a camera. Using the image data, the positions of one or more reference points within the surgical environment, are determined. Based on the positions of the reference points, the positions of guide points spaced from the reference point are estimated or determined, in some cases using predetermined offsets. The guide points or guide path is displayed as an overlay of the image data on an image display. In an embodiment using the system for a sleeve gastrectomy procedure, the reference points are input by a user or determined by the system with reference to a bougie that has been positioned within a stomach at the operative site, and the guide path is used as a guide for stapling and resection to form the sleeve.

Abstract: A system for capturing records of a surgical procedure includes a camera that captures real time video images of a surgical treatment site. The system detects completion of a predetermined step of a surgical procedure by analyzing the image data using computer vision or analyzing kinematic information from a robotic component. Upon detecting completion of the predetermined step, the system stores a frame from the image data in a surgical record for the procedure.

Abstract: Calibration of parameters of an endoscopic or laparoscopic camera is conducted while the camera is in use capturing images of a surgical procedure at a surgical site. The surgical procedure is performed using a surgical instrument marked with a fiducial pattern. A processor receives image data from images captured by the camera and uses machine vision to detect points of the pattern, and carries out an optimization to determine the 3D pose of the surgical instrument relative to the camera and the camera parameters.

Abstract: Surgical assistance is provided in the form of an image of the surgical site that has overlays marking anatomical structures of interest to the surgeon. The system uses computer vision to detect the anatomical structures of interest that are visible to the camera, and predicts the location, shape and/or orientation of portions of the anatomical structures that are not visible to the camera. The overlays mark both the visible portions of the anatomical structures and the predicted location, shape and orientation of the invisible portions.

Abstract: Intrinsic and extrinsic calibration parameters are determined in real time for a camera positioned to capture images of a surgical site in a body cavity. The camera is positioned on a manipulator arm and used to capture a plurality of frames of images of the surgical site using the at least one camera while moving the camera within the body cavity. 3D position information corresponding to positions of the camera is recorded during capture of said images. A plurality of features between two or more frames of the captured images are matched, and a 3D structure of the plurality of features is reconstructed using multi-frame triangulation. A penalty measure is estimated using a reprojection error. The distance in the image plane between the projected 3D feature and the measurement. Intrinsic calibration parameters are estimated for the at least one camera, and refined to minimize the penalty measure.

Abstract: A system for measuring distances within a surgical site includes a camera positionable to capture 3D image data corresponding to a treatment site. Using the image data, the system determines the relative 3D positions of identified measurement points at the treatment site captured in the images, and it estimates or determines the distance between the measurement points. Output is generated communicating the measured distances to the user. The measurement taken follows the 3D topography of the tissue positioned between the measurement points.

Abstract: The following invention is a vocally activated control system for controlling an apparatus in a surgical setting, the system comprises: a. a voice sensor configured to detect vocal commands generated by surgeons during surgery; b. a signal transmitter connected to the voice sensor, the transmitter is configured to convert a vocal command into a transmittable signal and transmit it; c. a processor connected to a signal transmitter configured to receive a transmittable vocal signal, the processor is configured to convert a vocal signal to a predetermined set of operative instructions associated with the apparatus, the predetermined set of operative instructions comprising at least one instruction; and d. control means connected to the processor and apparatus; the control means is configured to receive a predetermined set of operative instructions and to cause the apparatus to operate accordingly; Said voice sensor and said transmitter are integrated within a wearable element.

Abstract: The following invention is a vocally activated control system for controlling an apparatus in a surgical setting, the system comprises: a. a voice sensor configured to detect vocal commands generated by surgeons during surgery; b. a signal transmitter connected to the voice sensor, the transmitter is configured to convert a vocal command into a transmittable signal and transmit it; c. a processor connected to a signal transmitter configured to receive a transmittable vocal signal, the processor is configured to convert a vocal signal to a predetermined set of operative instructions associated with the apparatus, the predetermined set of operative instructions comprising at least one instruction; and d. control means connected to the processor and apparatus; the control means is configured to receive a predetermined set of operative instructions and to cause the apparatus to operate accordingly; Said voice sensor and said transmitter are integrated within a wearable element.

Abstract: The following invention is a vocally activated control system for controlling an apparatus in a surgical setting, the system comprises: a. a voice sensor configured to detect vocal commands generated by surgeons during surgery; b. a signal transmitter connected to the voice sensor, the transmitter is configured to convert a vocal command into a transmittable signal and transmit it; c. a processor connected to a signal transmitter configured to receive a transmittable vocal signal, the processor is configured to convert a vocal signal to a predetermined set of operative instructions associated with the apparatus, the predetermined set of operative instructions comprising at least one instruction; and d. control means connected to the processor and apparatus; the control means is configured to receive a predetermined set of operative instructions and to cause the apparatus to operate accordingly; Said voice sensor and said transmitter are integrated within a wearable element.