Abstract: In a method of using a structured-light based system, real-time 2D images of a portion of a field of view are captured using an endoscope. A portion of an object in the field of view is illuminated with a structured light pattern, and light reflected from the field of view is detected. From the reflected light, a 3D image of the field of view is constructed, and 3D locations of points on a surface of the object are determined. The real time 3D spatial position of the endoscope and/or a surgical tool is determined. If a distance between the surface the endoscope and/or surgical tool, as determined using the 3D spatial position, falls below a predetermined distance, an alert is generated to notify a user.

Abstract: A method and system for controlling at least one robotically controlled surgical tool via vocal activation include detecting a vocal command generated by a surgeon, converting said at least one surgeon in said surgical setting via said voice sensor. The vocal command is converted to operative instructions associated with a robotically controlled surgical tool to generate instructions according to a predetermined set of rules including at least one of a no fly zone rule and collision prevention rule.

Abstract: An image of a field of view is captured by the endoscope and a portion of the image is displayed on a display. A popup image is generated and displayed upon image analysis identification of an occurrence of bleeding or motion of edges of an incision that is within the field of view, but outside the portion of the field of view displayed on the display.

Abstract: A proximity detection system allows monitoring of proximity between the end effectors of first and second independent robotic manipulators. Imagers are circumferentially positioned around the end effector of at least one of the robotic manipulators. Image data from the imagers is analyzed to determine proximity between the end effectors. When determined proximity falls below a defined threshold, the system issues an alert to the user or slows/suspends manipulator motion.

Abstract: A system comprises an endoscope with a wide-angle lens and a camera. An image of a field of view is captured by the endoscope and a portion of the image is displayed on the display. A popup image is generated and displayed upon image analysis identification of an occurrence of bleeding or motion of edges of an incision that is within the field of view, but outside the portion of the field of view displayed on the display.

Abstract: Calibration of parameters of an endoscopic or laparoscopic camera is conducted while the camera is in use capturing images of a surgical procedure at a surgical site. The surgical procedure is performed using a surgical instrument marked with a fiducial pattern. A processor receives image data from images captured by the camera and uses machine vision to detect points of the pattern, and carries out an optimization to determine the 3D pose of the surgical instrument relative to the camera and the camera parameters. Once calibration parameters are determined, 3D measurements between measurement points in the body cavity may be estimated.

Abstract: An image of a field of view is captured by the endoscope and a portion of the image is displayed on a display. A popup image is generated and displayed upon image analysis identification of an occurrence of bleeding or motion of edges of an incision that is within the field of view, but outside the portion of the field of view displayed on the display.

Abstract: A method of calibrating a camera used for robotic surgery makes use of a target pattern positioned on a robotic manipulator. The camera to be calibrated is moved by the robotic manipulator to move the camera to a plurality of positions and orientations, ideally in closed loop fashion to maintain the target pattern within the camera's image plane. The camera captures images of the target pattern while in the plurality of positions and orientations. A processor receives the captured images and analyzes them to determine calibration parameters for the camera.

Abstract: In a system and method of calibrating a stereoscopic camera, a target pattern is electronically displayed on a display. The camera to be calibrated captures images of the target pattern using a stereoscopic camera while the displayed target pattern is altered over time. The captured images are analyzed and calibration parameters for the stereoscopic camera are determined.

Abstract: A system and method apply computer vision to images captured of a surgical site in order to measure the distance between surgical instruments at the surgical site. Images of the site are captured and analyzed to estimate a width in pixels of a first surgical instrument positioned at the treatment site. The depth coordinate Z of the tip of the first surgical instrument is determined by comparing the estimated tool width to the known physical width of the first surgical instrument, and the corresponding X and Y coordinates are estimated based on the Z coordinate. The coordinates of the tip of a second instrument are determined in the same manner, and the distance between the tips is then calculated, and the distance value is displayed as an overlay on the image display.

Abstract: An intelligent surgical tool control system, comprising a tool management system; an indicating means to indicate at least one surgical event; a communicable database for storing, for each item of interest, its identity, its present 3D position and at least one previous 3D position; and at least one processor to identify, from a surgical event, an output surgical procedure. The tool management system can comprise a maneuvering mechanism to maneuver a surgical tool in at least two dimensions; and a controller to control at least one of activation and deactivation of a surgical tool and articulation of a surgical tool. The indicating means can indicate a surgical event selected from movement of a moving element and presence of an item of interest, where movement is determinable if the current 3D position of the moving element is substantially different from a previous 3D position of the same.

Abstract: A method and system for controlling at least one robotically controlled surgical tool via vocal activation include detecting a vocal command generated by a surgeon, converting said at least one surgeon in said surgical setting via said voice sensor. The vocal command is converted to operative instructions associated with a robotically controlled surgical tool to generate instructions according to a predetermined set of rules including at least one of a no fly zone rule and collision prevention rule.

Abstract: A surgical controlling system, comprising: at least one surgical tool configured to be inserted into a surgical environment of a human body; at least one location estimating means configured for real-time localization of the 3D spatial position of said at least one surgical tool at any given time t; at least one movement detection means communicable with a movement's database and with said location estimating means; a controller having a processing means communicable with a controller's database; said controller's database is in communication with said movement detection means; and at least one display configured to real time provide an image of at least a portion of said surgical environment; wherein said controller is configured to direct said surgical tool to said location via said instructions provided by said controller; further wherein said location is real time updated on said display as said at least one surgical tool is moved.

Abstract: The following invention is a vocally activated control system for controlling an apparatus in a surgical setting, the system comprises: a. a voice sensor configured to detect vocal commands generated by surgeons during surgery; b. a signal transmitter connected to the voice sensor, the transmitter is configured to convert a vocal command into a transmittable signal and transmit it; c. a processor connected to a signal transmitter configured to receive a transmittable vocal signal, the processor is configured to convert a vocal signal to a predetermined set of operative instructions associated with the apparatus, the predetermined set of operative instructions comprising at least one instruction; and d. control means connected to the processor and apparatus; the control means is configured to receive a predetermined set of operative instructions and to cause the apparatus to operate accordingly; Said voice sensor and said transmitter are integrated within a wearable element.

Abstract: A surgical controlling system for controlling the 3D spatial position of at least one articulating surgical tool includes a controller configured to provide instructions to control movement of the surgical tool. The controller comprises a processor to determine a location of the surgical tool using at least one location estimating feature and to determine movement of the surgical tool using a database in communication with at least one movement detection feature. The location estimating feature is configured to real-time locate the 3D spatial position of the surgical tool at any given time t, and the movement detection feature is configured to detect movement of the surgical tool.

Abstract: In a method of using a structured-light based system, real-time 2D images of a portion of a field of view are captured using an endoscope. A portion of an object in the field of view is illuminated with a structured light pattern, and light reflected from the field of view is detected. From the reflected light, a 3D image of the field of view is constructed, and 3D locations of points on a surface of the object are determined. The real time 3D spatial position of the endoscope and/or a surgical tool is determined. If a distance between the surface the endoscope and/or surgical tool, as determined using the 3D spatial position, falls below a predetermined distance, an alert is generated to notify a user.

Abstract: A system and method allow creation of surgical annotations during surgery. A digital image of a treatment site in a region of patient anatomy is captured using a camera and is displayed on an image display. The system displays an overlay marking an anatomic region of interest in the displayed image. During the course of the surgery, panning of the image results in the anatomic region of interest being outside the displayed field of view. In response to user input, the displayed image is subsequently panned such that the anatomic region of interest and the displayed overlay return to the displayed field of view.

Abstract: Surgical assistance is provided in the form of an image of the surgical site that has overlays marking anatomical structures of interest to the surgeon. The system uses computer vision to detect the anatomical structures of interest that are visible to the camera, and predicts the location, shape and/or orientation of portions of the anatomical structures that are not visible to the camera. The overlays mark both the visible portions of the anatomical structures and the predicted location, shape and orientation of the invisible portions.

Abstract: Calibration of parameters of an endoscopic or laparoscopic camera is conducted while the camera is in use capturing images of a surgical procedure at a surgical site. The surgical procedure is performed using a surgical instrument marked with a fiducial pattern. A processor receives image data from images captured by the camera and uses machine vision to detect points of the pattern, and carries out an optimization to determine the 3D pose of the surgical instrument relative to the camera and the camera parameters.

Abstract: A system and method for measuring an area of interest such as a hernia defect within a body cavity, in which real time image data is captured at a treatment site that includes the area of interest. Computer vision is applied to identify the extents of the area of interest within images captured using the camera, and dimensions of the area of interest are measured using the image data. The user is given output, which may include recommendations of hernia mesh shape/size and or positioning, based on the measured dimensions.