Abstract: The systems and methods of the present disclosure are used for guiding a medical instrument towards a target, the method positioning a medical instrument at a first location within a patient anatomy, wherein the medical instrument comprises at least one sensor, determining a first biomarker measurement using the at least one sensor, determining a second biomarker measurement using the at least one sensor, comparing the first biomarker measurement with the second biomarker measurement to determine a proximity to the target to provide a first comparison, and providing guidance for moving the medical instrument based on results of the first comparison.

Abstract: A teleoperational medical system includes a teleoperational control system, a teleoperational manipulator, an operator controller. The control system includes a processing unit configured to: determine a position and orientation of a marker on an operator of the operator controller; determine whether a head portion of the operator is directed toward or away from a display region of a display; based on a determination that the head portion is directed toward the display region, initiate an operator following mode in which a movement of the operator controller provides a corresponding movement to the teleoperational manipulator; and based on a determination that the head portion is directed away, suspend the operator following mode. The determination that the head portion is directed away from the display region of the display device includes determining whether the head portion of the operator is directed away from the display region for a threshold time period.

Abstract: A stereo imaging system comprises a stereoscopic camera having left and right image capturing elements for capturing stereo images; a stereo viewer; and a processor configured to modify the stereo images prior to being displayed on the stereo viewer so that a disparity between corresponding points of the stereo images is adjusted as a function of a depth value within a region of interest in the stereo images after the depth value reaches a target depth value.

Abstract: A coordination system for coordinating remote assistance of a medical system involves a support agent executed by a computer processor. The support agent is configured to receive a first support request submitted by a user of the medical system, and identify a support person to respond to the first support request from a multitude of support persons by applying a support person identification logic to a system-internal context. The system-internal context is obtained from a system awareness engine configured to detect information about the medical system. The support agent is further configured to facilitate contact of the support person.

Abstract: A teleoperated system comprises a display, a master input device, and a control system. The control system is configured to determine an orientation of an end effector reference frame relative to a field of view reference frame, determine an orientation of a master input device reference frame relative to a display reference frame, establish an alignment relationship between the master input device reference frame and the display reference frame, and command, based on the alignment relationship, a change in a pose of the end effector in response to a change in a pose of the master input device. The alignment relationship is independent of a position relationship between the master input device reference frame and the display reference frame. In one aspect, the teleoperated system is a telemedical system such as a telesurgical system.

Abstract: Scene perception systems and methods are described herein. In certain illustrative examples, a system combines data sets associated with imaging devices included in a dynamic multi-device architecture and uses the combined data sets to perceive a scene (e.g., a surgical scene) imaged by the imaging devices. To illustrate, the system may access tracking data for imaging devices capturing images of a scene and fuse, based on the tracking data, data sets respectively associated with the imaging devices to generate fused sets of data for the scene. The tracking data may represent a change in a pose of at least one of the image devices that occurs while the imaging devices capture images of the scene. The fused sets of data may represent or be used to generate perceptions of the scene. In certain illustrative examples, scene perception is dynamically optimized using a feedback control loop.

Abstract: A system may comprise an image capture device to capture an image of a work site. The system may also comprise a processor to determine whether a tool disposed at the work site is energized and determine a first area of the captured image of the work site, including or adjacent to an image of a portion of the tool in the captured image. The processor may also determine a second area of the captured image including a remainder of the captured image that does not include the first area. Conditioned upon determining that the tool is energized, at least one of the first area and the second area of the captured image of the work site may be processed to indicate that the tool in the first area is being energized. The image of the work site with the first area and the second area may be displayed.

Abstract: A system comprises a processor and a memory having computer readable instructions stored thereon. The computer readable instructions, when executed by the processor, cause the system to display a surgical environment image. The surgical environment image includes a virtual control element for controlling a component of a surgical system. The virtual control element includes a real-time image of the component of the surgical system in the surgical environment image. The computer readable instructions also cause the system to display an image of a body part of a user used to interact with the virtual control element, receive a gesture of the body part of the user in a predetermined motion via a gesture based input device, and adjust a setting of the component of the surgical system based on the received gesture.

Abstract: A bifurcated navigation control system defines a path whereby a manipulator cart is to navigate from an initial location to a target location. The system directs the manipulator cart to navigate, in a first bifurcated navigation control mode associated with a primary control interface that facilitates both operator control of steering and propulsion, along at least part of a first portion of the path from the initial to an intermediate location. The system further directs the manipulator cart to navigate, in a second bifurcated navigation control mode associated with a secondary control interface that facilitates operator control of propulsion but not steering, along at least part of a second portion of the path from the intermediate to the target location. In the first and second bifurcated navigation control modes, the system autonomously controls the steering of the manipulator cart while allowing operator control of the propulsion of the manipulator cart.

Abstract: A robotic system includes a processor that is programmed to determine and cause work site measurements for user specified points in the work site to be graphically displayed in order to provide geometrically appropriate tool selection assistance to the user. The processor is also programmed to determine an optimal one of a plurality of tools of varying geometries for use at the work site and to cause graphical representations of at least the optimal tool to be displayed along with the work site measurements.

Abstract: A system may comprise an image capture device disposed to capture an image of a work site, a display, and a processor. The processor may be configured to determine whether a tool disposed at the work site is energized and determine an area of the captured image of the work site, including or adjacent to an image of a portion of the tool in the captured image, to be filtered to indicate the tool has been energized. Conditioned upon determining that the tool is energized, the processer may also filter the area of the captured image of the work site to generate a filtered area of the captured image of the work site so as to create a glowing effect on the image of the portion of the energized tool, indicating that the tool is being energized. The image of the work site with the filtered area may be displayed on the display.

Abstract: A bifurcated navigation control system defines a path whereby a manipulator cart is to navigate from an initial location to a target location, and identifies a navigation condition associated with a navigation of the manipulator cart along the path. Based on the navigation condition, the bifurcated navigation control system defines a propulsion limitation for the manipulator cart during the navigation of the manipulator cart along the path. The bifurcated navigation control system directs the manipulator cart to navigate along at least part of the path in a bifurcated navigation control mode in which the bifurcated navigation control system is configured to autonomously control a steering of the manipulator cart while allowing operator control of a propulsion of the manipulator cart in accordance with the propulsion limitation. Corresponding methods and systems are also disclosed.

Abstract: A method comprises displaying a surgical environment image. The surgical environment image includes a virtual control element for controlling a component of a surgical system, and the virtual control element includes a real-time image of the component of the surgical system in the surgical environment image. The method further comprises displaying an image of a body part of a user. The body part is used to interact with the virtual control element. The method further comprises receiving a gesture of the body part of the user in a predetermined motion, via a gesture based input device registering movement of the body part of the user, while the body part interacts with the virtual control element. The method further comprises adjusting a setting of the component of the surgical system based on the received gesture.

Abstract: Techniques for virtually recreating devices include a replay service configured to access a log associated with a first device, extract a first log entry from the log, recreate a first recreated service request for the one or more shared services based on information associated with the first log entry, and send the first recreated service request to a shared service to recreate an event associated with the first log entry; one or more shared services configured to receive and respond to a second service request from a second device coupled to the logging and replay device; and a recreation engine configured to receive the first recreated service request, a response to the first recreated service request, the second service request, and a response to the second service request and create a virtual recreation of the first device.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a second joint, and a first link. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data, and generate a second joint state estimate of the second joint. The processing unit is further configured to apply a first weight to the first joint state estimate to generate a first weighted joint state estimate, apply a second weight to the second joint state estimate to generate a second weighted joint state estimate, and control the first and second joints based on the first weighted joint state estimate and second weighted joint state estimate.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a first link, and a second link. The second link is between the first joint and the first link in the serial chain. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data and a kinematic model of the robotic manipulator, and control the first joint based on the first joint state estimate.

Abstract: In one embodiment of the invention, a a minimally invasive surgical system is disclosed. The system configured to capture and display camera images of a surgical site on at least one display device at a surgeon console; switch out of a following mode and into a masters-as-mice (MaM) mode; overlay a graphical user interface (GUI) including an interactive graphical object onto the camera images; and render a pointer within the camera images for user interactive control. In the following mode, the input devices of the surgeon console may couple motion into surgical instruments. In the MaM mode, the input devices interact with the GUI and interactive graphical objects. The pointer is manipulated in three dimensions by input devices having at least three degrees of freedom. Interactive graphical objects are related to physical objects in the surgical site or a function thereof and are manipulatable by the input devices.

Abstract: A robotic system includes a processor that is programmed to determine and cause work site measurements for user specified points in the work site to be graphically displayed in order to provide geometrically appropriate tool selection assistance to the user. The processor is also programmed to determine an optimal one of a plurality of tools of varying geometries for use at the work site and to cause graphical representations of at least the optimal tool to be displayed along with the work site measurements.

Abstract: A minimally invasive surgical system is disclosed including a processor coupled to a stereoscopic endoscope and a stereoscopic video display device. The processor generates an operative image of an anatomic structure in the surgical site, overlays the operative image onto the captured stereo video images for display on the stereoscopic video display device, generates and overlays a pointer onto the operative image or the captured stereo video images to display the pointer on the stereoscopic video display device with a three dimensional appearance, and switches between a first mode for input devices of a surgeon console used to couple motion into surgical instruments and a second mode used to control an interactive graphical user interface to allow interactions with the pointer and the operative image in three dimensions using input devices having at least three degrees of freedom.

Abstract: A system and method of aggregating logs for replay includes a processor configured to execute a replay service. The replay service is configured to access a plurality of logs, aggregate the plurality of logs to create a composite log, extract a first log entry from the composite log, recreate a service request based on information associated with the first log entry, and send the service request to a corresponding service to recreate an event associated with the first log entry. In some embodiments, to aggregate the plurality of logs to create the composite log, the replay service is configured to aggregate a first entry from a first log of the plurality of logs with a corresponding related second entry from a second log of the plurality of logs to create a composite entry for the composite log.