Abstract: A system comprises a processor and a memory having computer readable instructions stored thereon. The computer readable instructions, when executed by the processor, cause the system to display a surgical environment image. The surgical environment image includes a virtual control element for controlling a component of a surgical system. The virtual control element includes a real-time image of the component of the surgical system in the surgical environment image. The computer readable instructions also cause the system to display an image of a body part of a user used to interact with the virtual control element, receive a gesture of the body part of the user in a predetermined motion via a gesture based input device, and adjust a setting of the component of the surgical system based on the received gesture.

Abstract: A bifurcated navigation control system defines a path whereby a manipulator cart is to navigate from an initial location to a target location. The system directs the manipulator cart to navigate, in a first bifurcated navigation control mode associated with a primary control interface that facilitates both operator control of steering and propulsion, along at least part of a first portion of the path from the initial to an intermediate location. The system further directs the manipulator cart to navigate, in a second bifurcated navigation control mode associated with a secondary control interface that facilitates operator control of propulsion but not steering, along at least part of a second portion of the path from the intermediate to the target location. In the first and second bifurcated navigation control modes, the system autonomously controls the steering of the manipulator cart while allowing operator control of the propulsion of the manipulator cart.

Abstract: A robotic system includes a processor that is programmed to determine and cause work site measurements for user specified points in the work site to be graphically displayed in order to provide geometrically appropriate tool selection assistance to the user. The processor is also programmed to determine an optimal one of a plurality of tools of varying geometries for use at the work site and to cause graphical representations of at least the optimal tool to be displayed along with the work site measurements.

Abstract: A system may comprise an image capture device disposed to capture an image of a work site, a display, and a processor. The processor may be configured to determine whether a tool disposed at the work site is energized and determine an area of the captured image of the work site, including or adjacent to an image of a portion of the tool in the captured image, to be filtered to indicate the tool has been energized. Conditioned upon determining that the tool is energized, the processer may also filter the area of the captured image of the work site to generate a filtered area of the captured image of the work site so as to create a glowing effect on the image of the portion of the energized tool, indicating that the tool is being energized. The image of the work site with the filtered area may be displayed on the display.

Abstract: A bifurcated navigation control system defines a path whereby a manipulator cart is to navigate from an initial location to a target location, and identifies a navigation condition associated with a navigation of the manipulator cart along the path. Based on the navigation condition, the bifurcated navigation control system defines a propulsion limitation for the manipulator cart during the navigation of the manipulator cart along the path. The bifurcated navigation control system directs the manipulator cart to navigate along at least part of the path in a bifurcated navigation control mode in which the bifurcated navigation control system is configured to autonomously control a steering of the manipulator cart while allowing operator control of a propulsion of the manipulator cart in accordance with the propulsion limitation. Corresponding methods and systems are also disclosed.

Abstract: A method comprises displaying a surgical environment image. The surgical environment image includes a virtual control element for controlling a component of a surgical system, and the virtual control element includes a real-time image of the component of the surgical system in the surgical environment image. The method further comprises displaying an image of a body part of a user. The body part is used to interact with the virtual control element. The method further comprises receiving a gesture of the body part of the user in a predetermined motion, via a gesture based input device registering movement of the body part of the user, while the body part interacts with the virtual control element. The method further comprises adjusting a setting of the component of the surgical system based on the received gesture.

Abstract: Techniques for virtually recreating devices include a replay service configured to access a log associated with a first device, extract a first log entry from the log, recreate a first recreated service request for the one or more shared services based on information associated with the first log entry, and send the first recreated service request to a shared service to recreate an event associated with the first log entry; one or more shared services configured to receive and respond to a second service request from a second device coupled to the logging and replay device; and a recreation engine configured to receive the first recreated service request, a response to the first recreated service request, the second service request, and a response to the second service request and create a virtual recreation of the first device.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a second joint, and a first link. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data, and generate a second joint state estimate of the second joint. The processing unit is further configured to apply a first weight to the first joint state estimate to generate a first weighted joint state estimate, apply a second weight to the second joint state estimate to generate a second weighted joint state estimate, and control the first and second joints based on the first weighted joint state estimate and second weighted joint state estimate.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a first link, and a second link. The second link is between the first joint and the first link in the serial chain. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data and a kinematic model of the robotic manipulator, and control the first joint based on the first joint state estimate.

Abstract: In one embodiment of the invention, a a minimally invasive surgical system is disclosed. The system configured to capture and display camera images of a surgical site on at least one display device at a surgeon console; switch out of a following mode and into a masters-as-mice (MaM) mode; overlay a graphical user interface (GUI) including an interactive graphical object onto the camera images; and render a pointer within the camera images for user interactive control. In the following mode, the input devices of the surgeon console may couple motion into surgical instruments. In the MaM mode, the input devices interact with the GUI and interactive graphical objects. The pointer is manipulated in three dimensions by input devices having at least three degrees of freedom. Interactive graphical objects are related to physical objects in the surgical site or a function thereof and are manipulatable by the input devices.

Abstract: A robotic system includes a processor that is programmed to determine and cause work site measurements for user specified points in the work site to be graphically displayed in order to provide geometrically appropriate tool selection assistance to the user. The processor is also programmed to determine an optimal one of a plurality of tools of varying geometries for use at the work site and to cause graphical representations of at least the optimal tool to be displayed along with the work site measurements.

Abstract: A minimally invasive surgical system is disclosed including a processor coupled to a stereoscopic endoscope and a stereoscopic video display device. The processor generates an operative image of an anatomic structure in the surgical site, overlays the operative image onto the captured stereo video images for display on the stereoscopic video display device, generates and overlays a pointer onto the operative image or the captured stereo video images to display the pointer on the stereoscopic video display device with a three dimensional appearance, and switches between a first mode for input devices of a surgeon console used to couple motion into surgical instruments and a second mode used to control an interactive graphical user interface to allow interactions with the pointer and the operative image in three dimensions using input devices having at least three degrees of freedom.

Abstract: A system and method of aggregating logs for replay includes a processor configured to execute a replay service. The replay service is configured to access a plurality of logs, aggregate the plurality of logs to create a composite log, extract a first log entry from the composite log, recreate a service request based on information associated with the first log entry, and send the service request to a corresponding service to recreate an event associated with the first log entry. In some embodiments, to aggregate the plurality of logs to create the composite log, the replay service is configured to aggregate a first entry from a first log of the plurality of logs with a corresponding related second entry from a second log of the plurality of logs to create a composite entry for the composite log.

Abstract: A system and method of coordinated motion among heterogeneous devices includes a medical device with one or more movable elements and one or more processors. Any of the processors uses a shared interface to access services. The medical device requests a movement token from a token service through the shared interface, receive the movement token from the token service, exchange configuration data, kinematic data, or planned motion data through the shared interface, plan a first motion for a first movable element of the movable elements based on the movement token and the configuration data, the kinematic data, or the planned motion data, and execute the first motion. In some embodiments, the movement token is selected from a group consisting of an exclusive-motion token, a master follow-me token, a slave follow-me token, a master collision-avoidance token, a slave collision-avoidance token, and a passive collision-avoidance token.

Abstract: A method is performed by a computing system. The method includes receiving image data from an imaging device, and determining, using the image data, a plurality of image-space tools, each image-space tool associated with a tool of a plurality of tools, each tool controlled by a manipulator of a plurality of manipulators. The method further includes determining a first correspondence between a first image-space tool of the plurality of image-space tools and a first tool of the plurality of tools based on a first disambiguation setting associated with the first tool.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a first link, and a second link. The second link is between the first joint and the first link in the serial chain. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data and a kinematic model of the robotic manipulator, and control the first joint based on the first joint state estimate.

Abstract: A mixed reality presentation system determines a device-specific parameter characterizing an extent of a field of view of an active imaging device. The active imaging device captures imagery of an internal view of a body, and the determining of the device-specific parameter is based on data received from the active imaging device. The mixed reality presentation system also determines a spatial pose of the active imaging device as the active imaging device captures the imagery of the internal view of the body. Based on the device-specific parameter and the spatial pose of the active imaging device, the mixed reality presentation system directs a display device to display a shape overlay indicative of the extent of the field of view relative to the body. The shape overlay is displayed together with an external view of the body in a mixed reality presentation. Corresponding systems and methods are also disclosed.

Abstract: A mixed reality presentation system identifies an operating condition associated with an operation performed on a body while an active imaging device captures imagery of an internal view of the body. The mixed reality presentation system also determines, based on the identified operating condition, that a display device is to toggle a display of a shape overlay that is displayed together with an external view of the body and that is indicative of an extent of a field of view of the active imaging device relative to the body. Based on the determining that the display device is to toggle the display of the shape overlay, the mixed reality presentation system directs the display device to toggle the display of the shape overlay. Corresponding systems and methods are also disclosed.

Abstract: A system includes a tool, an input device, a shape sensor system, and a processing unit. The processing unit is configured to determine a state estimate of the input device based on shape information from the shape sensor system, and control the tool based on the state estimate.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a second joint, and a first link. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data, and generate a second joint state estimate of the second joint. The processing unit is further configured to apply a first weight to the first joint state estimate to generate a first weighted joint state estimate, apply a second weight to the second joint state estimate to generate a second weighted joint state estimate, and control the first and second joints based on the first weighted joint state estimate and second weighted joint state estimate.