Abstract: A system and method of coordinated motion among heterogeneous devices includes a medical device with one or more movable elements and one or more processors. Any of the processors uses a shared interface to access services. The medical device requests a movement token from a token service through the shared interface, receive the movement token from the token service, exchange configuration data, kinematic data, or planned motion data through the shared interface, plan a first motion for a first movable element of the movable elements based on the movement token and the configuration data, the kinematic data, or the planned motion data, and execute the first motion. In some embodiments, the movement token is selected from a group consisting of an exclusive-motion token, a master follow-me token, a slave follow-me token, a master collision-avoidance token, a slave collision-avoidance token, and a passive collision-avoidance token.

Abstract: A method is performed by a computing system. The method includes receiving image data from an imaging device, and determining, using the image data, a plurality of image-space tools, each image-space tool associated with a tool of a plurality of tools, each tool controlled by a manipulator of a plurality of manipulators. The method further includes determining a first correspondence between a first image-space tool of the plurality of image-space tools and a first tool of the plurality of tools based on a first disambiguation setting associated with the first tool.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a first link, and a second link. The second link is between the first joint and the first link in the serial chain. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data and a kinematic model of the robotic manipulator, and control the first joint based on the first joint state estimate.

Abstract: A system includes a tool, an input device, a shape sensor system, and a processing unit. The processing unit is configured to determine a state estimate of the input device based on shape information from the shape sensor system, and control the tool based on the state estimate.

Abstract: A mixed reality presentation system determines a device-specific parameter characterizing an extent of a field of view of an active imaging device. The active imaging device captures imagery of an internal view of a body, and the determining of the device-specific parameter is based on data received from the active imaging device. The mixed reality presentation system also determines a spatial pose of the active imaging device as the active imaging device captures the imagery of the internal view of the body. Based on the device-specific parameter and the spatial pose of the active imaging device, the mixed reality presentation system directs a display device to display a shape overlay indicative of the extent of the field of view relative to the body. The shape overlay is displayed together with an external view of the body in a mixed reality presentation. Corresponding systems and methods are also disclosed.

Abstract: A mixed reality presentation system identifies an operating condition associated with an operation performed on a body while an active imaging device captures imagery of an internal view of the body. The mixed reality presentation system also determines, based on the identified operating condition, that a display device is to toggle a display of a shape overlay that is displayed together with an external view of the body and that is indicative of an extent of a field of view of the active imaging device relative to the body. Based on the determining that the display device is to toggle the display of the shape overlay, the mixed reality presentation system directs the display device to toggle the display of the shape overlay. Corresponding systems and methods are also disclosed.

Abstract: A system includes a robotic manipulator including a serial chain comprising a first joint, a second joint, and a first link. The system further includes a processing unit including one or more processors. The processing unit is configured to receive first link data from a first sensor system located at the first link, generate a first joint state estimate of the first joint based on the first link data, and generate a second joint state estimate of the second joint. The processing unit is further configured to apply a first weight to the first joint state estimate to generate a first weighted joint state estimate, apply a second weight to the second joint state estimate to generate a second weighted joint state estimate, and control the first and second joints based on the first weighted joint state estimate and second weighted joint state estimate.

Abstract: A surgical method is provided for use with a teleoperated surgical system (surgical system), the method comprising: recording surgical instrument kinematic information indicative of surgical instrument motion produced within the surgical system during the occurrence of the surgical procedure; determining respective kinematic signatures associated with respective surgical instrument motions; producing an information structure in a computer readable storage device that associates respective kinematic signatures with respective control signals; comparing, during a performance of the surgical procedure surgical instrument kinematic information during the performance with at least one respective kinematic signature; launching, during a performance of the surgical procedure an associated respective control signal in response to a match between surgical instrument kinematics during the performance and a respective kinematic signature.

Abstract: A system and method of coordinated motion includes a device with one or more movable elements and one or more processors. The device is configured to discover a second device, determine a desired type of coordinated motion based on the discovery of the second device, request a movement token of a first type from a token service, the first type being selected based on the desired type of coordinated motion, receive the movement token from the token service, receive configuration data, kinematic data, or planned motion data for the second device, plan a first motion for a movable element of the movable elements based on the first type of the movement token and the configuration data, the kinematic data, or the planned motion data for the second device, and execute the first motion while the device holds the movement token. The device and the second device are different heterogeneous devices.

Abstract: A method of estimating a pose of a surgical table comprises reading one or more fiducial markers on a base of the surgical table with a camera mounted to a manipulator assembly adjacent the surgical table. The surgical table includes a substrate and a support structure supporting the substrate. The substrate has a support surface. The support structure is movable such that the support surface of the substrate is positionable along one or more degrees of freedom (DOF).

Abstract: A robotic system may comprise a first robotic arm operatively coupleable to a first tool. The first tool has a first working end. The system may also comprise an image capture device, a display, and a processor. The processor may be configured to cause an image of a work site, which was captured by the image capture device from a perspective of an image reference frame, to be displayed on the display. The image of the work site includes an image of the first working end of the first tool. The processor may also determine a position of the first working end of the first tool in the image of the work site and render a tool information overlay at the position of the first working end of the first tool in the image of the work site. The tool information overlay visually indicates state information for the first tool. The processor may also change the tool information overlay while the first tool is in a first operational state by changing a brightness of the tool information overlay.

Abstract: A surgical method is provided for use with a teleoperated surgical system (surgical system), the method comprising: recording surgical instrument kinematic information indicative of surgical instrument motion produced within the surgical system during the occurrence of the surgical procedure; determining respective kinematic signatures associated with respective surgical instrument motions; producing an information structure in a computer readable storage device that associates respective kinematic signatures with respective control signals; comparing, during a performance of the surgical procedure surgical instrument kinematic information during the performance with at least one respective kinematic signature; launching, during a performance of the surgical procedure an associated respective control signal in response to a match between surgical instrument kinematics during the performance and a respective kinematic signature.

Abstract: A surgical module is supported by manipulators that are removably attached to the surgical module. The surgical module may enable operation of surgical tools by providing an integration between actuating mechanisms of the manipulators and actuating mechanisms of the surgical tools. Alternatively or additionally, the surgical module may enable operation of the surgical tools by providing physical access for deploying surgical tools that are operatively connected to the manipulators.

Abstract: Methods and systems for registering a manipulator assembly and independently positionable surgical table are provided herein. In one aspect, methods include reading a fiducial marker on the surgical table with a sensor associated with the manipulator assembly and localizing the manipulator assembly and surgical table with respect to a common reference frame. Methods may further include translating a 3D configuration of the surgical table to a 2D frame of reference so as to estimate a 3D pose of the surgical table relative the manipulator assembly for use in coordinating movements therebetween.

Abstract: An operator telerobotically controls tools to perform a procedure on an object at a work site while viewing real-time images of the work site on a display. Tool information is provided in the operator's current gaze area on the display by rendering the tool information over the tool so as not to obscure objects being worked on at the time by the tool nor to require eyes of the user to refocus when looking at the tool information and the image of the tool on a stereo viewer.

Abstract: A surgical module is supported by manipulators that are removably attached to the surgical module. The surgical module may enable operation of surgical tools by providing an integration between actuating mechanisms of the manipulators and actuating mechanisms of the surgical tools. Alternatively or additionally, the surgical module may enable operation of the surgical tools by providing physical access for deploying surgical tools that are operatively connected to the manipulators.

Abstract: A teleoperated system includes a master grip and a ratcheting system coupled to the master grip. The ratcheting system is configured to align the master grip with a slave instrument commanded by the master grip by determining grip rotation values describing an orientation of the master grip, determining instrument rotation values describing an orientation of the instrument, determining an orientation error between an orientation of the master grip and the orientation of the instrument based on the grip rotation values and the instrument rotation values, and reducing the orientation error by low pass filtering the grip rotation values or the instrument rotation values.

Abstract: In a minimally invasive surgical system, a plurality of video images is acquired. Each image includes a hand pose image. Depth data for the hand pose image is also acquired or synthesized. The hand pose image is segmented from the image using the depth data. The segmented image is combined with an acquired surgical site image using the depth data. The combined image is displayed to a person at a surgeon's console of the minimally invasive surgical system. Processing each of the video images in the plurality video images in this way reproduces the hand gesture overlaid on the video of the surgical site in the display.

Abstract: A method comprises displaying a surgical environment image. The surgical environment image includes a virtual control element for controlling a component of a surgical system. The method also includes displaying an image of a body part of a user used to interact with the virtual control element. The method also comprises receiving a user input from the user with a gesture based input device while the body part interacts with the virtual control element. The method also comprises adjusting a setting of the component of the surgical system based on the received user input.

Abstract: A teleoperated system includes a master grip and a ratcheting system coupled to the master grip. The ratcheting system is configured to align the master grip with a slave instrument commanded by the master grip by determining a grip orientation by which the master grip is gripped by an operator, determining a commanded velocity for the slave instrument based on a manipulation of the master grip by the operator, determining an error, and altering the commanded velocity for the slave instrument based on at least the grip orientation and the error. The error includes a parameter selected from a group consisting of a position error between a position of the master grip and a position of the slave instrument and an orientation error between an orientation of the master grip and an orientation of the slave instrument.