Abstract: A virtual surgical robot being built from kinematic data is rendered to a display. A user input is received to effect a movement or a configuration of the virtual surgical robot. The kinematic data is modified based on evaluation of the movement or the configuration of the virtual surgical robot.

Abstract: Mobile virtual reality system for simulation, training or demonstration of a surgical robotic system can include a virtual reality processor. The processor can generate a virtual surgical robot and render the virtual surgical robot on a display. The virtual surgical robot can include a virtual surgical tool. A handheld user input device (UID) can sense a hand input from a hand. A foot input device can sense a foot input from a foot. The virtual reality processor can be configured to control a movement or action of the virtual surgical robot based on the hand input, and change which of the virtual surgical instruments is controlled by the one or more handheld UIDs based on the foot input. Other embodiments and aspects are disclosed and claimed.

Abstract: A surgical robotic system senses position or orientation of an object, which may be a trocar that has a magnetic field. Magnetic field sensors are coupled to a surgical robotic arm. A machine learning model coupled to the magnetic field sensors is trained to output three-dimensional position and/or three-dimensional orientation of the trocar or other object. Other aspects are also described.

Abstract: Surgical robotic system includes a surgical robotic arm, a handheld user interface device (UID) having a camera. Images from the camera are processed to detect a marker in the user environment. A pose of the handheld UID is determined based on the detected marker. A movement of the surgical robotic arm is effected based on the pose of the handheld UID.

Abstract: A surgical robotic user input apparatus has a fiber optic cable with a handheld user input device attached at one end, and a connector attached at another end. Multiple intrinsic sensors, such as fiber Bragg grating sensors, are in the fiber optic cable. The intrinsic sensors are used to detect a pose of the handheld user input device. Other embodiments are also described and claimed.

Abstract: A surgical robotic system has a tool drive coupled to a distal end of a robotic arm that has a plurality of actuators. The tool drive has a docking interface to receive a trocar. One or more sensors in the docking interface sense a magnetic field generated by the trocar. One or more processors are configured to determine a position and orientation of the trocar based on the sensed magnetic field, and then drive the actuators to orient the docking interface to the determined orientation of the trocar, or otherwise guide the robotic arm toward the determined position of the trocar. Other aspects are also described and claimed.

Abstract: A medical image viewer can render to a display, a three-dimensional view of patient anatomy, a multi planar reconstruction (MPR) view of the patient anatomy, and an intra-operational view. Some of the views can be synchronized to show a common focal point of the anatomy. Other embodiments are described and claimed.

Abstract: A surgical robotic user input apparatus has a fiber optic cable with a handheld user input device attached at one end, and a connector attached at another end. Multiple intrinsic sensors, such as fiber Bragg grating sensors, are in the fiber optic cable. The intrinsic sensors are used to detect a pose of the handheld user input device. Other embodiments are also described and claimed.

Abstract: A surgical robotic system has a tool drive coupled to a distal end of a robotic arm that has a plurality of actuators. The tool drive has a docking interface to receive a trocar. The system also includes one or more sensors that are operable to visually sense a surface feature of the trocar. One or more processors determine a position and orientation of the trocar, based on the visually sensed surface feature. In response, the processor controls the actuators to orient the docking interface to the determined orientation of the trocar and to guide the robotic arm toward the determined position of the trocar. Other aspects are also described and claimed.

Abstract: A virtual operating room (OR) is generated that includes virtual surgical equipment based on existing OR models and existing equipment models. Sensor feedback is received that defines a physical OR having physical equipment, the physical equipment including a surgical robotic system within the physical OR. The virtual OR and the virtual surgical equipment is updated based on the sensor feedback. A layout of the virtual surgical equipment is optimized in the virtual OR. The virtual OR and the virtual surgical equipment are rendered on a display.

Abstract: Planning surgical robotic workflow with a surgical robotic system can include generating a virtual surgical environment, the virtual surgical environment including a virtual surgical robotic arm and a virtual patient. A workspace can be determined in the virtual patient. A position of a virtual tool, attached to the virtual surgical robotic arm, can be determined. A position of the virtual surgical robotic arm can be determined to maintain a reach of the virtual tool in the workspace.

Abstract: A system for modelling a portion of a patient includes a processing unit, a manipulator, a sensor, and a display device. The processing unit is configured to receive patient data and to process the patient data to generate a model of the portion of the patient. The sensor is configured to capture manipulator data. The processing unit is configured to receive the manipulator data from the sensor and to process the manipulator data to determine a position of the manipulator, an orientation of the manipulator, or both. The display device is configured to display the model on or in the manipulator.

Abstract: An autostereoscopic three-dimensional display system for surgical robotics has an autostereoscopic three-dimensional display configured to receive and display video from a surgical robotics camera, and a first sensor assembly and a second sensor assembly. A processor is configured to detect and track an eye position or a head position of a user relative to the display based on processing output data of the first sensor assembly, and to detect and track a gaze of the user based on processing output data of the second sensor assembly. The processor further is configured to modify or control an operation of the display system based on the detected gaze of the user. A spatial relationship of the display also can be automatically adjusted in relation to the user based on the detected eye or head position of the user to optimize the user's visualization of three-dimensional images on the display.

Abstract: An autostereoscopic three-dimensional display system for surgical robotics has an autostereoscopic three-dimensional display configured to receive and display video from a surgical robotics camera, and a first sensor assembly and a second sensor assembly. A processor is configured to detect and track an eye position or a head position of a user relative to the display based on processing output data of the first sensor assembly, and to detect and track a gaze of the user based on processing output data of the second sensor assembly. The processor further is configured to modify or control an operation of the display system based on the detected gaze of the user. A spatial relationship of the display also can be automatically adjusted in relation to the user based on the detected eye or head position of the user to optimize the user's visualization of three-dimensional images on the display.

Abstract: A control system for surgical robots based on proximity sensing, the control system including a proximity sensor coupled to a component of a surgical robot, the surgical robot component including a table, robotic arms coupled to the table, and surgical tools mounted on the robotic arms, the proximity sensor configured to sense a movement of a nearby controlling object in one or more degrees of freedom; and a processor configured to drive the component of the surgical robot to follow the movement of the controlling object.

Abstract: A surgical robotic system including a surgical table, a surgical robotic manipulator coupled to the surgical table and comprising a plurality of links coupled together by a plurality of joints that are operable to move with respect to one another to move the surgical robotic manipulator, at least one of the plurality of links or the plurality of joints having a portion that faces another of the plurality of links or the plurality of joints, a proximity sensing assembly coupled to the portion of the at least one of the plurality of links or the plurality of joints, the proximity sensing assembly operable to detect an object prior to the surgical robotic manipulator colliding with the object and to output a corresponding detection signal, and a processor operable to receive the corresponding detecting signal and cause the manipulator or the object to engage in a collision avoidance operation.

Abstract: A surgical robotic system has a tool drive coupled to a distal end of a robotic arm that has a plurality of actuators. The tool drive has a docking interface to receive a trocar. One or more sensors in the docking interface sense a magnetic field generated by the trocar. One or more processors are configured to determine a position and orientation of the trocar based on the sensed magnetic field, and then drive the actuators to orient the docking interface to the determined orientation of the trocar, or otherwise guide the robotic arm toward the determined position of the trocar. Other aspects are also described and claimed.

Abstract: An autostereoscopic three-dimensional display system for surgical robotics has an autostereoscopic three-dimensional display configured to receive and display video from a surgical robotics camera, and a first sensor assembly and a second sensor assembly. A processor is configured to detect and track an eye position or a head position of a user relative to the display based on processing output data of the first sensor assembly, and to detect and track a gaze of the user based on processing output data of the second sensor assembly. The processor further is configured to modify or control an operation of the display system based on the detected gaze of the user. A spatial relationship of the display also can be automatically adjusted in relation to the user based on the detected eye or head position of the user to optimize the user's visualization of three-dimensional images on the display.

Abstract: A system and method for tracking a position of an interventional medical device configured to be deployed into a subject during a medical procedure. An anatomical dataset of the subject is acquired while at least one electrical sensor is disposed along the interventional medical device. A feedback signal is generated from the electrical sensor and is processed and analyzed to identify positional information of the interventional medical device. A position of the interventional medical device with respect to the anatomical dataset is identified and the position of the interventional medical device with respect to the anatomical dataset is shown on a display.

Abstract: A surgical robotic user input apparatus has a fiber optic cable with a handheld user input device attached at one end, and a connector attached at another end. Multiple intrinsic sensors, such as fiber Bragg grating sensors, are in the fiber optic cable. The intrinsic sensors are used to detect a pose of the handheld user input device. Other embodiments are also described and claimed.