Abstract: A method of surgical robot operation includes moving a surgical instrument through a medium using the surgical robot, where the surgical robot attempts to move the surgical instrument to a desired position. The method further includes measuring an actual position of the surgical instrument, and calculating a difference between the actual position and the desired position. The actual position of the surgical instrument is adjusted to the desired position using the surgical robot.

Abstract: A method of surgical robot operation includes moving a surgical instrument through a medium using the surgical robot, where the surgical robot attempts to move the surgical instrument to a desired position. The method further includes measuring an actual position of the surgical instrument, and calculating a difference between the actual position and the desired position. The actual position of the surgical instrument is adjusted to the desired position using the surgical robot.

Abstract: Image guided surgery includes capturing a primary modality image of a surgical field of a patient with a camera system, obtaining a secondary modality image of the surgical field registered to the primary image, generating a surgical guidance image based at least in part upon the secondary modality image, and projecting the surgical guidance image onto the patient. The surgical guidance image presents visual augmentations on or over the patient to inform a medical practitioner during a surgical procedure.

Abstract: A system for the detection of blood vessels includes an image sensor coupled to generate video data including a sequence of images of the blood vessels, and a heart rate monitor to measure a heart rate of a patient and to generate heart rate data. A controller is coupled to the image sensor to receive the video data, and coupled to the heart rate monitor to receive the heart rate data. The controller includes logic that when executed by the controller causes the controller to perform operations including isolate localized motion of the blood vessels in the video data using the heart rate data. The controller also computes a blood vessel mask (that includes differences between the video data and the localized motion of the blood vessels) and combined video data (that includes the video data and the blood vessel mask).

Abstract: A method for comparing videos of a surgical procedure is disclosed. The method comprising selecting a plurality of videos from a surgical video database. Each of the plurality of videos including video data of a first surgical procedure comprising a plurality of surgical steps. The method further including identifying a first surgical step included in the plurality of surgical steps within a first video segment in each of the plurality of videos. The method also including warping the first video segment to standardize a dimension of the first video segment in each of the plurality of videos.

Abstract: A robotic system includes a communication system coupled to communicate with one or more external devices that include first user preferences. The robotic system also includes memory storing a database including second user preferences. A controller is coupled to the user interface, the communication system, and the memory, and the controller includes logic that when executed by the controller causes the robotic system to perform operations. Operations may include retrieving the first user preferences using the communication system; retrieving the second user preferences from the database; resolving conflicts between the first user preferences and the second user preferences to create a final set of user preferences; and configuring the robotic system with the revised set of user preferences.

Abstract: A user interface system includes one or more controllers configured to move freely in three dimensions, where the one or more controllers include an inertial sensor coupled to measure movement of the one or more controllers, and output movement data including information about the movement. The user interface system further includes a processor coupled to receive the movement data, where the processor includes logic that, when executed by the processor, causes the user interface system to perform operations, including receiving the movement data with the processor, identifying an unintentional movement in the movement data with the processor, and outputting unintentional movement data that identifies the unintentional movement.

Abstract: A method for pathology data capture includes magnifying a pathology sample with a microscope to form magnified pathology images, and recording the magnified pathology images with a digital camera. The method further includes transferring the magnified pathology images to a processing apparatus, where the processing apparatus performs operations including: stitching together the magnified pathology images to form a plurality of high-resolution images, where the plurality of high-resolution images include un-imaged holes; and generating image data that is not actual image data for the un-imaged holes.

Abstract: A simulator for simulating soft body deformation includes a display system, a user interface, and a controller. The controller includes one or more processors coupled to memory that stores instructions that when executed cause the system to perform operations. The operations include generating a simulated soft body having a shape represented, at least in part, by a plurality of nodes. The operations further include determining, with at least a first machine learning model, a displacement of individual nodes included in the plurality of nodes in response to a simulated force applied to the simulated soft body. The operations further include rendering a deformation of the simulated soft body in substantially real time in response to the simulated force based, at least in part, on the displacement.

Abstract: A surgical simulator for simulating a surgical scenario comprises a display system, a user interface, and a controller. The controller includes one or more processors coupled to memory that stores instructions that when executed cause the system to perform operations. The operations include generating simulated surgical videos, each representative of the surgical scenario. The operations further include associating simulated ground truth data from the simulation with the simulated surgical videos. The ground truth data corresponds to context information of at least one of a simulated surgical instrument, a simulated anatomical region, a simulated surgical task, or a simulated action. The operations further include annotating features of the simulated surgical videos based, at least in part, on the simulated ground truth data for training a machine learning model.

Abstract: A system includes a display, and a database including surgical videos, images of organs in a human body obtained from a medical imaging device, and images of disease in a human body obtained from the medical imaging device. A controller including a processor is coupled to memory, the database, and the display, and the memory stores information that when executed by the processor causes the system to perform operations. For example, the processor may determine first organ information from the images of the organs, and first disease information from the images of the disease. The processor my calculate a similarity score between the first organ information and the first disease information and second disease information and second organ information indexed to the surgical videos. The processor selects one or more of the surgical videos based on the similarity score, and displays the surgical videos on the display.

Abstract: A system for robot-assisted surgery includes an image sensor and a display. The system further includes a controller coupled to the image sensor and the display, where the controller includes logic that when executed by the controller causes the system to perform operations. The operations may include acquiring first images of a surgical procedure with the image sensor, and analyzing the first images with the controller to identify a surgical step in the surgical procedure. The operations may further include displaying second images on the display in response to identifying the surgical step; the second images may include at least one of a diagram of human anatomy, a preoperative image, an intraoperative image, or an annotated image of one of the first images.

Abstract: A system includes a microscope configured to magnify a pathology sample, a camera positioned to record magnified pathology images from the microscope, and a display configured to show the magnified pathology images. A processing apparatus is coupled to the camera, and the display, and the processing apparatus includes instructions that when executed by the processing apparatus cause the system to perform operations, including: identifying, using a machine learning algorithm, one or more regions of interest in the magnified pathology images; and alerting, using the display, a user of the microscope to the one or more regions of interest in the magnified pathology images while the pathology sample is being magnified with the microscope.

Abstract: Systems and methods for segmenting surgical videos are disclosed. One example method includes receiving, by a processor of a computing device, surgical video, the surgical video comprising at least a sequence of video frames of a surgical procedure; in response to receiving an identification of a video frame, generating, by the processor, a bookmark based on the video frame; associating, by the processor, the bookmark with the video frame; and storing, by the processor, the bookmark in a non-transitory computer-readable medium.

Abstract: A system for video-assisted surgery includes one or more displays, a memory including at least one preoperative image, and a camera coupled to capture a video. A controller is coupled to the memory, the camera, and the one or more displays, and the controller includes logic that when executed by the controller causes the system to perform a variety of operations. The system may capture a video of a surgical area, including anatomical features, using the camera, and display the video of the surgical area on the one or more displays. The system may also display the at least one preoperative image on the one or more displays at the same time as the video. The location of the anatomical features shown in the video is displayed as an accentuated region on the at least one preoperative image.

Abstract: A method for pathology data capture includes magnifying a pathology sample with a microscope to form magnified pathology images, and recording the magnified pathology images with a digital camera. The method further includes transferring the magnified pathology images to a processing apparatus, where the processing apparatus performs operations including: stitching together the magnified pathology images to form a plurality of high-resolution images, where the plurality of high-resolution images include un-imaged holes; and generating image data that is not actual image data for the un-imaged holes.

Abstract: A system includes a microscope configured to magnify a pathology sample, a camera positioned to record magnified pathology images from the microscope, a pathology database including reference pathology images, and a display configured to show the magnified pathology images. A processing apparatus is coupled to the camera, the pathology database, and the display. The processing apparatus includes instructions that when executed by the processing apparatus cause the system to perform operations. The operations include comparing the magnified pathology images to the reference pathology images included in the pathology database; identifying one or more regions of interest in the magnified pathology images; and alerting, using the display, a user of the microscope to the one or more regions of interest in the magnified pathology images while the pathology sample is being magnified with the microscope.

Abstract: A system for robotic surgery includes a surgical robot with one or more arms, where at least some of the arms in the one or more arms holds a surgical instrument. An image sensor is coupled to capture a video of a surgery performed by the surgical robot, and a display is coupled to receive an annotated video of the surgery. A processing apparatus is coupled to the surgical robot, the image sensor, and the display. The processing apparatus includes logic that when executed by the processing apparatus causes the processing apparatus to perform operations including identifying anatomical features in the video using a machine learning algorithm, and generating the annotated video. The anatomical features from the video are accentuated in the annotated video. The processing apparatus also outputs the annotated video to the display in real time.

Abstract: A method for pathology data capture includes magnifying a pathology sample with a microscope to form magnified pathology images. The method also includes recording the magnified pathology images with a digital camera optically coupled to the microscope, and recording voice annotations from a user of the microscope with a microphone. The magnified pathology images and the voice annotations are transferred to a processing apparatus electrically coupled to the digital camera and the microphone. The processing apparatus performs operations including recording the magnified pathology images and the voice annotations to a storage medium, indexing the magnified pathology images and the voice annotations with respect to recording time, and tagging the voice annotations of the user to one or more specific locations in the magnified pathology images.

Abstract: A system includes a microscope configured to magnify a pathology sample, a camera positioned to record magnified pathology images from the microscope, a pathology database including reference pathology images, and a display configured to show the magnified pathology images. A processing apparatus is coupled to the camera, the pathology database, and the display. The processing apparatus includes instructions that when executed by the processing apparatus cause the system to perform operations. The operations include comparing the magnified pathology images to the reference pathology images included in the pathology database; identifying one or more regions of interest in the magnified pathology images; and alerting, using the display, a user of the microscope to the one or more regions of interest in the magnified pathology images while the pathology sample is being magnified with the microscope.