Abstract: A head-mounted extended reality (XR) display device includes a rigid mounting element coupled to a frame. The XR display device further includes right-side and left-side visible light cameras coupled to the rigid mounting element, right-side and left-side near-infrared (NIR) cameras coupled to the rigid mounting element, and an NIR light-emitting diode (LED) configured to illuminate a region within a field of view of the NIR cameras. The visible light cameras are configured to capture stereoscopic visible light images within a field of view of the user when the user is wearing the frame, and the NIR cameras are configured to capture stereoscopic NIR images within the field of view of the user when the user is wearing the frame.

Abstract: A surgical robot system includes a robot having a robot base and a robot arm coupled to the robot base, and a surgical retractor. The surgical retractor includes a retractor frame, a coupler, blades, a force and/or torque sensor, and a force and/or torque feedback determination unit. The retractor frame includes arms that are translatably and/or pivotably connected to the retractor frame. The coupler releasably attaches the retractor frame to the robot arm. The blades are each coupled to and extend away from a distal end of one of the arms. The force and/or torque sensor is connected to the blades and indicates an amount of force and/or torque being applied to the blades from contact with material of a surgical site. The force and/or torque feedback determination unit determines the amount of force and/or torque being applied to the blades based on the indication.

Abstract: A surgical robot system includes a robot having a robot base and a robot arm coupled to the robot base, and a surgical retractor. The surgical retractor includes a retractor frame, a coupler, blades, a force and/or torque sensor, and a force and/or torque feedback determination unit. The retractor frame includes arms that are translatably and/or pivotably connected to the retractor frame. The coupler releasably attaches the retractor frame to the robot arm. The blades are each coupled to and extend away from a distal end of one of the arms. The force and/or torque sensor is connected to the blades and indicates an amount of force and/or torque being applied to the blades from contact with material of a surgical site. The force and/or torque feedback determination unit determines the amount of force and/or torque being applied to the blades based on the indication.

Abstract: A head-mounted extended reality (XR) display device includes a rigid mounting element coupled to a frame. The XR display device further includes right-side and left-side visible light cameras coupled to the rigid mounting element, right-side and left-side near-infrared (NIR) cameras coupled to the rigid mounting element, and an NIR light-emitting diode (LED) configured to illuminate a region within a field of view of the NIR cameras. The visible light cameras are configured to capture stereoscopic visible light images within a field of view of the user when the user is wearing the frame, and the NIR cameras are configured to capture stereoscopic NIR images within the field of view of the user when the user is wearing the frame.

Abstract: A virtual model a planned instrument attachment can be provided to ensure correct selection of a physical instrument attachment. An XR headset controller can generate a shape and a pose of the virtual model of the planned instrument attachment based on predetermined information associated with the planned instrument attachment and based on a pose of an instrument relative to the XR headset. An XR headset can display the virtual model on a see-through display screen of the XR headset that is configured to allow at least a portion of a real-world scene to pass therethrough.

Abstract: The present disclosure is directed to augmented reality navigation systems and methods of their use that, inter alia, address the need for systems and methods of robotic surgical system navigation with reduced distraction to surgeons. Augmented reality navigation systems disclosed herein enable a surgeon to maintain focus on a surgical site and/or surgical tool being used in a surgical procedure while obtaining a wide range of navigational information relevant to the procedure. Navigational information can appear in the augmented reality navigation system as being presented on virtual displays that sit in a natural field of view of a surgeon during a procedure. Navigational information can also appear to be overlaid over a patient’s anatomy. Augmented reality navigation systems comprise a head mounted display comprising an at least partially transparent display screen, at least one detector connected to the head mounted display for identifying real-world features, and a computer subsystem.

Abstract: A method and system for determining an extent of matter removed from a targeted anatomical structure are disclosed. The method includes acquiring an initial representation of a targeted anatomical structure and then removing matter from the targeted anatomical structure. An instrument is then navigated within the targeted anatomical structure. The instrument includes a tracking array, and a relative position of the instrument within the targeted anatomical structure is determined by the tracking array. The method includes recording the relative position of the instrument within the targeted anatomical structure to determine a final representation of the targeted anatomical structure. Finally, the method includes determining an extent of matter removed from the targeted anatomical structure by comparing the initial representation of the targeted anatomical structure with the final representation of the targeted anatomical structure.

Abstract: A virtual model a planned instrument attachment can be provided to ensure correct selection of a physical instrument attachment. An XR headset controller can generate a shape and a pose of the virtual model of the planned instrument attachment based on predetermined information associated with the planned instrument attachment and based on a pose of an instrument relative to the XR headset. An XR headset can display the virtual model on a see-through display screen of the XR headset that is configured to allow at least a portion of a real-world scene to pass therethrough.

Abstract: A method and system for determining an extent of matter removed from a targeted anatomical structure are disclosed. The method includes acquiring an initial representation of a targeted anatomical structure and then removing matter from the targeted anatomical structure. An instrument is then navigated within the targeted anatomical structure. The instrument includes a tracking array, and a relative position of the instrument within the targeted anatomical structure is determined by the tracking array. The method includes recording the relative position of the instrument within the targeted anatomical structure to determine a final representation of the targeted anatomical structure. Finally, the method includes determining an extent of matter removed from the targeted anatomical structure by comparing the initial representation of the targeted anatomical structure with the final representation of the targeted anatomical structure.

Abstract: A method and system for determining an extent of matter removed from a targeted anatomical structure are disclosed. The method includes acquiring an initial representation of a targeted anatomical structure and then removing matter from the targeted anatomical structure. An instrument is then navigated within the targeted anatomical structure. The instrument includes a tracking array, and a relative position of the instrument within the targeted anatomical structure is determined by the tracking array. The method includes recording the relative position of the instrument within the targeted anatomical structure to determine a final representation of the targeted anatomical structure. Finally, the method includes determining an extent of matter removed from the targeted anatomical structure by comparing the initial representation of the targeted anatomical structure with the final representation of the targeted anatomical structure.

Abstract: A camera tracking system for computer assisted navigation during surgery operatively determines a first pose of a second extended-reality (XR) headset relative to stereo tracking cameras located on a first XR headset based on first tracking information from the stereo tracking cameras. The camera tracking system determines a second pose of eyes of a user wearing the second XR headset relative to the stereo tracking cameras located on the first XR headset based on second tracking information from the stereo tracking cameras. The camera tracking system also calibrates an eye-to-display relationship defining pose of the eyes of the user wearing the second XR headset to a display device of the second XR headset based on the determined first and second poses. The camera tracking system also controls where symbols are displayed on the display device of the second XR headset based on the eye-to-display relationship.

Abstract: A camera tracking system for computer assisted navigation during surgery operatively determines a first pose of a second extended-reality (XR) headset relative to stereo tracking cameras located on a first XR headset based on first tracking information from the stereo tracking cameras. The camera tracking system determines a second pose of eyes of a user wearing the second XR headset relative to the stereo tracking cameras located on the first XR headset based on second tracking information from the stereo tracking cameras. The camera tracking system also calibrates an eye-to-display relationship defining pose of the eyes of the user wearing the second XR headset to a display device of the second XR headset based on the determined first and second poses. The camera tracking system also controls where symbols are displayed on the display device of the second XR headset based on the eye-to-display relationship.

Abstract: A virtual model a planned instrument attachment can be provided to ensure correct selection of a physical instrument attachment. An XR headset controller can generate a shape and a pose of the virtual model of the planned instrument attachment based on predetermined information associated with the planned instrument attachment and based on a pose of an instrument relative to the XR headset. An XR headset can display the virtual model on a see-through display screen of the XR headset that is configured to allow at least a portion of a real-world scene to pass therethrough.

Abstract: A head-mounted extended reality (XR) display device includes a rigid mounting element coupled to a frame. The XR display device further includes right-side and left-side visible light cameras coupled to the rigid mounting element, right-side and left-side near-infrared (NIR) cameras coupled to the rigid mounting element, and an NIR light-emitting diode (LED) configured to illuminate a region within a field of view of the NIR cameras. The visible light cameras are configured to capture stereoscopic visible light images within a field of view of the user when the user is wearing the frame, and the NIR cameras are configured to capture stereoscopic NIR images within the field of view of the user when the user is wearing the frame.

Abstract: A virtual model a planned instrument attachment can be provided to ensure correct selection of a physical instrument attachment. An XR headset controller can generate a shape and a pose of the virtual model of the planned instrument attachment based on predetermined information associated with the planned instrument attachment and based on a pose of an instrument relative to the XR headset. An XR headset can display the virtual model on a see-through display screen of the XR headset that is configured to allow at least a portion of a real-world scene to pass therethrough.

Abstract: A camera tracking system for computer assisted navigation during surgery operatively determines a first pose of a second extended-reality (XR) headset relative to stereo tracking cameras located on a first XR headset based on first tracking information from the stereo tracking cameras. The camera tracking system determines a second pose of eyes of a user wearing the second XR headset relative to the stereo tracking cameras located on the first XR headset based on second tracking information from the stereo tracking cameras. The camera tracking system also calibrates an eye-to-display relationship defining pose of the eyes of the user wearing the second XR headset to a display device of the second XR headset based on the determined first and second poses. The camera tracking system also controls where symbols are displayed on the display device of the second XR headset based on the eye-to-display relationship.

Abstract: A head-mounted extended reality (XR) display device includes a rigid mounting element coupled to a frame. The XR display device further includes right-side and left-side visible light cameras coupled to the rigid mounting element, right-side and left-side near-infrared (NIR) cameras coupled to the rigid mounting element, and an NIR light-emitting diode (LED) configured to illuminate a region within a field of view of the NIR cameras. The visible light cameras are configured to capture stereoscopic visible light images within a field of view of the user when the user is wearing the frame, and the NIR cameras are configured to capture stereoscopic NIR images within the field of view of the user when the user is wearing the frame.

Abstract: A head-mounted extended reality (XR) display device includes a rigid mounting element coupled to a frame. The XR display device further includes right-side and left-side visible light cameras coupled to the rigid mounting element, right-side and left-side near-infrared (NIR) cameras coupled to the rigid mounting element, and an NIR light-emitting diode (LED) configured to illuminate a region within a field of view of the NIR cameras. The visible light cameras are configured to capture stereoscopic visible light images within a field of view of the user when the user is wearing the frame, and the NIR cameras are configured to capture stereoscopic NIR images within the field of view of the user when the user is wearing the frame.

Abstract: A shroud for an extended reality (XR) headset may include an opacity filter including a first region having a first light transmissivity, and a second region having a second light transmissivity different from the first light transmissivity. The shroud may also include a connection mechanism for removably connecting the opacity filter to the XR headset to position at least a portion of the opacity filter in a user's field of view when viewing a real-world scene while the XR headset is worn by the user.

Abstract: A virtual model a planned instrument attachment can be provided to ensure correct selection of a physical instrument attachment. An XR headset controller can generate a shape and a pose of the virtual model of the planned instrument attachment based on predetermined information associated with the planned instrument attachment and based on a pose of an instrument relative to the XR headset. An XR headset can display the virtual model on a see-through display screen of the XR headset that is configured to allow at least a portion of a real-world scene to pass therethrough.