Abstract: An augmented reality display system and method for use during a surgical procedure are disclosed. An imaging device captures a real image of a surgical area during the surgical procedure generates a first data feed. A sensor device generates a second data feed. An augmented reality device including an augmented reality display generates an augmented overlay based on the first and second data feeds. The augmented overlay includes a visual portion and a non-visual visual portion. A processor receives the first and second data feeds and combines the first and second data feeds to generate the augmented overlay.

Abstract: An augmented reality display system used during a surgical procedure is disclosed. The augmented reality display system includes an imaging device to capture a real image of a surgical area during the surgical procedure. An augmented reality display presents an overlay of an operational aspect of a surgical instrument being actively visualized onto the real image of the surgical area. The overlay combines aspects of tissue interaction in the surgical area with functional data received from the surgical instrument. A processor receives the functional data from the surgical instrument, determines the overlay related to the operational aspect of the surgical instrument, and combines the aspect of the tissue in the surgical area with the functional data received from the surgical instrument.

Abstract: Apparatuses, systems, and methods for displaying interactive overlays for multiple users are disclosed herein. In some aspects, a system for displaying interactive overlays for multiple users can include a surgical hub configured to receive a plurality of data streams related to a surgical procedure and a first augmented reality display device communicably coupled to the surgical hub. In one aspect, the first augment reality display device is linked to a first user. In another aspect, the first augmented reality display device displays a first interactive overlay customized for the first user based on at least one of the plurality of data streams.

Abstract: A camera system integrated into a trocar. The camera system allows for wide field of view of an internal surgical site and 3D mapping of fiducial markers during a laparoscopic procedure. Upon entry into the patient, the camera system is configured to deploy from a recessed position at the distal end of the trocar. In various aspects, the internal camera system is configured to keep the trocar ports free for surgical instruments and provide the surgical staff members with an increased view of the surgical environment.

Abstract: A method for presenting surgical data onto a livestream of a surgical field on a display during a surgical procedure is disclosed. The method includes detecting, by a control module, surgical data, assigning, by the control module, display priority values to the surgical data, determining, by the control module, a display arrangement of the surgical data on the display based on the display priority values, and presenting onto the livestream visual representations of the surgical data in accordance with the display arrangement.

Abstract: A surgical system is disclosed including an imaging device, a display, and a control system operably coupled to the imaging device and the display. The display is configured to show a livestream of a surgical field of a surgical procedure. The livestream is captured by the imaging device. The control system is configured to overlay, on the livestream, a first amount of information associated with the surgical procedure, detect an event configured to cause the first amount of information being overlaid to increase to a second amount of information being overlaid, compare the second amount of information to a distraction threshold, adjust the second amount of information to a third amount of information based on the comparison, and overlay, on the livestream, the third amount of information based on the second amount of information being adjusted. The third amount of information is less than the distraction threshold.

Abstract: Apparatuses, systems, and methods for displaying a cooperative overlays based on interacting instruments are disclosed herein. In one aspect, a method for displaying cooperative overlays includes communicatively coupling a first augmented reality display device to a surgical hub; generating, by the first augmented reality display device, a first intraoperative display of a surgical field; displaying, by the first intraoperative display, a first data overlay based on operation of a first surgical instrument; interacting the first surgical instrument with a second surgical instrument; and displaying, by the first intraoperative display, an indicator of the interaction based on the interaction between the first and second surgical instruments.

Abstract: A method of assessing inter-device communication pairing in a surgical setting, may include transmitting, by a first intelligent medical device, wireless communication data within the surgical setting, receiving, by a second intelligent medical device, the wireless communication data from the first intelligent medical device, determining, by the second intelligent medical device, communication pairing data indicative of an inter-device communication pairing of the second intelligent medical device with the first intelligent medical device, transmitting, by the second intelligent medical device, the communication pairing data to a modular control tower, and displaying, by the modular control tower on a display device, an augmented reality display comprising one or more virtual objects indicative of the inter-device communication pairing. An interactive surgical system may include multiple intelligent medical devices and displays which can form communication pairs in this manner.

Abstract: An augmented reality display system used during a surgical procedure is disclosed. The system includes an imaging device to capture a real image of a surgical area during the surgical procedure. An augmented reality display presents functional overlay data associated with critical operations of a surgical instrument being actively visualized and interactions of the surgical instrument with tissue in the surgical area. The functional data is overlaid onto the real image of the surgical area. The functional data overlay is a combination of aspects of the critical operations of the surgical instrument and the interaction of the surgical instrument with the tissue in the surgical area. A processor receives functional data from the surgical instrument, determines the overlaid data related to the functional aspect of the surgical instrument, and combines the aspect of the tissue in the surgical area with the functional data received from the surgical instrument.

Abstract: A system, method, and device for displaying relevant augmented reality (AR) content to a surgical staff members without over-saturating the augmented display with virtual elements. The surgical staff members (e.g., doctors, nurses, surgeons, technicians, etc.) require AR content that is delicately balanced between displaying helpful information without distracting the surgeon. A surgical hub receives a plurality of inputs related to surgical environment and displays only necessary information to allow the surgeon to provide effective care to the patient, based on a determined trigger event.

Abstract: Apparatuses, systems, and methods for method for mixed reality visualization are disclosed herein. In one aspect, a method for mixed reality visualization includes capturing, by a first camera of a first visualization system, an image of an object in a surgical field, wherein a first portion of the object is outside of a field of view of the first camera; tracking, by a tracking system, a position of a second portion of the object; determining, by a surgical hub, an attribute of the object based on the tracked position of the second portion of the object, wherein the attribute of the object is related to the first portion of the object outside of a field of view of the camera; and displaying, by an augmented reality display device, the captured image of the object in the surgical field and a graphic based on the attribute of the object.

Abstract: A method of distributing data among members of a surgical team may include receiving imaging data, by a modular control tower, from a plurality of imaging devices, receiving device-dependent data, by the modular control tower, from each of a plurality of intelligent surgical instruments, associating, by the modular control tower, a display device with a member of the surgical team, defining, by the modular control tower, a functional role for the member of the surgical team, and displaying, by the modular control tower, a augmented reality display by the display device. The augmented reality display on the display device may include virtual objects based on the imaging data, the device-dependent data, the functional role of the member of the surgical team, and a surgical activity by the member of the surgical team. An interactive surgical system may implement this method.

Abstract: A surgical system is for use in a surgical procedure. The surgical system includes a surgical instrument configured to treat tissue in the surgical procedure; an imaging device; a display configured to show a livestream of a surgical field of the surgical procedure, wherein the livestream is captured by the imaging device; and a control module. The control module is configured to detect a surgical risk; assign a display priority to the surgical risk; and determine a display arrangement of the surgical risk based on the display priority, wherein the display arrangement comprises overlaying an alert feature on the livestream.

Abstract: A surgical visualization system is disclosed including an imaging device, a display configured to show a livestream of a surgical field of a surgical procedure, a control module configured to detect surgical data competing for presentation on the display, generate visual contents based on the surgical data, and selectively overlay the visual contents onto the livestream of the surgical field based on the surgical data. The livestream is captured by the imaging device.

Abstract: A surgical visualization system is disclosed including an imaging device, a display configured to show a livestream of a surgical field of a surgical procedure, and a control module configured to detect surgical data, assign display priority values to the surgical data, determine a display arrangement of the surgical data based on the display priority values, and present the surgical data on the display in accordance with the display arrangement. The livestream is captured by the imaging device.

Abstract: A surgical system is disclosed including an imaging device, a display configured to show a livestream of a surgical field of a surgical procedure, and a control system operably coupled to the imaging device and the display. The livestream is captured by the imaging device. The control system is configured to overlay, on the livestream, information associated with the surgical procedure, detect an occurrence of a triggering event, and adjust the overlaid information based on the occurrence of the triggering event.

Abstract: A surgical instrument comprising a handle, a shaft rotatable relative to the handle, and an articulation joint rotatable relative to the shaft. The surgical instrument further comprises a motor-driven articulation system including articulation controls which are flipped by a control system of the surgical instrument when the shaft is rotated upside down relative to the handle.

Abstract: Various methods are disclosed. One method includes reading an RFID card that is associated with a surgical instrument when the RFID card is located proximal to an RFID reader and initiating a wireless pairing process with the surgical instrument by the header module. Another method includes receiving an accessory at an accessory port of a modular energy system, communicating via a flexible serial communication interface coupled between a processor and the accessory port, the flexible serial communication interface configured to support multiple communication protocols, and detecting, by a presence detection circuit coupled between the accessory port and the processor, presence of the accessory connected to the accessory port. Other methods include a flexible serial bus power configuration method for a modular energy system and a remote power control interface method for a modular energy system.

Abstract: Systems, methods and devices for initiating pairing of wireless surgical instruments to a modular energy system are disclosed herein. In various aspects, a modular energy system includes an a radio frequency identification (RFID) reader that is configured to read an RFID card that is uniquely associated with a surgical instrument. When the RFID reader reads the RFID card, the RFID reader initiates a wireless pairing process with the surgical instrument. Initiating the wireless pairing process may include causing the module energy system to wirelessly search for the surgical instrument and causing a display screen to display instructions for wirelessly pairing the surgical instrument.

Abstract: A surgical instrument operable to sever tissue includes a body assembly and a selectively coupleable end effector assembly. The end effector assembly may include a transmission assembly and an end effector. The body assembly includes a trigger and a casing configured to couple with the transmission assembly. An information transmission system transmits instrument information received from a sensor, for example, to a secure server via a secure gateway connected to the instrument. The instrument may be previously tested on a calibration kit to pre-determine and load surgeon-specific settings onto the instrument prior to use.