Abstract: A system comprises a teleoperational assembly including an operator control system and a first teleoperational manipulator configured for operation by an operator control device of the operator control system. The first teleoperational manipulator is configured to control the operation of a first medical instrument in a surgical environment. The system also comprises a processing unit including one or more processors. The processing unit is configured to display an image of a field of view of the surgical environment and display a menu proximate to an image of the first medical instrument in the image of the field of view. The menu includes at least one icon representing a function for the first medical instrument.

Abstract: An illustrative apparatus may perform gaze-based auto-exposure management of image frames. For example, the apparatus may determine, for an image frame within an image frame sequence captured by an image capture system, auto-exposure values for pixel units into which the image frame is divided. The apparatus may then assign weight values to the pixel units based on a gaze direction of a viewer viewing the image frame sequence. Based on the auto-exposure values determined for the pixel units and the weight values assigned to the pixel units, the apparatus may update one or more auto-exposure parameters for use by the image capture system to capture an additional image frame within the image frame sequence. Corresponding apparatuses, systems, and methods for gaze-based auto-exposure management of image frames are also disclosed.

Abstract: An apparatus includes a controller coupled to a stereoscopic image viewer. The stereoscopic image viewer includes a viewing lens, an image capture unit, and a plurality of light sources. The image capture unit is configured to capture image frames. The controller receives the captured image frames. The controller is configured to switch light output from one of the plurality of light sources to another one of the plurality of light sources if a corneal reflection in a captured image frame intersects a fixed reflection in the captured image frame.

Abstract: A system comprises a display configured to present a graphical user interface including a viewing area and one or more user interface elements. The system also comprises a gaze tracking device configured to detect a change in a gaze of a user while the user views the graphical user interface presented on the display. The system also comprises an audio input device configured to receive audio information. The system also comprises one or more processors configured to process the audio information received at the audio input device in accordance with a first mode to direct the received audio information as audio output to one or more audio output devices; and in response to the gaze tracking device detecting the change in the gaze of the user, process the received audio information in accordance with a second mode, the second mode being distinct from the first mode.

Abstract: A teleoperational medical system comprises a teleoperational assembly including an operator control system, a first teleoperational manipulator, a first medical instrument, and a processing unit including one or more processors. The first teleoperational manipulators is configured for operation by an operator control device of the operator control system. The first teleoperational manipulator is configured to control the operation of the first medical instrument in a surgical environment. The processing unit is configured to display an image of a field of view of the surgical environment and display a menu proximate to an image of the first medical instrument in the image of the field of view. The menu includes at least one state indicator representing a state of a component of the first medical instrument.

Abstract: A method comprises displaying an image of an environment on a graphical user interface, the environment including personnel elements visible in the image, detecting a gaze of a user directed at an image of a first personnel element corresponding to a first information conveyor of a plurality of information conveyors, changing a state of the first information conveyor corresponding to the first personnel element from a passive state to a listening state in response to detecting that the gaze is directed at the image of the first personnel element, wherein the first information conveyor includes a first audio output device, receiving audio information through an audio input device, and when the state of the first information conveyor is changed to the listening state, controlling an output operation, wherein the output operation includes directing the received audio information as an audio output through the first audio output device.

Abstract: An apparatus includes a controller coupled to a stereoscopic image viewer. The stereoscopic image viewer includes a viewing lens, an image capture unit, and a plurality of light sources. The image capture unit is configured to capture image frames. The controller receives the captured image frames. The controller is configured to switch light output from one of the plurality of light sources to another one of the plurality of light sources if a corneal reflection in a captured image frame intersects a fixed reflection in the captured image frame.

Abstract: An apparatus includes a controller coupled to a stereoscopic image viewer. The stereoscopic image viewer includes a viewing lens, an image capture unit, and a plurality of light sources. The image capture unit is configured to capture image frames. The controller receives the captured image frames. The controller is configured to switch light output from one of the plurality of light sources to another one of the plurality of light sources if a corneal reflection in a captured image frame intersects a fixed reflection in the captured image frame.

Abstract: A system comprises a teleoperational assembly including an operator control system and a first teleoperational manipulator configured for operation by an operator control device of the operator control system. The first teleoperational manipulator is configured to control the operation of a first medical instrument in a surgical environment. The system also comprises a processing unit including one or more processors. The processing unit is configured to display an image of a field of view of the surgical environment and display a menu proximate to an image of the first medical instrument in the image of the field of view. The menu includes at least one icon representing a function for the first medical instrument.

Abstract: An exemplary system may access head presence data and eye tracking data. The head presence data indicates a presence or an absence of a head of a user within a vicinity of a viewer console. The eye tracking data indicates whether an eye of a user is gazing at a display device. If the head presence data indicates that the head of the user is present and the eye tracking data indicates that the eye of the user is gazing at the display device, the system directs the user control system to operate in a first operating mode. If the head presence data indicates that the head of the user is present and the eye tracking data indicates that the eye of the user is not gazing at the display device, the system directs the user control system to operate in a second operating mode different from the first operating mode.

Abstract: An exemplary image viewing system includes a display device configured to emit visible light representative of an image provided for display by the image viewing system, an infrared light source configured to emit infrared light, a viewing lens, and a black glass mirror provided between the infrared light source and the viewing lens. The black glass mirror comprises black glass and a coating on an outer surface of the black glass. The coating is configured to reflect a first portion of the visible light emitted from the display device towards the viewing lens, pass a second portion of the visible light emitted from the display device to the black glass, and pass a portion of the infrared light emitted from the infrared light source toward the viewing lens. The black glass is configured to pass the portion of the infrared light and absorb the second portion of the visible light.

Abstract: An exemplary processing system of a computer-assisted surgical system identifies an object in a surgical space, accesses a model of the object, identifies a pose of the object in the surgical space, aligns the model with the pose of the object to define depth data for the model in the surgical space, and generates a depth map in the surgical space for at least a portion of the object based on the depth data for the model. Corresponding systems and methods are also described.

Abstract: A method comprises displaying an image of a field of view of a surgical environment. A first medical instrument in the field of view may be coupled to a first manipulator in a teleoperational assembly. The method may comprise displaying a menu proximate to an image of the first medical instrument in the image of the field of view. The menu may include a plurality of icons wherein each icon is associated with a function for the first medical instrument. The method may also comprise identifying a selected icon from the plurality of icons based upon a movement of an operator control device of a teleoperational operator control system.

Abstract: An illustrative apparatus may perform gaze-based auto-exposure management of image frames. For example, the apparatus may determine, for an image frame within an image frame sequence captured by an image capture system, auto-exposure values for pixel units into which the image frame is divided. The apparatus may then assign weight values to the pixel units based on a gaze direction of a viewer viewing the image frame sequence. Based on the auto-exposure values determined for the pixel units and the weight values assigned to the pixel units, the apparatus may update one or more auto-exposure parameters for use by the image capture system to capture an additional image frame within the image frame sequence. Corresponding apparatuses, systems, and methods for gaze-based auto-exposure management of image frames are also disclosed.

Abstract: An apparatus includes a controller coupled to a stereoscopic image viewer. The stereoscopic image viewer includes a viewing lens, an image capture unit, and a plurality of light sources. The image capture unit is configured to capture image frames. The controller receives the captured image frames. The controller is configured to switch light output from one of the plurality of light sources to another one of the plurality of light sources if a corneal reflection in a captured image frame intersects a fixed reflection in the captured image frame.

Abstract: A console for a medical includes a display, an eye tracking input module, a second input module, and one or more processors coupled to the display, the eye tracking input module, and the second input module. The one or more processors is configured to perform operations including receiving a first user input from the eye tracking input module; determining, based on the first user input, whether a gaze of a user is directed toward an activation region of the display; in response to determining that the gaze of the user is directed toward the activation region, displaying, via the display, a control; receiving a second user input from the second input module; and adjusting the control based on the second user input.

Abstract: A method comprises displaying an image of a field of view of a surgical environment. A first medical instrument in the field of view may be coupled to a first manipulator in a teleoperational assembly. The method may comprise displaying a menu proximate to an image of the first medical instrument in the image of the field of view. The menu may include a plurality of icons wherein each icon is associated with a function for the first medical instrument. The method may also comprise identifying a selected icon from the plurality of icons based upon a movement of an operator control device of a teleoperational operator control system.

Abstract: A system comprises a teleoperational assembly including an operator control system and a first teleoperational manipulator configured for operation by an operator control device of the operator control system. The first teleoperational manipulator is configured to control the operation of a first medical instrument in a surgical environment. The system also comprises a processing unit including one or more processors. The processing unit is configured to display an image of a field of view of the surgical environment and display a menu proximate to an image of the first medical instrument in the image of the field of view. The menu includes at least one icon representing a function for the first medical instrument.

Abstract: An apparatus includes a stereoscopic image viewer. The stereoscopic image viewer includes a housing, a first lens, a second lens, and an eye-tracking image capture unit. The first lens, the second lens, and the eye-tracking image capture unit are contained in the housing. The eye-tracking image capture unit has a field of view and the eye-tracking image capture unit being is mounted within the housing with the first lens and the second lens within the field of view.

Abstract: A method and apparatus for gaze-based communications. A gaze of a user is detected being directed at an element in a graphical user interface. The graphical user interface is displayed on a display system located in an environment. A state of the element is changed in response to detecting that the gaze is directed at the element. Audio information is received through an audio input device. An output operation is controlled based on the state of the element such that the audio information received through the audio input device is directed to an information conveyor corresponding to the element.