Abstract: Techniques for control using hand tracking include a system having an input control configured to be manipulated by an operator of the system, a hand tracking unit, and a controller. The controller is configured to determine a first position of the input control; receive, from the hand tracking unit, a second position of a hand of the operator; determine whether the first position is within a threshold distance of the second position; and control display of a user interface based on the determination of whether the first position is within the threshold distance of the second position.

Abstract: A patient side cart for a teleoperated surgical system may comprise a column extending from a base, the column having a first end connected to the base and a second end opposite the first end. The patient side cart may further include a surgical instrument manipulator arm coupled proximate the second end of the column, and an obstacle indication system comprising an illumination source mounted on the patient side cart at a height above a location the surgical instrument manipulator arm is coupled to the second end of the column, the height being measured in a direction the column extends from the base.

Abstract: A teleoperated system includes a robotic arm configured to support an instrument, a grip configured to be manipulated by an operator to command motion of the instrument, and a control system communicatively coupled to the robotic arm and the grip. To align the grip with the instrument by the grip, the control system is configured to determine grip rotation values describing an orientation of the grip, determine instrument rotation values describing an orientation of the instrument, determine an orientation error between the orientation of the grip and the orientation of the instrument based on the grip rotation values and the instrument rotation values, produce a motion command by selectively imposing, based on the orientation error, an artificial joint limit on a commanded movement of the instrument, and command the robotic arm to move in accordance with the motion command.

Abstract: Techniques for maintaining instrument position and orientation include a computer-assisted device having a control unit and an articulated structure. The articulated structure includes a plurality of joints and is configured to support an instrument. The control unit is configured to determine an error that is introduced, by movement of a first joint of the plurality of joints, to a position of the instrument, an orientation of the instrument, or both the position of the instrument and the orientation of the instrument; and in response at least to determining that a second joint of the plurality of joints is at a range of motion limit or is about to reach the range of motion limit drive one or more joints of the plurality of joints to partially reduce the error, the one or more joints not comprising the second joint, and provide feedback that not all of the error is reduced.

Abstract: Techniques for monitoring control points of a computer-assisted device include one or more articulated arms having one or more control points and a control unit coupled to the one or more articulated arms. Each control point of the one or more control points is located based on a kinematic configuration of the one or more articulated arms. The control unit is configured to determine, during a movement of a table that causes a change in the kinematic configuration of the one or more articulated arms, an actual spatial configuration of the one or more control points; determine, based on a comparison of the actual spatial configuration with an expected spatial configuration of the one or more control points, whether to perform a remedial action; and perform the remedial action in response to a determination to perform the remedial action.

Abstract: A system comprises a processor and a memory having computer readable instructions stored thereon. The computer readable instructions, when executed by the processor, cause the system to display a surgical environment image. The surgical environment image includes a virtual control element for controlling a component of a surgical system. The virtual control element includes a real-time image of the component of the surgical system in the surgical environment image. The computer readable instructions also cause the system to display an image of a body part of a user used to interact with the virtual control element, receive a gesture of the body part of the user in a predetermined motion via a gesture based input device, and adjust a setting of the component of the surgical system based on the received gesture.

Abstract: Techniques for registering a computer-assisted device to a table include a computer-assisted device having an articulated arm and a control unit coupled to the articulated arm. The control unit is configured to receive information of a first motion of a table. The first motion of the table caused a corresponding second motion of a point associated with the articulated arm. The control unit further configured to receive information of the second motion and determine a first angular relationship between the table and the computer-assisted device based on the first motion and the second motion.

Abstract: A medical robotic system includes a viewer, a gaze tracker, and a processor programmed to: draw an area or volume defining shape overlaid on an image based on the tracked gaze point after the user has gazed on the tracked gaze point for a programmed period of time; in response to receiving a user-selected action command, assign a fixed virtual constraint to the area or volume defining shape and constrain movement of a robotic tool; map points of the robotic tool in a tool reference frame to a viewer reference frame; determine a closest object to the tracked gaze point is the robotic tool based at least in part on the mapped points; display an object including text identifying the robotic tool, overlaid on the image, proximate to the robotic tool based on determining the robotic tool is the closest object; and perform an action indicated by the object.

Abstract: Techniques for monitoring control points of a computer-assisted device include a first articulated arm associated with a first control point and a control unit coupled to the first articulated arm. The control unit is configured to determine a first expected path representative of expected positions of the first control point over time when a movement of the first control point is caused by a movement of a table that moves the first articulated arm, the table being separate from the computer-assisted device; determine a first actual path representative of actual positions of the first control point over time when the movement of the first control point is caused by the movement of the table; determine whether to perform a first remedial action based on the first actual path and the first expected path; and in response to a determination to perform the first remedial action, perform the first remedial action.

Abstract: A robotic system provides user selectable actions associated with gaze tracking according to user interface types. User initiated correction and/or recalibration of the gaze tracking may be performed during the processing of individual of the user selectable actions.

Abstract: A computer-assisted device includes an articulated arm configured to support an end effector and a control unit. When coupled to the articulated arm and a table, the control unit is configured to detect movement of the articulated arm caused by movement of the table, determine a movement of the table based on motion data received from the table, and drive one or more first joints of the articulated arm based on the movement of the articulated arm and the determined movement of the table.

Abstract: A system comprises a teleoperational assembly including an operator input system and a teleoperational manipulator configured for operation by the operator input system. The teleoperational manipulator is coupled to a medical instrument in a surgical environment. The system also comprises a processing unit including one or more processors. The processing unit is configured to retrieve a user profile for a user. The user profile includes a parameter established during a prior teleoperational procedure. The parameter includes anthropometric data for the user. The processing unit is also configured to provide instructions to configure an operation of the teleoperational assembly based upon the parameter. The operation includes adjusting an operating table based on the anthropometric data.

Abstract: A method comprises displaying an image of an environment on a graphical user interface, the environment including personnel elements visible in the image, detecting a gaze of a user directed at an image of a first personnel element corresponding to a first information conveyor of a plurality of information conveyors, changing a state of the first information conveyor corresponding to the first personnel element from a passive state to a listening state in response to detecting that the gaze is directed at the image of the first personnel element, wherein the first information conveyor includes a first audio output device, receiving audio information through an audio input device, and when the state of the first information conveyor is changed to the listening state, controlling an output operation, wherein the output operation includes directing the received audio information as an audio output through the first audio output device.

Abstract: A system comprises a first robotic arm adapted to support and move a tool and a second robotic arm adapted to support and move a camera configured to capture an image of a camera field of view. The system further comprises an input device, a display, and a processor. The processor is configured to display a first synthetic image including a first synthetic image of the tool. The first synthetic image of the tool includes a portion of the tool outside of the camera field of view. The processor is also configured to receive a user input at the input device and responsive to the user input, change the display of the first synthetic image to a display of a second synthetic image including a second synthetic image of the tool that is different from the first synthetic image of the tool.

Abstract: Systems and methods for instrument control include a first actuator for controlling a first mechanical degree of freedom, a second actuator for controlling a second mechanical degree of freedom, and a controller. The controller is configured to command the first actuator to maintain the first actuator at a first position, wherein actuation of the first actuator is subject to a first torque limit; command the second actuator to maintain the second actuator at a second position, wherein actuation of the second actuator is subject to a second torque limit; and in response to detecting that the first actuator cannot be maintained at the first position using actuation subject to the first torque limit and the second actuator can be maintained at the second position using actuation subject to the second torque limit, terminate the command to the first actuator to maintain the first actuator at the first position.

Abstract: A method and system for hand tracking in a robotic system includes a hand tracking system and a controller coupled to the hand tracking system. The controller is configured to receive, from the hand tracking system, a plurality of locations of a hand; determine if the hand is in a first hand pose based on the plurality of locations; in response to determining that the hand is in the first hand pose, and switch the robotic system to a hand trajectory detection mode. While in the hand trajectory detection mode, the control unit is configured to detect, based on hand tracking information from the hand tracking system, that the hand has performed a first hand trajectory of a plurality of known hand trajectories; and in response to detecting the first hand trajectory, change a mode of operation of the robotic system.

Abstract: A system and method of maintaining a tool position and orientation for a computer-assisted device include a control unit and an articulated structure coupled to the control unit and including a plurality of joints. The articulated structure is configured to support an instrument. The control unit is configured to determine an error that is introduced to a position of the instrument, an orientation of the instrument, or both the position of the instrument and the orientation of the instrument by movement of a first joint of the plurality of joints; and drive at least a second joint of the plurality of joints to reduce the error.

Abstract: Systems and methods of assisting operator engagement with input devices include an input device configured to be operated by a hand of an operator, a repositionable structure coupled to the input device, a hand detection system, and a control unit. The control unit is configured to detect a position and an orientation of the hand using the hand detection system, determine, based on the position of the hand, a target position for the input device, wherein moving the input device from a current position of the input device to the target position moves the input device closer to a grasping position for the hand, and in response to determining that an orientation difference between the orientation of the hand and a current orientation of the input device is not greater than a threshold orientation difference, cause one or more actuators to move the input device toward the target position.

Abstract: Techniques for monitoring control points of a computer-assisted device include a first articulated arm associated with a first control point and a control unit coupled to the first articulated arm. The control unit is configured to determine a first expected path representative of expected positions of the first control point over time when a movement of the first control point is caused by a movement of a table that moves the first articulated arm, the table being separate from the computer-assisted device; determine a first actual path representative of actual positions of the first control point over time when the movement of the first control point is caused by the movement of the table; determine whether to perform a first remedial action based on the first actual path and the first expected path; and in response to a determination to perform the first remedial action, perform the first remedial action.

Abstract: A region-based augmentation system is configured to determine that a viewpoint of a user of a display device is directed within an anchor region of a physical world containing the user. In response, the system directs the display device to present augmented content in an evident manner The evident manner comprises anchoring the augmented content relative to the viewpoint, such that the augmented content follows the viewpoint as the user moves the viewpoint within the anchor region. The system further determines that the viewpoint is directed outside of the anchor region by the user. In response, the system directs the display device to present the augmented content in a less evident manner. The presenting of the augmented content in the less evident manner comprises presenting the augmented content less visibly than the evident manner and/or unanchoring the augmented content relative to the viewpoint. Corresponding methods and systems are also disclosed.