Abstract: An apparatus for remote interaction with an object includes a local system and a remote system. The local system includes a mixed reality headset configured to display a virtual representation of a physical element relative to the object, and a device configured to obtain information relating to or dependent on position and orientation of the physical element interacting with the object. The remote system includes an electronic device including a display for displaying an image based on the information, and an input device for remote user interaction and configured to control position and orientation of the virtual representation displayed on the mixed reality headset. The remote system is configured to communicate with the local system with latency of 0.5 seconds or less, facilitating alignment of the physical element with the position and orientation of the virtual representation.

Abstract: An apparatus for remote interaction with a patient includes a local system including a physical element to interact with the patient, and a local electronic device with a camera to capture images of the physical element interacting with the patient, and a local display to display the images with a virtual representation of the physical element relative to the patient. The local electronic device obtains information dependent on position and orientation of the physical element interacting with the patient. A remote system includes a remote electronic device with a display to display the images of the physical element interacting with the patient, and an image based on the information. A remote input device controls position and orientation of the virtual representation displayed on the local display. The remote system communicates with the local system with low latency for alignment of the physical element with the virtual representation.