Abstract: Mixed reality images are inked with strokes and other annotations, permitting a headset, robot, or drone operator to see coherent graphical additions made by a remote user. Some embodiments get an ink stroke in an annotation which includes multiple points in a screen space, select a ray origin point based on at least samples of the annotation points, obtain a spatial mesh representation of at least a portion of an object which is at least partially shown in a mixed reality image, cast a ray from the ray origin point to the spatial mesh representation, thereby determining a ray-mesh intersection point, choose a projection plane, project the annotation onto the projection plane, and configure a display with the mixed reality image including the projected ink stroke or other annotation.

Abstract: Mixed reality images are inked with strokes and other annotations, permitting a headset, robot, or drone operator to see coherent graphical additions made by a remote user. Some embodiments get an ink stroke in an annotation which includes multiple points in a screen space, select a ray origin point based on at least samples of the annotation points, obtain a spatial mesh representation of at least a portion of an object which is at least partially shown in a mixed reality image, cast a ray from the ray origin point to the spatial mesh representation, thereby determining a ray-mesh intersection point, choose a projection plane, project the annotation onto the projection plane, and configure a display with the mixed reality image including the projected ink stroke or other annotation.