Abstract: Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an Interaction Engine that generates within a unified three-dimensional (3D) coordinate space: (i) a 3D virtual medical model positioned according to a model pose and (ii) at least one virtual object associated with a physical instrument, the physical instrument having a current instrument pose based at least on current coordinates of one or more fiducial markers disposed on the physical instrument, in the unified 3D coordinate space. The Interaction Engine renders an Augmented Reality (AR) display that includes concurrent display of the 3D virtual medical model and the virtual object. The Interaction Engine modifies the AR display according to a virtual interaction related to the virtual object that incorporates the change of the physical instrument pose.

Abstract: Various embodiments of a physical instrument are described herein. The physical instrument includes a main tubular body with a first terminal portion and a second terminal portion. The physical instrument further comprises a code platform proximate to the first terminal portion of the main tubular body. The code platform includes a plurality of different codes. The physical instrument also includes at least one of: (i) a flat tip at the second terminal portion and (ii) a passage internal to the main tubular body and extending from the first terminal portion to the second terminal portion.

Abstract: Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to Field Visualization Engine. The Field Visualization Engine tracks one or more collimator poses relative to one or more Augmented Reality (AR) headset device poses. Each respective collimator pose and each respective headset device pose corresponds to a three-dimensional (3D) unified coordinate space (“3D space”). The Field Visualization Engine generates an AR representation of a beam emanating from the collimator based at least on a current collimator pose and a current headset device pose. The Field Visualization Engine further generates an AR visualization of emanation of the beam throughout an AR display of medical data.

Abstract: Various embodiments of a physical instrument are described herein. The physical instrument comprises a reference array platform having a top surface and a bottom surface. A reference array including one or more different fiducial markers is disposed on the top surface of the reference array platform. The reference array platform may have a bent physical configuration or a tilted physical configuration. An adhesive layer is disposed on the bottom surface of the reference array platform.

Abstract: Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an Interaction Engine. According to various embodiments, the Interaction Engine generates, within a unified three-dimensional (3D) coordinate space: a virtual 3D medical model positioned according to a current model pose and at least one a virtual 3D hand representation. The Interaction Engine renders an AR display that includes display of the virtual 3D medical model positioned according to the current model pose and the virtual 3D hand representation. The Interaction Engine identifies selection of a slate virtual interaction based on the type of movement of the virtual 3D hand representation. The Interaction Engine modifies the AR display by rendering a virtual slate concurrently displayed with the virtual 3D medical model. The Interaction Engine adjusts the display of the virtual 3D medical model based at least on subsequent physical gestures with respect to the virtual slate.

Abstract: Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an Interaction Engine. The Interaction Engine generates, within a unified three-dimensional (3D) coordinate space: (i) a 3D virtual medical model positioned according to a model pose and (ii) at least one 3D virtual slice that corresponds with a view of respective slice layer from a plurality of slice layers associated with the 3D virtual medical model. The Interaction Engine renders an Augmented Reality (AR) display that includes concurrent display of the 3D virtual medical model and the 3D virtual slice(s). The Interaction Engine detects one or more physical gestures associated with the user and the physical instrument. The Interaction Engine identifies at least one interaction associated with the detected physical gestures and modifies the AR display according to the identified interaction.

Abstract: Disclosed herein in a method and system for creating and utilizing a virtual-three dimensional image in a real-world environment to guide medical procedures. The process may include receiving image data related to a patient and creating and displaying a three-dimensional holographic model of the patient utilizing the received image data, wherein creating a three-dimensional holographic model of the patient utilizing the received image data. Exemplary methods include organizing image data in a unique manner and utilizing subvolume skipping techniques allowing for more efficient accurate rendering of holographic images.

Abstract: Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an Interaction Engine. According to various embodiments, the Interaction Engine generates within a unified three-dimensional (3D) coordinate space: (i) a virtual 3D model container; (ii) a virtual 3D medical model positioned according to a model pose within the virtual 3D model container; and (iii) a virtual 3D representation of at least a portion of at least one of a user's hands. The Interaction Engine renders an Augmented Reality (AR) display that includes concurrent display of the virtual 3D medical model container, the virtual 3D medical model and the virtual 3D representation of the user's hands. The Interaction Engine detects one or more physical gestures associated with the user and one or more types of virtual interacts associated with a detected physical gesture(s).

Abstract: Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to Registration Engine. The Registration Engine tracks positions of a fiducial marker relative to positions of an Augmented Reality (AR) headset device worn by a user. The Registration Engine receives respective registration landmarks that correspond with locations of a physical region of a patient's anatomy and the tracked positions of the fiducial marker. The Registration Engine generates an AR display of medical data at the AR headset device in alignment with the physical region of a patient's anatomy that corresponds with the respective registration landmarks.

Abstract: Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to a workstation computing system(s) (“workstation”) that receives camera pose data from a camera(s) disposed on a first headset device. The workstation receives camera image data comprising one or more images portraying a current view of the camera. The workstation generates a rendering of the 3D model of medical data for display within a composite image. The workstation generates one or more portions of the composite image based on the camera image data. The workstation generates and displays a composite image to include simultaneous and contiguous portrayal of the rendering of the 3D model of medical data with the one or more portions based on the camera image data.

Abstract: Disclosed herein in a method and system for creating and utilizing a virtual-three dimensional image in a real-world environment to guide medical procedures. The process may include receiving image data related to a patient and creating and displaying a three-dimensional holographic model of the patient utilizing the received image data, wherein creating a three-dimensional holographic model of the patient utilizing the received image data. Exemplary methods include organizing image data in a unique manner and utilizing subvolume skipping techniques allowing for more efficient accurate rendering of holographic images.

Abstract: Disclosed herein in a method and system for creating and utilizing a virtual-three dimensional image in a real-world environment to guide medical procedures. The process may include receiving image data related to a patient and creating and displaying a three-dimensional holographic model of the patient utilizing the received image data, wherein creating a three-dimensional holographic model of the patient utilizing the received image data. Exemplary methods include organizing image data in a unique manner and utilizing subvolume skipping techniques allowing for more efficient accurate rendering of holographic images.