Abstract: Systems and methods for virtual reality or augmented reality (VR/AR) visualization of 3D medical images using a VR/AR visualization system are disclosed that includes a computing device operatively coupled to a VR/AR device, which includes a holographic display and at least one sensor. The holographic display is configured to display a holographic image to an operator. The computing device is configured to receive at least one stored 3D image of a subject's anatomy and at least one real-time 3D position of at least one surgical instrument, to register the at least one real-time 3D position of the at least one surgical instrument to correspond to the at least one 3D image of the subject's anatomy, and to generate the holographic image comprising the at least one real-time position of the at least one surgical instrument overlaid on the at least one 3D image of the subject's anatomy.

Abstract: A mixed reality (MR) visualization system includes an MR device comprising a holographic display configured to display a holographic image to an operator, a hand-held ultrasound imaging device configured to obtain a real-time ultrasound image of a subject's anatomy, and a computing device communicatively coupled to the MR device and the hand-held ultrasound imaging device. The computing device includes a non-volatile memory and a processor. The computing device is configured to receive the real-time ultrasound image, determine a real-time 3D position and orientation of the hand-held ultrasound imaging device, generate a modified real-time ultrasound image by modifying the real-time ultrasound image to correspond to the real-time 3D position and orientation of the hand-held ultrasound imaging device, and transmit the modified real-time ultrasound image to the MR device for display as the holographic image positioned at a predetermined location relative to the hand-held ultrasound imaging device.

Abstract: Systems and methods for virtual reality or augmented reality (VR/AR) visualization of 3D medical images using a VR/AR visualization system are disclosed that includes a computing device operatively coupled to a VR/AR device, which includes a holographic display and at least one sensor. The holographic display is configured to display a holographic image to an operator. The computing device is configured to receive at least one stored 3D image of a subject's anatomy and at least one real-time 3D position of at least one surgical instrument, to register the at least one real-time 3D position of the at least one surgical instrument to correspond to the at least one 3D image of the subject's anatomy, and to generate the holographic image comprising the at least one real-time position of the at least one surgical instrument overlaid on the at least one 3D image of the subject's anatomy.

Abstract: Systems and methods for virtual reality or augmented reality (VR/AR) visualization of 3D medical images using a VR/AR visualization system are disclosed that includes a computing device operatively coupled to a VR/AR device, which includes a holographic display and at least one sensor. The holographic display is configured to display a holographic image to an operator. The computing device is configured to receive at least one stored 3D image of a subject's anatomy and at least one real-time 3D position of at least one surgical instrument, to register the at least one real-time 3D position of the at least one surgical instrument to correspond to the at least one 3D image of the subject's anatomy, and to generate the holographic image comprising the at least one real-time position of the at least one surgical instrument overlaid on the at least one 3D image of the subject's anatomy.

Abstract: Systems and methods for virtual reality or augmented reality (VR/AR) visualization of 3D medical images using a VR/AR visualization system are disclosed. The VR/AR visualization system includes a computing device operatively coupled to a VR/AR device, and the VR/AR device includes a holographic display and at least one sensor. The holographic display is configured to display a holographic image to an operator. The computing device is configured to receive at least one stored 3D image of a subject's anatomy and at least one real-time 3D position of at least one surgical instrument. The computing device is further configured to register the at least one real-time 3D position of the at least one surgical instrument to correspond to the at least one 3D image of the subject's anatomy, and to generate the holographic image comprising the at least one real-time position of the at least one surgical instrument overlaid on the at least one 3D image of the subject's anatomy.

Abstract: Systems and methods for virtual reality or augmented reality (VR/AR) visualization of 3D medical images using a VR/AR visualization system are disclosed. The VR/AR visualization system includes a computing device operatively coupled to a VR/AR device, and the VR/AR device includes a holographic display and at least one sensor. The holographic display is configured to display a holographic image to an operator. The computing device is configured to receive at least one stored 3D image of a subject's anatomy and at least one real-time 3D position of at least one surgical instrument. The computing device is further configured to register the at least one real-time 3D position of the at least one surgical instrument to correspond to the at least one 3D image of the subject's anatomy, and to generate the holographic image comprising the at least one real-time position of the at least one surgical instrument overlaid on the at least one 3D image of the subject's anatomy.

Abstract: Approaches discussed herein enable a computing device, such as a phone or tablet computer, to automatically recalibrate cameras used to capture three-dimensional images. A proximity sensor can be used to determine the distance to an object at approximately the time that three-dimensional image data is captured that includes a representation of the object. The apparent distance to the object, from the disparity of the three-dimensional image data, can be compared against the detected distance as measured by the proximity sensor. The difference between the detected and apparent distance, or the corresponding disparity values, can be used to determine a relative misalignment among the cameras, when then can be used to update the appropriate calibration parameters.

Abstract: A computing device can obtain information about how the device is held, moved, and/or used by a hand of a user holding the device. The information can be obtained utilizing one or more sensors of the device independently or working in conjunction. For example, an orientation sensor can determine whether a left hand or a right hand is likely rotating, tilting, and/or moving, and thus holding, the device. In another example, a camera and/or a hover sensor can obtain information about a finger position of the user's hand to determine whether the hand is likely a left hand or a right hand. In a further example, a touch sensor can determine a shape of an imprint of a portion of the user's hand to determine which hand is likely holding the device. Based on which hand is holding the device, the device can improve one or more computing tasks.

Abstract: A computing device can obtain information about how the device is held, moved, and/or used by a hand of a user holding the device. The information can be obtained utilizing one or more sensors of the device independently or working in conjunction. For example, an orientation sensor can determine whether a left hand or a right hand is likely rotating, tilting, and/or moving, and thus holding, the device. In another example, a camera and/or a hover sensor can obtain information about a finger position of the user's hand to determine whether the hand is likely a left hand or a right hand. In a further example, a touch sensor can determine a shape of an imprint of a portion of the user's hand to determine which hand is likely holding the device. Based on which hand is holding the device, the device can improve one or more computing tasks.

Abstract: The amount of power and processing capacity needed to process gesture input for a computing device is reduced by splitting the management of sensors, used to detect the input, among different processing components. A relatively high power processor system can monitor the sensors and determine the presence of gestures when the device is in an active state. When the device, or at least the processor system, enters a sleep mode, sensor management is transferred to a microcontroller connected to at least a portion of the sensors, such that the microcontroller can analyze the sensor data to determine the likelihood of a wake action being performed. In response to detecting a wake action, the microcontroller can contact a component such as a power management component to place the processor system in an active state, and enable the processor system to resume management of the sensors.