Abstract: Described herein are embodiments of methods and apparatuses for a wearable augmented reality system for fitness. The embodiments may include producing an avatar in the field of vision of the wearer, wherein a perceived position of the avatar may be controlled to aid the wearer in fitness training by establishing a controllable pace for the wearer. The embodiments may further include an optical system that produces the avatar such that it may be perceived by the wearer of the wearable augmented reality apparatus as a virtual running partner.

Abstract: Systems, methods and techniques for automatically recognizing two or 3-dimensional real world objects with an augmented reality display device (smartphone or glasses etc.), and augmenting or enhancing the display of such real world objects by superimposing virtual images such as a still or video advertisement, an opportunity to buy, a story or other virtual image presentation. In non-limiting embodiments, the real world object includes visible features including visible security features and a recognition process takes the visible security features into account when recognizing the object and/or displaying superimposed virtual images.

Abstract: A display control device includes a controller configured to simultaneously displays a bird's-eye view image of a vehicle viewed from above, a video image behind the vehicle captured by an imager, and guide images extending rearward at intervals in the vehicle width direction of the vehicle on one screen of a display. The controller is configured to display on the display a first image in the lower right or lower left of the bird's-eye view image according to the position of an obstacle located in a rear right or rear left of the vehicle and to display a second image outside the guide image at the right end or left end of the video image.

Abstract: Systems, methods, and instrumentalities are described herein for constructing a multi-view patient model (e.g., a 3D human mesh model) based on multiple single-view models of the patient. Each of the single-view models may be generated based on images captured by a sensing device and, dependent on the field of the view of the sensing device, may depict some keypoints of the patient's body with a higher accuracy and other keypoints of the patient's body with a lower accuracy. The multi-view patient model may be constructed using respective portions of the single-view models that correspond to accurately depicted keypoints. This way, a comprehensive and accurate depiction of the patient's body shape and pose may be obtained via the multi-view model even if some keypoints of the patient's body are blocked from a specific sensing device.

Abstract: Various embodiments disclosed herein relate to improved systems and methods for performing computer-aided surgical navigation using augmented reality. A tracking device having one or more optically detectable points is mounted to a patient via surgical hardware. The detectable points are identified by a sensor. One or more augmented reality marker assemblies that each have an augmented reality fiducial marker or symbol are also attached to the tracking device. A computer system generates augmented reality information and transmits it to a display screen, such as a near-eye augmented reality device, for display thereon. The near-eye augmented reality device also has an image capture device capable of capturing the augmented reality symbols, and thereby determining a three-dimensional orientation associated with the tracking device, augmented reality markers, and a user wearing the near-eye reality device.

Abstract: Provided is a method that encompasses the determination of a virtual trajectory within a virtual representation of a patient's anatomy, wherein the trajectory is defined by a user's visual axis relative to the three-dimensional virtual representation of the patient's anatomy. The method includes acquiring image data which describes the three-dimensional virtual representation of the patient's body part, acquiring position data which describes a spatial position of a user's visual axis within a virtual-world co-ordinate system, determining visualization data based on the image data and the position data, and determining virtual-world trajectory data based on the position data.

Abstract: Methods, apparatuses, and systems for performing custom surgical implant design are disclosed. The disclosed apparatus designs and installs customized surgical implants to minimize the recovery of the patient. The system uses imaging data to identify the least invasive installation path, and determines the maximum dimensions of the surgical implant components. The system virtually segments the surgical implant into subcomponents, and modifies the subcomponents such that they can be inserted following the identified least invasive path.

Abstract: A computing system (118) includes a computer readable storage medium (122) with computer executable instructions (124), including: a biophysical simulator (126) configured to simulate coronary or carotid flow and pressure effects induced by a cardiac valve device implantation, using cardiac image data and a device model (212). The computing system further includes a processor (120) configured to execute the biophysical simulator to simulate the coronary or carotid flow and the pressure effects induced by the device implantation with the cardiac image data and the device model. The computing system further includes a display configured to display results of the simulation.

Abstract: An ambulation simulating apparatus, including: a user feet interface comprising foot rests; a dynamic inertia mechanism configured to provide an inertial load when a downwardly force is applied due to the foot and a kinematic connection configured to enable transmission of motion from the feet interface to the dynamic inertia and backwards as the dynamic inertia mechanism resists said downwardly applied force by the foot.

Abstract: A surgical visualization system that can include a structured light emitter, a spectral light emitter, an image sensor, and a control circuit is disclosed herein. The structured light emitter can emit a structured pattern of electromagnetic radiation onto an anatomical structure. The spectral light emitter can emit electromagnetic radiation including a plurality of wavelengths. At least one of the wavelengths can penetrate a portion of the anatomical structure and reflect off a subject tissue. The image sensor can detect the structured pattern of electromagnetic radiation reflected off the anatomical structure and the at least one wavelength reflected off the subject tissue. The control circuit can receive signals from the image sensor, construct a model of the anatomical structure, detect a location of the subject tissue, and determine a margin about the subject tissue, based on at least one signal received from the image sensor.

Abstract: A system for dynamic augmented reality imaging of an anatomical site includes at least one image capturing device for capturing first real-time images of the site, at least one display device for displaying first images captured by the device, storage means for storing at least one second anatomical image of the site and an electronic device having a controller configured to receive and process the images. The controller is configured to display an augmented reality image obtained by superimposing images on the first image by identifying common reference elements of the images, and the common reference elements are identified automatically or manually. The second image is in 3D made up of portions of fan or cone beam computed tomography or magnetic resonance images depicting internal structures of the site. The augmented reality image is clipped to the site by the elements, irrespective of the spatial position of the electronic device.

Abstract: A surgical visualization system includes multiple light sources, at least one optical sensor, at least one display device, and a controller. The controller is configured to control at least one light source to illuminate surgical tissue in a first manner, and control the optical sensor to receive an image from the surgical tissue illuminated in the first manner. The controller calculates a three-dimensional image from the tissue illuminated in the first manner. The controller is configured to control at the least one light source to illuminate surgical tissue in a second manner, and control the optical sensor to receive an image from the surgical tissue illuminated in the second manner. The controller calculates one or more subsurface critical anatomic structures from the tissue illuminated in the second manner and combines the display of the three-dimensional image and the anatomic structures on the at least one display device.

Abstract: Aspects of the disclosure relate to a system comprising a head mounted display, an adjusting mechanism, and a connecting mechanism, wherein the head mounted display is configured to be worn on a head of a user under a cover of a surgical helmet so that the display of the head mounted display is adjacent to a transparent portion of the cover, wherein the adjusting mechanism is configured to adjust at least a position of the head mounted display in relationship to the user's eyes, and wherein the connecting mechanism is configured to couple the head mounted display to the surgical helmet. Aspects of the disclosure relate to a system for tracking a head mounted display when used in connection with a surgical helmet and a cover for the surgical helmet.

Abstract: A method and process for providing three dimensional (3-D) virtual image of a patient in metaverse for surgical or other procedures, wherein the avatar of a medical practitioner, imitating the actions of the practitioner performing the procedure, can visually identify the organs and the location of the instruments in real-time inside patient. Such an image reconstruction with spatial coordination provides a usable metaverse implementation with the medical professional's persona as avatar, usable as a training and supportive tool in medical applications. It is a very valuable, especially for the medical and surgical community. The implemented Metaverse provides all details collected and combined from the multiple imaging systems. It is usable as a diagnostic tool, a practice tool a teaching tool and real time direction and feedback tool by the medical community for procedures. The real-time image implemented as Metaverse provides critical visual capabilities during procedures.

Abstract: Various embodiments of a system and associated method for determining a volume of surgical freedom using a plurality of three-dimensional extrema points along a surgical corridor in relation to a structure of interest are disclosed herein.

Abstract: A surgical navigation method includes selecting one or more two-dimensional images from a three-dimensional image. The method further includes adjusting a portion of the two-dimensional images along a viewing direction. The method also includes superimposing the portion of the two-dimensional images along the viewing direction to form a two-dimensional superimposed image. The method further incudes guiding movement of a virtual surgical instrument into the two-dimensional superimposed image.

Abstract: A cross reality system enables any of multiple devices to efficiently access previously stored maps. Both stored maps and tracking maps used by portable devices may have any of multiple types of location metadata associated with them. The location metadata may be used to select a set of candidate maps for operations, such as localization or map merge, that involve finding a match between a location defined by location information from a portable device and any of a number of previously stored maps. The types of location metadata may prioritized for use in selecting the subset. To aid in selection of candidate maps, a universe of stored maps may be indexed based on geo-location information. A cross reality platform may update that index as it interacts with devices that supply geo-location information in connection with location information and may propagate that geo-location information to devices that do not supply it.

Abstract: A method of imaging a surgical site is disclosed. The method comprises obtaining, by a controller of an automated surgical hub system, first image data of a surgical site, controlling, by the controller, at least one illumination source to illuminate a visible surface of the surgical site in a first manner by projecting structured light onto the visible surface, obtaining, by the controller, second image data of the visible surface of the surgical site under illumination in the first manner by the at least one illumination source, calculating, by the controller, a three-dimensional model of the visible surface based on the second image data obtained by the controller, and integrating, by the controller, the three-dimensional model of the visible surface with the first image data of the surgical site. The second image data is based on sensing the structured light projected onto the visible surface.

Abstract: In various example embodiments, a system and method for data mesh visualization are presented. An avatar representation of a user is generated. A user-specified change to the avatar representation is received and the avatar representation is changed based on the user-specified change. A current status of the user is determined, and an updated avatar representation is generated based on the current status of the user. The updated avatar is displayed in association with a communication of the user.

Abstract: An augmented reality-based content authoring tool is presented. A content author arranges machine-recognizable markers in a physical environment. A computing device operating as the authoring tool recognizes the markers and their arrangement based on a captured digital representation of the physical environment. Once recognized, augmented reality primitives corresponding to the markers can be bound together via their primitive interfaces to give rise to a content set. The individual primitives and content set are instantiated based on the nature of the marker's arrangement.