Abstract: Systems and methods for determining rendering parameters based on authored snapshots or templates. In one aspect, clinically relevant snapshots of patient medical data are created by experts to support educational or clinical workflows. Alternatively, the snapshots are created by automation processes from AI-based organ and disease segmentations. In another aspect, clinically relevant templates are generated. Rendering parameters are derived from the snapshots or templates, stored, and then applied for either rendering new data or interactive viewing of existing data.

Abstract: For real-time rendering of medical images from volumetric data obtained from a medical scanner, one or more optical properties of the received volumetric data are determined. A light volume associated to a spherical light source is constructed. The light volume comprises a series of consecutive spherical slices through which light propagates while determining a fraction of the light propagating from one spherical slice to a neighboring spherical slice depending on the optical properties. The constructed light volume is sampling with a gradient-free shading that depends on the determined optical properties. At least one medical image is rendered in relation to the received volumetric data based on the sampled light volume.

Abstract: An overlay of an aligned medical image obtained from a medical scanner is rendered with a reference image. The reference image has a reference structure of a body. An intermediate representation of the reference structure is determined. The medical image has structures corresponding to the reference structures, and an intermediate representation of the structures in the medical image is determined. A rendering parameter is optimized by comparing the intermediate representations of the medical image and of the reference image. The medical image is aligned and overlayed with the reference image based on the optimized rendering parameter, and the aligned and overlayed image is rendered.

Abstract: Computer-implemented methods for rendering a volumetric dataset and a surface embedded in the volumetric dataset are described. One method includes performing a volume rendering process to generate a volume rendering of the volume. Based on the volume rendering process, depths of respective locations in the volume rendering are determined, and the depths stored in association with the respective locations. A surface rendering process generates a surface rendering of a surface using the depths and respective locations. The volume rendering and the surface rendering are combined into a combined rendering of the volume and surface. In another method, depths of the surface at respective locations with respect to the volume are determined and the depths are stored in association with the locations. A volume rendering process includes a shading process which uses the depths of the surface and the respective locations.

Abstract: Systems and methods for automatically determining an image quality assessment of a rendered medical image are provided. A rendered medical image is received. One or more measures of interest are extracted from the rendered medical image. An image quality assessment of the rendered medical image is determined using a machine learning based image quality assessment network based on the one or more measures of interest. The image quality assessment of the rendered medical image is output.

Abstract: Optical guidance is provided during a surgical procedure. Data indicative of an anatomical structure in relation to a surgical procedure is received. An overlay image of the anatomical structure is generated from the received data. A background structure serving as a background for the generated overlay image of the anatomical structure is determined. The generated overlay image of the anatomical structure is blended by a depth enhancement algorithm relative to the determined background structure. The blended image of the anatomical structure is overlaid on the determined background structure.

Abstract: A layout of labels for annotating a rendered image is optimized. A rendered image is obtained, and locations of a plurality of regions of interest in the rendered image are determined. Semantic information associated with the plurality of regions of interest is obtained. Based on the semantic information and the locations of the plurality of regions of interest, and, taking into account a visibility of the labels and a further visibility of the regions of interest in the rendered image, the layout of the labels for annotating the plurality of regions of interest in the rendered image is determined.

Abstract: Distance estimation is optimized in virtual or augmented reality. A distance map of a surgical instrument to a region of interest is determined, at least at the beginning and when a position of the surgical instrument has changed. A render-image is rendered based on a medical 3D image and the position of the surgical instrument, at least at the beginning and when the position of the surgical instrument has changed. At least the region of interest and those parts of the surgical instrument positioned in the volume of the render-image are shown in the render-image. Based on the distance map, at least for a predefined area of the region of interest, visible, acoustic, and/or haptic distance-information is added.

Abstract: An automatic light arrangement for medical visualization includes: providing a medical 3D image (D), providing spatial information about a region of interest (R) in this 3D-image (D) and spatial information about a virtual camera (C), determining a plurality of possible arrangements for light sources (L) by using depth information based on the 3D image (DI) together with the spatial information about the region of interest (R) in this image and the spatial information about the virtual camera (C), wherein valid arrangements are those where shadows (S) on the region of interest (R) are below a predefined threshold, and/or wherein the determination or arrangements is based on a number of predefined perceptual metrics applied specifically to the regions of interest, prioritizing the determined arrangements, and choosing the arrangement with the best prioritization.

Abstract: The present embodiment relates to a renderer and an interactive method for image editing of medical 3D anatomical data. The method includes receiving a dataset with volumetric image data, which have been acquired from an image acquisition modality, and providing a signed distance field data structure of the received dataset. Further, editing operations are received from a user interface for editing at least a part of the provided signed distance field data structure. A visualization of the editing operations is calculated and displayed on a display.

Abstract: Computer-implemented methods for rendering a volumetric dataset and a surface embedded in the volumetric dataset are described. One method includes performing a volume rendering process to generate a volume rendering of the volume. Based on the volume rendering process, depths of respective locations in the volume rendering are determined, and the depths stored in association with the respective locations. A surface rendering process generates a surface rendering of a surface using the depths and respective locations. The volume rendering and the surface rendering are combined into a combined rendering of the volume and surface. In another method, depths of the surface at respective locations with respect to the volume are determined and the depths are stored in association with the locations. A volume rendering process includes a shading process which uses the depths of the surface and the respective locations.

Abstract: For three-dimensional rendering, a machine-learnt model is trained to generate representation vectors for rendered images formed with different rendering parameter settings. The distances between representation vectors of the images to a reference are used to select the rendered image and corresponding rendering parameters that provides a consistency with the reference. In an additional or different embodiment, optimized pseudo-random sequences are used for physically-based rendering. The random number generator seed is selected to improve the convergence speed of the renderer and to provide higher quality images, such as providing images more rapidly for training compared to using non-optimized seed selection.

Abstract: Physically-based volume rendering generates a lightfield. The locations of scattering modeled in physically-based rendering are used to assign depths for the lightfield. The previously assigned depths and previously rendered lightfield are used for lightfield rendering, which may be performed more rapidly than the physically-based volume rendering.

Abstract: Systems and methods are provided for generating smooth transitions between volume rendering presets when the volume rendering is used as part of an animation system. A windowing-compensated look-up table interpolation is used to interpolate between adjacent keyframes that include user defined rendering presets. A constrained spline interpolation may be used to prevent overshooting.

Abstract: An embodiment suggests a method for visualizing an image data set, in particular a medical image data set, wherein the visualized data set displays a three dimensional arrangement having at least a first object and a second object. The method includes assigning a first set of parameter to the first object; assigning a second set of parameters to the second object; dividing the medical image data set into a first sub-region and a second sub-region; and providing a visualisation of the three dimensional arrangement by a volume rendering method, in particular by a ray-casting method or a photorealistic volumetric path tracing, the first set of parameter being applied to the first sub-region for visualizing the first object and the second set of parameter being applied to the second sub-region for visualizing the second object.

Abstract: For three-dimensional rendering, a machine-learnt model is trained to generate representation vectors for rendered images formed with different rendering parameter settings. The distances between representation vectors of the images to a reference are used to select the rendered image and corresponding rendering parameters that provides a consistency with the reference. In an additional or different embodiment, optimized pseudo-random sequences are used for physically-based rendering. The random number generator seed is selected to improve the convergence speed of the renderer and to provide higher quality images, such as providing images more rapidly for training compared to using non-optimized seed selection.

Abstract: For interactive rendering in medical imaging, physically-based volume rendering of a volume of a patient may better assist physicians in diagnosis, prognosis, and/or planning. To provide for more rapid interaction, direct volume rendering is used during interaction. The rendering then transitions to physically-based rendering when there is no interaction. For smoothing the transition and/or preserving cohesive perceptual details, images from the different types of rendering may be blended in the transition and/or during interaction.

Abstract: A 2D medical image is colorized. In one approach, a deep-learnt classifier is trained to colorize from color 2D medical images. The color 2D medical images for training are cinematically rendered from slabs to add color. In another approach, a deep machine-learnt generator creates slices as if adjacent to the 2D medical image. The slices and 2D medical image form a slab, which is cinematically rendered to add color. The result is a colorized 2D medical image.

Abstract: For three-dimensional rendering, a machine-learnt model is trained to generate representation vectors for rendered images formed with different rendering parameter settings. The distances between representation vectors of the images to a reference are used to select the rendered image and corresponding rendering parameters that provides a consistency with the reference. In an additional or different embodiment, optimized pseudo-random sequences are used for physically-based rendering. The random number generator seed is selected to improve the convergence speed of the renderer and to provide higher quality images, such as providing images more rapidly for training compared to using non-optimized seed selection.

Abstract: For rendering in medical imaging, a transfer function is determined. A simple approach to setting the transfer function uses a combination of a rendered image and the voxel data, providing a hybrid of both image and data-driven approaches. A region of interest on a rendered image is used to select some of the voxel data. A characteristic of the selected voxel data is used to determine the transfer function for rendering another image. Both the visual aspect of the rendered image and the voxel data from the scan are used to set the transfer function.