Abstract: An illustrative system may be configured to obtain a visual image of an anatomical scene that includes an imaging probe and to augment the visual image with a synthetic element that indicates, within the visual image, an area of the anatomical scene being imaged by the imaging probe. The synthetic element may include spatial markers. The visualization system may be further configured to direct a display device to display the augmented image. The visualization system may be further configured to direct the display device to display, at a position outside of the synthetic element, a probe image captured by the imaging probe.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for generating three-dimensional meshes representing two-dimensional images for editing the two-dimensional images. The disclosed system utilizes a first neural network to determine density values of pixels of a two-dimensional image based on estimated disparity. The disclosed system samples points in the two-dimensional image according to the density values and generates a tessellation based on the sampled points. The disclosed system utilizes a second neural network to estimate camera parameters and modify the three-dimensional mesh based on the estimated camera parameters of the pixels of the two-dimensional image. In one or more additional embodiments, the disclosed system generates a three-dimensional mesh to modify a two-dimensional image according to a displacement input.

Abstract: Systems and methods are provided that automatically determine whether a prospective surgical trajectory will collide with a patient's skull using a 3D representation of the patient's cranial region (including the patient's scalp, skull, and brain) adapted from imaging data of the patient's cranial region (e.g., MRI data or CT data). Taking a particular patient's varied skull thickness into account, examples can determine bone collision during a trajectory planning stage before or after a stereotactic frame is mounted to the patient. Accordingly, examples may preemptively alert a clinician to a potential bone collision before the prospective surgical trajectory is underway/has been executed, thereby reducing the risk of bone collision during the surgical procedure.

Abstract: The present technology generates a simplified version of a complex avatar by capturing images and 3-D volume information of segments of the complex avatar while the complex avatar is rendered. When the complex avatar is requested in an environment in which it is not desirable to display the complex avatar, the captured images and 3-D volume information can be used to provide a simplified version of the avatar. The simplified version of the avatar can have a similar visual appearance but can be easier to render. However, the present technology permits the user with the complex avatar to continue to have approximately the same visual appearance while avoiding the degraded performance on systems not capable of rendering the complex avatar quickly enough.

Abstract: Technologies for transitioning between two-dimensional (2D) and three-dimensional (3D) display views for video conferencing are described herein. Video conferencing applications can have multiple display views for a user participating in a video conference. In certain situations a user may want to transition from a 2D display view of the video conference to a more immersive 3D display view. These transitions can be visually jarring and create an uncomfortable user experience. The transition from a 2D display view to a 3D display view can be improved by executing the transition to a 3D display view by manipulating visual properties of a virtual camera that is employed to generate the display views.

Abstract: When performing tile-based rendering a first, pre-pass operation in which primitives in a sequence of primitives for a tile are processed to determine visibility information for the sequence of primitives, the visibility information being usable to determine whether or not fragments for a primitive in the sequence of primitives should subsequently be processed further for the render output, is performed. Thereafter a second, main pass operation is performed in which the further processing of fragments for primitives that were processed during the first, pre-pass operation is controlled based on the determined visibility information for the sequence of primitives, such that for fragments for which the visibility information indicates that the fragments should not be processed further for the render output some or all of the processing during the second, main pass is omitted. The visibility information indicates which primitives should be rendered for which sampling positions of the render output.

Abstract: A method of reality augmentation, including: (a) determining the identity of a container, for example chocolate spread, with a top opening; (b) acquiring an image of the container from a top thereof; (c) estimating a geometry of the filling of said container based on said identity and said image; and (d) overlaying an augmentation, for example, a coupon or a toy, on an image, based on said estimation.

Abstract: Examples of the present disclosure describe systems and methods for an auto-focus tool for multimodality image review. In aspects, an image review system may provide for the display of a set of medical images representing one or more imaging modalities. The system may also provide an auto-focus tool that may be used during the review of the set of medical images. After receiving an instruction to activate the auto-focus tool, the system may receive a selection of an ROI in at least one of the images in the set of medical images. The auto-focus tool may identify the location of the ROI within the image and use the identified ROI location to identify a corresponding area or ROI in the remaining set of medical images. The auto-focus tool may orient and display the remaining set of medical images such that the identified area or ROI is prominently displayed.

Abstract: Methods and devices are provided to encode and decode a data stream carrying data representative of a three-dimensional scene, the data stream comprising color pictures packed in a color image; depth pictures packed in a depth image; and a set of patch data items comprising de-projection data; data for retrieving a color picture in the color image and geometry data. Two types of geometry data are possible. The first type of data describes how to retrieve a depth picture in the depth image. The second type of data comprises an identifier of a parametric function and a list of parameter values for the identified parametric function.

Abstract: A Virtual reality system which comprises a head mounted display and positional tracking to determine the position and orientation of the head mounted display. A player wearing the head mounted display would view a virtual world. External physical objects such as a cup can be identified and displayed inside the virtual world displayed inside the head mounted display so that a player can drink out of the cup without having to remove the head mounted display.

Abstract: Video or graphics, received by a render engine within a graphics processing unit, may be segmented into a region of interest such as foreground and a region of less interest such as background. In other embodiments, an object of interest may be segmented from the rest of the depiction in a case of a video game or graphics processing workload. Each of the segmented portions of a frame may themselves make up a separate surface which is sent separately from the render engine to the display engine of a graphics processing unit. In one embodiment, the display engine combines the two surfaces and sends them over a display link to a display panel. The display controller in the display panel displays the combined frame. The combined frame is stored in a buffer and refreshed periodically.

Abstract: Systems and techniques are described for Light Detection and Ranging (LiDAR) noise modeling using machine learning (ML). An example method can include collecting, using one or more sensors, a first set of data for a simulation environment and generating, using the first set of data, a point cloud that represents and/or describes the simulation environment. The method can further include collecting, using the one or more sensors, a second set of data for a real-world environment and generating a noise model using the second set of data and a neural network. The method can also include generating, using the noise model and the point cloud, a noisy point cloud that represents and/or describes the real-world environment.

Abstract: A computer-implemented inverse rendering method comprising: computing an adjoint image by differentiating an objective function that evaluates the quality of a rendered image, image elements of the adjoint image encoding the sensitivity of spatially corresponding image elements of the rendered image with respect to the objective function; sampling the adjoint image at a respective sample position to determine a respective adjoint radiance value associated with the respective sample position; emitting the respective adjoint radiance value into a scene model characterised by scene parameters; determining an interaction location of a respective incident adjoint radiance value with a surface and/or volume of the scene model; determining a respective incident radiance value or an approximation thereof at the interaction location; and updating a scene parameter gradient.

Abstract: Embodiments include receiving image data comprising a representation of teeth, wherein the image data comprises a set of pixel locations for the teeth and depth values associated with the pixel locations in the set of pixel locations. The method further includes generating new image based on applying one or more functions to the image data, wherein the one or more functions adjust one or more color channels of the set of pixel locations based at least in part on the depth values.

Abstract: The following generally relates to using Augmented Reality (AR) to enhance the in-vehicle experience. In some examples, AR techniques are applied to provide AR indications of vehicle safety indicia to alert vehicle occupants to information that may not otherwise be perceptible. In other example, AR techniques are applied to provide emergency vehicle warnings to improve the likelihood of safe responses thereto. In yet other examples, AR techniques are applied to generated personalized outdoor displays, for example, to ameliorate conditions that may impair safe operation of a vehicle.

Abstract: A mechanism is described for facilitating dynamic cache allocation in computing devices in computing devices. A method of embodiments, as described herein, includes facilitating monitoring one or more bandwidth consumptions of one or more clients accessing a cache associated with a processor; computing one or more bandwidth requirements of the one or more clients based on the one or more bandwidth consumptions; and allocating one or more portions of the cache to the one or more clients in accordance with the one or more bandwidth requirements.

Abstract: A method includes receiving, from a camera, one or more frames of image data of a scene comprising a background and one or more three-dimensional objects, wherein each frame comprises a raster of pixels of image data; detecting layer information of the scene, wherein the layer information is associated with a depth-based distribution of the pixels in the one or more frames; and determining a multi-layer model for the scene, the multi-layer model comprising a plurality of discrete layers comprising first and second discrete layers, wherein each discrete layer is associated with a unique depth value relative to the camera. The method further includes mapping the pixels to the layers of the plurality of discrete layers; rendering the pixels as a first image of the scene as viewed from a first perspective; and rendering the pixels as a second image of the scene as viewed from a second perspective.

Abstract: Provided is a method for augmented reality-based image translation performed by one or more processors, which includes storing a plurality of frames representing a video captured by a camera, extracting a first frame that satisfies a predetermined criterion from the stored plurality of frames, translating a first language sentence (or group of words) included in the first frame into a second language sentence (or group of words), determining a translation region including the second language sentence (or group of words) included in the first frame, and rendering the translation region in a second frame.

Abstract: In an exemplary process, a set of parameters corresponding to characteristics of a physical setting of a user is obtained. Based on the parameters, at least one display placement value and a fixed boundary location corresponding to the physical setting are obtained. In accordance with a determination that the at least one display placement value satisfies a display placement criterion, a virtual display is displayed at the fixed boundary location corresponding to the physical setting.

Abstract: A process for receiving, from a computing device, a series of captured building images. The process continues by processing, in real-time, each building image in the series of captured building images to determine if each building image meets a minimum criterion, wherein the minimum criteria includes applicability to be used in constructing a specific digital multi-dimensional building model. The process continues by aggregating each image meeting the minimum criteria, determining when a base set of building images has been aggregated, wherein the base set of building images includes a threshold number images to model at least a partial multi-dimensional building model representing the series of captured building images, determining one or more facades present in the partial multi-dimensional building model, determining preliminary dimensions for one or more architectural features of the one or more facades and returning, incrementally (in real-time), the preliminary dimensions to the computing device.