Abstract: A remote device utilizes ray tracing to compute a light field for a scene to be rendered, where the light field includes information about light reflected off surfaces within the scene. This light field is then compressed utilizing one or more video compression techniques that implement temporal reuse, such that only differences between the light field for the scene and a light field for a previous scene are compressed. The compressed light field data is then sent to a client device that decompresses the light field data and uses such data to obtain the light field for the scene at the client device. This light field is then used by the client device to compute global illumination for the scene. The global illumination may be used to accurately render the scene at the mobile device, resulting in a realistic scene that is presented by the mobile device.

Abstract: During the rendering of an image, specific pixels in the image are identified where antialiasing would be helpful. Antialiasing is then performed on these identified pixels, where anti-aliasing is a technique used to add greater realism to a digital image by smoothing jagged edges. This reduces a cost of performing antialiasing by reducing a number of pixels within an image on which antialiasing is performed.

Abstract: Apparatuses, systems, and techniques are presented to estimate user gaze. In at least one embodiment, one or more neural networks are used to determine coarse and fine gaze estimates for one or more users.

Abstract: An augmented reality display system includes a first beam path for a foveal inset image on a holographic optical element, a second beam path for a peripheral display image on the holographic optical element, and pupil position tracking logic that generates control signals to set a position of the foveal inset as perceived through the holographic optical element, to determine the peripheral display image, and to control a moveable stage.

Abstract: A global illumination data structure (e.g., a data structure created to store global illumination information for geometry within a scene to be rendered) is computed for the scene. Additionally, reservoir-based spatiotemporal importance resampling (RESTIR) is used to perform illumination gathering, utilizing the global illumination data structure. The illumination gathering includes identifying light values for points within the scene, where one or more points are selected within the scene based on the light values in order to perform ray tracing during the rendering of the scene.

Abstract: During the rendering of an image, specific pixels in the image are identified where antialiasing would be helpful. Antialiasing is then performed on these identified pixels, where anti-aliasing is a technique used to add greater realism to a digital image by smoothing jagged edges. This reduces a cost of performing antialiasing by reducing a number of pixels within an image on which antialiasing is performed.

Abstract: An augmented reality display system includes a first beam path for a foveal inset image on a holographic optical element, a second beam path for a peripheral display image on the holographic optical element, and pupil position tracking logic that generates control signals to set a position of the foveal inset as perceived through the holographic optical element, to determine the peripheral display image, and to control a moveable stage.

Abstract: Sampling a function is used for many applications, such as rendering images. The challenge is how to select the best samples to minimize computations and produce accurate results. An alternative is to use a larger number of samples that may not be carefully selected in an attempt to increase accuracy. For a function that is an integral, such as functions used to render images, a sample distribution may be computed by inverting the integral. Unfortunately, for many integrals, it is neither easy nor practical to compute the inverted integral. Instead, warp functions may be combined to provide a sample distribution that accurately approximates the factors of the product being integrated. Each warp function approximates an inverted term of the product while accounting for the effects of warp functions approximating other factors in the product. The selected warp functions are customized or “fitted” to implement importance sampling for the approximated product.

Abstract: The disclosure provides features or schemes that improve a user's experience with an interactive computer product by reducing latency through late latching and late warping. The late warping can be applied by imaging hardware based on late latch inputs and is applicable for both local and cloud computing environments. In one aspect, the disclosure provides a method of operating an imaging system employing late latching and late warping. In one example the method of operating an imaging system includes: (1) rendering a rendered image based on a user input from an input device and scene data from an application engine, (2) obtaining a late latch input from the input device, (3) rendering, employing imaging hardware, a warped image by late warping at least a portion of the rendered image based on the late latch input, and (4) updating state information in the application engine with late latch and warp information.

Abstract: A method, computer readable medium, and system are disclosed for redirecting a user's movement through a physical space while the user views a virtual environment. A temporary visual suppression event is detected when a user's eyes move relative to the user's head while viewing a virtual scene displayed on a display device, an orientation of the virtual scene relative to the user is modified to direct the user to physically move along a planned path through a virtual environment corresponding to the virtual scene, and the virtual scene is displayed on the display device according to the modified orientation.

Abstract: Global illumination in computer graphics refers to the modeling of how light is bounced off of one or more surfaces in a computer generated image onto other surfaces in the image (i.e. indirect light), rather than simply determining the light that hits a surface in an image directly from a light source (i.e. direct light). Rendering accurate global illumination effects in such images makes them more believable. However, simulating physically-based global illumination with offline numerical solvers has traditionally been time consuming and/or noisy and has not adapted well for dynamic scenes. The present disclosure provides a probe-based dynamic global illumination technique for computer generated scenes.

Abstract: The present invention facilitates efficient and effective image processing. A network can comprise: a first system configured to perform a first portion of lighting calculations for an image and combing results of the first portion of lighting calculations for the image with results of a second portion of lighting calculations; and a second system configured to perform the second portion of lighting calculations and forward the results of the second portion of the lighting calculations to the first system. The first and second portion of lighting calculations can be associated with indirect lighting calculations and direct lighting calculations respectively. The first system can be a client in a local location and the second system can be a server in a remote location (e.g., a cloud computing environment). The first system and second system can be in a cloud and a video is transmitted to a local system.

Abstract: An augmented reality display system includes a first beam path for a foveal inset image on a holographic optical element, a second beam path for a peripheral display image on the holographic optical element, and pupil position tracking logic that generates control signals to set a position of the foveal inset as perceived through the holographic optical element, to determine the peripheral display image, and to control a moveable stage.

Abstract: The present invention facilitates efficient and effective image processing. A network can comprise: a first system configured to perform a first portion of lighting calculations for an image and combing results of the first portion of lighting calculations for the image with results of a second portion of lighting calculations; and a second system configured to perform the second portion of lighting calculations and forward the results of the second portion of the lighting calculations to the first system. The first and second portion of lighting calculations can be associated with indirect lighting calculations and direct lighting calculations respectively. The first system can be a client in a local location and the second system can be a server in a remote location (e.g., a cloud computing environment). The first system and second system can be in a cloud and a video is transmitted to a local system.

Abstract: Methods are disclosed herein to blur an image to be displayed on a stereo display (such as virtual or augmented reality displays) based on the focus and convergence of the user. The methods approximate the complex effect of chromatic aberration on focus, utilizing three (R/G/B) simple Gaussian blurs. For transparency the methods utilize buffers for levels of blur rather than depth. The methods enable real-time chromatic-based blurring effects for VR/AR displays.

Abstract: A method, computer readable medium, and system are disclosed for redirecting a user's movement through a physical space while the user views a virtual environment. A temporary visual suppression event is detected when a user's eyes move relative to the user's head while viewing a virtual scene displayed on a display device, an orientation of the virtual scene relative to the user is modified to direct the user to physically move along a planned path through a virtual environment corresponding to the virtual scene, and the virtual scene is displayed on the display device according to the modified orientation.

Abstract: Methods are disclosed herein to blur an image to be displayed on a stereo display (such as virtual or augmented reality displays) based on the focus and convergence of the user. The methods approximate the complex effect of chromatic aberration on focus, utilizing three (R/G/B) simple Gaussian blurs. For transparency the methods utilize buffers for levels of blur rather than depth. The methods enable real-time chromatic-based blurring effects for VR/AR displays.

Abstract: A method, computer readable medium, and system are disclosed for redirecting a user's movement through a physical space while the user views a virtual environment. A temporary visual suppression event is detected when a user's eyes move relative to the user's head while viewing a virtual scene displayed on a display device, an orientation of the virtual scene relative to the user is modified to direct the user to physically move along a planned path through a virtual environment corresponding to the virtual scene, and the virtual scene is displayed on the display device according to the modified orientation.

Abstract: A method, computer readable medium, and system are disclosed for computing a path for a user to move along within a physical space while viewing a virtual environment in a virtual reality system. A path for a user to physically move along through a virtual environment is determined based on waypoints and at least one characteristic of the physical environment within which the user is positioned, position data for the user is received indicating whether and how much a current path taken by the user has deviated from the path, and an updated path is computed through the virtual environment based on the waypoints and the at least one characteristic of the physical environment.

Abstract: During the rendering of an image, specific pixels in the image are identified where antialiasing would be helpful. Antialiasing is then performed on these identified pixels, where anti-aliasing is a technique used to add greater realism to a digital image by smoothing jagged edges. This reduces a cost of performing antialiasing by reducing a number of pixels within an image on which antialiasing is performed.