Abstract: A captured image acquisition section 50 acquires from an imaging apparatus 12 data of a polarized image captured of a target object and stores the image data into an image data storage section 52. A region division section 58 in a normal line information acquisition section 54 divides a plane of the image into regions according to a predetermined criterion such as a polarization degree or luminance. A normal line calculation section 60 obtains a distribution of normal lines of each region by applying either a specular reflection model or a diffuse reflection model thereto. An integration section 62 integrates the distributions of normal lines of the regions into a normal line distribution of the entire image. An output data generation section 56 performs information processing using the normal line distribution and outputs the result of the processing.

Abstract: Methods for rendering augmented reality (AR) content are presented. An a priori defined 3D albedo model of an object is leveraged to adjust AR content so that is appears as a natural part of a scene. Disclosed devices recognize a known object having a corresponding albedo model. The devices compare the observed object to the known albedo model to determine a content transformation referred to as an estimated shading (environmental shading) model. The transformation is then applied to the AR content to generate adjusted content, which is then rendered and presented for consumption by a user.

Abstract: A method, apparatus, and non-transitory computer readable medium for video tone mapping. The method includes receiving the video and determining parameters of a tone mapping function defined by a Bezier curve for processing the video. The method also includes generating, by at least one processor, a tone mapped video by applying the tone mapping function to the video using the determined parameters.

Abstract: A method for synthetic data generation and analysis including: determining a set of parameter values; generating a scene based on the parameter values; rendering a synthetic image of the scene; and generating a synthetic dataset including a set of synthetic images.

Abstract: Using computing devices to perform automated operations related to, with respect to a computer model of a house or other building's interior, generating and displaying simulated lighting information in the model based on sunlight or other external light that is estimated to enter the building and be visible in particular rooms of the interior under specified conditions, such as using ambient occlusion and light transport matrix calculations. The computer model may be a 3D (three-dimensional) or 2.5D representation that is generated after the house is built and that shows physical components of the actual house's interior (e.g., walls), and may be displayed to a user of a client computing device in a displayed GUI (graphical user interface) via which the user specifies conditions for which the simulated lighting display is generated.

Abstract: Implementations of the subject matter described herein relate to mixed reality rendering of objects. According to the embodiments of the subject matter described herein, while rendering an object, a wearable computing device takes lighting conditions in the real world into account, thereby increasing the reality of the rendered object. In particular, the wearable computing device acquires environment lighting information of an object to be rendered and renders the object to a user based on the environment lighting information. In this way, the object rendered by the wearable computing device can be more real and accurate. The user will thus have a better interaction experience.

Abstract: In a ray tracer, a cache for streaming workloads groups ray requests for coherent successive bounding volume hierarchy traversal operations by sending common data down an attached data path to all ray requests in the group at the same time or about the same time. Grouping the requests provides good performance with a smaller number of cache lines.

Abstract: An information processing apparatus includes a display unit configured to display a captured image and an object representing a state of a virtual light source, a change unit configured to change the object and change the state of the virtual light source based on an elapsed time, and an adding unit configured to add a lighting effect to the captured image, the lighting effect being provided by the virtual light source the state of which has been changed by the change unit.

Abstract: A method to view inside three-dimensional volumetric build data takes in each pixel of the images and evaluates the density, usually represented by color, of the pixel. This evaluation allows the user to set threshold values and return exact representations of the data presented, instead of a culmination of all data along a ray trace. The user can determine the size and placement of a slice plane in relation to the volumetric data, and the software will remove the data in front of the plane so the user can view the internal structure of the data.

Abstract: In examples, the number of rays used to sample lighting conditions of a light source in a virtual environment with respect to particular locations in the virtual environment may be adapted to scene conditions. An additional ray(s) may be used for locations that tend to be associated with visual artifacts in rendered images. A determination may be made on whether to cast an additional ray(s) to a light source for a location and/or a quantity of rays to cast. To make the determination variables such as visibilities and/or hit distances of ray-traced samples of the light source may be analyzed for related locations in the virtual environment, such as those in a region around the location (e.g., within an N-by-N kernel centered at the location). Factors may include variability in visibilities and/or hit distances, differences between visibilities and/or hit distances relative to the location, and magnitudes of hit distances.

Abstract: An augmented reality (AR) device includes a 3D video camera to capture video images and corresponding depth information, a display device to display the video data, and an AR module to add a virtual 3D model to the displayed video data. A depth mapping module generates a 3D map based on the depth information, a dynamic scene recognition and tracking module processes the video images and the 3D map to detect and track a target object within a field of view of the 3D video camera, and an augmented video rendering module renders an augmented video of the virtual 3D model dynamically interacting with the target object. The augmented video is displayed on the display device in real time. The AR device may further include a context module to select the virtual 3D model based on context data comprising a current location of the augmented reality device.

Abstract: A system and method is provided for creating editable configurations of 3D models and for using a specific type of 2D vector, namely a straight line, to compute and display real world measurements on the 3D model. A 3D model with UV map is created corresponding to a manufacturing pattern and to scale by a Scale Factor. A 2D vector created and overlaid on the UV map of a 3D model, is used to create several virtual mesh groups outlined by the 2D vector, on the underlying 3D model. Two points on a 3D model is selected to create a 2D Vector comprising the points on the UV map corresponding to the selected points on the 3D model. The Path Length of created 2D Vector and the Real Length as Path Length*Scale Factor, are calculated.

Abstract: The orientation of imagery relative to a compass bearing may be determined based on the position of the sun or other information relating to celestial bodies captured in the image.

Abstract: A system including a rendering engine to render a field texture for a field display and a foveal texture for a steerable foveal display and a compositor including a field compositor to generate frames for the field display from the field texture and a foveal compositor to generate frames for the foveal display from the foveal texture. The system further including a composition manager designed to sequence and select what is presented including one or more of data in the field display and the foveal display.

Abstract: Rendering systems that can use combinations of rasterization rendering processes and ray tracing rendering processes are disclosed. In some implementations, these systems perform a rasterization pass to identify visible surfaces of pixels in an image. Some implementations may begin shading processes for visible surfaces, before the geometry is entirely processed, in which rays are emitted. Rays can be culled at various points during processing, based on determining whether the surface from which the ray was emitted is still visible. Rendering systems may implement rendering effects as disclosed.

Abstract: Some implementations of this application are directed to a server system including one or more CPUs, a plurality of GPUs, main dynamic memory storing programs and data for use by the CPUs and/or GPUs during program execution, a static memory pool stored in a non-volatile memory, and a memory controller configured to manage the static memory pool. Each of the GPUs includes a local cache and is configured to access the static memory pool via the memory controller. The server system executes a plurality of gaming sessions for a gaming title in parallel on the one or more CPUs. Each of the plurality of gaming sessions is associated with a static data item stored in the static memory pool, and requires a graphics operation executable by a respective GPU using the static data item.

Abstract: Examples described herein generally relate to generating a visualization of an image. A proprietary structure that specifies ray tracing instructions for generating the image using ray tracing is intercepted from a graphics processing unit (GPU) or a graphics driver. The proprietary structure can be converted, based on assistance information, to a visualization structure for generating the visualization of the image. The visualization of the image can be generated from the visualization structure.

Abstract: Techniques are disclosed relating to rendering graphics objects. In some embodiments, a graphics unit is configured to transform graphics objects from a virtual space into a second space according to different transformation parameters for different portions of the second space. This may result in sampling different portions of the virtual space at different sample rates, which may reduce the number of samples required in various stages of the rendering process. In the disclosed techniques, transformation may occur prior to rasterization and shading, which may further reduce computation and power consumption in a graphics unit, improve image quality as displayed to a user, and/or reduce bandwidth usage or latency of video content on a network. In some embodiments, a transformed image may be viewed through a distortion-compensating lens or resampled prior to display.

Abstract: Disclosed are systems, methods, and non-transitory computer-readable media for automatic image selection for online product catalogs. An image selection system gathers feature data for images of an item included in listings posted to an online marketplace. The image selection system uses the feature data as input in a machine learning model to determine probability scores indicating an estimated probability that each image is suitable to represent the item. The machine learning model is trained based on a set of training images of the item that have been labeled to indicate whether they are suitable to represent the image. The image selection system compares the probability scores and selects an image to represent the item as a stock image based on the comparison.

Abstract: A method for visualizing two-dimensional data with three-dimensional volume enables the end user to easily view abnormalities in sequential data. The two-dimensional data can be in the form of a tiled texture with the images in a set row and column, a media file with the images displayed at certain images in time, or any other way to depict a set of two-dimensional images. The disclosed method takes in each pixel of the images and evaluates the density, usually represented by color, of the pixel. This evaluation allows the user to set threshold values and return accurate representations of the data presented, instead of a culmination of all data along a ray trace.

Abstract: A system and method for facilitating lighting of objects during interactive gameplay by users on client computing platforms distinguishes activities performed prior to interactive gameplay and during interactive gameplay. Different client computing platforms may have different levels of graphics performance. Lighting may be defined by characteristics of one or more light sources that illuminate one or more objects in a multi-dimensional volume in a virtual space. Different lighting techniques or lighting features may be combined to create lighting during interactive gameplay. Some lighting techniques or lighting features may only be available and/or supported on high-performance computing platforms, whereas other lighting features may be available even on low-performance computing platforms.

Abstract: A point-of-interest information acquisition section of an information processing apparatus acquires, by using a polarization image captured from a viewpoint of an imaging apparatus, an incident plane relative to the viewpoint at a point of interest ‘a’ on a subject. A viewpoint control section determines a direction of movement of the imaging apparatus in such a manner as to suitably acquire an angle formed with the incident plane and presents the direction to a user. When the post-movement viewpoint of an imaging apparatus is determined to be appropriate, an incident plane at the point of interest ‘a’ is acquired by using the captured polarization image, and a line of intersection with the incident plane is assumed to be a normal vector n of the point of interest ‘a.

Abstract: A method and apparatus are disclosed for implementing a glow characteristic on part or all of a graphics object within a plurality of different graphics library environments. The glow characteristic is implemented by a client application utilizing a graphics library and drawing to no more than one frame buffer at a time. The same glow characteristic can be rendered on client devices utilizing different graphics libraries.

Abstract: A progressive photon mapping method based on statistical test includes launching rays from the viewpoint to each pixel on the image plane and intersecting the three-dimensional scene to be rendered. If an intersection with diffuse surface is found on the tracing path, it is recorded as the hit point; a photon pass is performed: 31) performing photon tracing step; 32) performing photon collection processing for each hit point; 33) if the current iteration of photon pass does not need chi-square test, then performing flux accumulation and keeping the collection radius unchanged; if chi-square is required, evaluating the photon distribution quality; computing a collection radius according to the estimated photon distribution, and performing the flux accumulation in the current photon pass; 34) if the photon collection radius is reduced, then performing distributed ray tracing, generating new hit points, and go to 31), otherwise go to 31), start a new iteration of photon pass.

Abstract: Embodiments provide for the rendering of illumination effects on real-world objects in augmented reality systems. An example method generally includes overlaying a shader on the augmented reality display. The shader generally corresponds to a three-dimensional geometry of an environment in which the augmented reality display is operating, and the shader generally comprises a plurality of vertices forming a plurality of polygons. A computer-generated lighting source is introduced into the augmented reality display. One or more polygons of the shader are illuminated based on the computer-generated lighting source, thereby illuminating one or more real-world objects in the environment with direct lighting from the computer-generated lighting source and reflected and refracted lighting from surfaces in the environment.

Abstract: An embodiment of a graphics processor pipeline apparatus may include a vertex fetcher to fetch vertices, a vertex shader communicatively coupled to the vertex fetcher to shade the fetched vertices, a primitive assembler communicatively coupled to the vertex shader to assemble primitives, and a primitive replicator communicatively coupled to the primitive assembler to replicate primitives for at least a first and a second viewport.

Abstract: An image processing apparatus comprises: an acquisition unit that acquires a first image obtained through shooting and distance information of the first image; a detection unit that detects a main subject from the first image; an extraction unit that extracts another subject from the first image based on the distance information of the main subject; a setting unit that sets parameters of one or more virtual light sources that emit virtual light to the main subject and the extracted other subject; and a processing unit that generates from the first image a second image in which the main subject and the other subject are illuminated with the virtual light using the parameters set by the setting unit.

Abstract: A method and apparatus provides for compiling a plurality of shaders, each shader having a plurality of computer-readable statements, into a plurality of computer-executable instructions. In one example, the method and apparatus, using a computing device, receives the plurality of shaders used in a process pipeline for performing at least one shading function, determines a shader type of each of the plurality of shaders based on the at least one shading function, and compiles the plurality of shaders by generating the computer-executable instructions using data including a shader descriptor for each of the plurality of shaders, resulting in the shading functions of the plurality of shaders combined together.

Abstract: Examples are described here that can be used to allocate commands from multiple sources to performance by one or more segments of a processing device. For example, a processing device can be segmented into multiple portions and each portion is allocated to process commands from a particular source. In the event a single source provides commands, the entire processing device (all segments) can be allocated to process commands from the single source. When a second source provides commands, some segments can be allocated to perform commands from the first source and other segments can be allocated to perform commands from the second source. Accordingly, commands from multiple applications can be executed by a processing unit at the same time.

Abstract: A method and system for editing a shader. The method comprises providing a graph (122) corresponding to the shader (e.g. a shader graph). The graph (122) comprises a plurality of entities (120), for example shader nodes, which are connected together by one or more edges (124). One or more of the entities each include a respective variable parameter. The method further comprises sending, from an editor entity (130-136), to an entity (120) within the graph (122), a message, the message specifying a change in a value of the variable parameter of that entity (120); responsive to the entity (120) receiving the message, changing the variable parameter of that entity (120), thereby providing a changed graph; and modifying the shader based on the changed graph.

Abstract: A method for visualizing a three-dimensional volume for use in a virtual reality environment is performed by uploading two-dimensional images for evaluation, creating planar depictions of the two-dimensional images, and using thresholds to determine if voxels should be drawn. A voxel volume is created from the planar depictions and voxels. A user defines a plane to be used for slicing the voxel volume, and sets values of the plane location and plane normal. The slice plane is placed within the voxel volume and defines a desired remaining portion of the volumetric plane to be displayed. All but the desired remaining portion of the voxel volume is discarded and the remaining portion is displayed.

Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for rendering makeup products on a user's face within an augmented reality environment in real-time. For example, the disclosed system can use blend a base makeup color of a selected makeup product with extracted luminance frequencies of a video stream. The disclosed system can then convert the blended makeup color to LAB color space and apply one or more shader models to the lightness of the color based on a material of the selected makeup product. The disclosed system can also apply additional operations for smoothing the skin of the user and matching the makeup product to a skin tone of the user. The disclosed system can then display the makeup product and any additional changes in an augmented reality environment in the video stream.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a dynamic mesh for rendering with a graphical user interface. Graphical objects are rendered onto a layer having transparent pixels. The system infers what areas of the layer are drawn areas, and a mesh is generated based on the drawn areas.

Abstract: An exemplary shader construction system receives a shader construction request that indicates a set of selected shader components and a platform indicator. The set of selected shader components is selected from a library of available shader components accessible to the shader construction system and includes first and second shader components configured to perform, respectively, first and second shader functions. The platform indicator is indicative of a selected platform for which a shader is to be constructed. Based on the shader construction request, the shader construction system accesses instructions associated with the first and second shader components. Based on these instructions, the shader construction system assembles a shader that implements the first and second shader functions. The shader is configured for use with the selected platform by a graphics rendering system to render an image. Corresponding methods and systems are also disclosed.

Abstract: A graphics processing apparatus comprising bounding volume hierarchy (BVH) construction circuitry to perform a spatial analysis and temporal analysis related to a plurality of input primitives and responsively generate a BVH comprising spatial, temporal, and spatial-temporal components that are hierarchically arranged, wherein the spatial components include a plurality of spatial nodes with children, the spatial nodes bounding the children using spatial bounds, and the temporal components comprise temporal nodes with children, the temporal nodes bounding their children using temporal bounds and the spatial-temporal components comprise spatial-temporal nodes with children, the spatial-temporal nodes bounding their children using spatial and temporal bounds; and ray traversal/intersection circuitry to traverse a ray or a set of rays through the BVH in accordance with the spatial and temporal components.

Abstract: Cloud-based real time rendering.

Abstract: Ray tracing units, processing modules and methods are described for generating one or more reduced acceleration structures to be used for intersection testing in a ray tracing system for processing a 3D scene. Nodes of the reduced acceleration structure(s) are determined, wherein a reduced acceleration structure represents a subset of the 3D scene. The reduced acceleration structure(s) are stored for use in intersection testing. Since the reduced acceleration structures represent a subset of the scene (rather than the whole scene) the memory usage for storing the acceleration structure is reduced, and the latency in the traversal of the acceleration structure is reduced.

Abstract: An embodiment of a graphics pipeline apparatus may include a vertex shader, a visibility shader communicatively coupled to an output of the vertex shader to construct a hierarchical visibility structure, a tile renderer communicatively coupled to an output of the vertex shader and to the visibility shader to perform a tile-based immediate mode render on the output of the vertex shader based on the hierarchical visibility structure, and a rasterizer communicatively coupled to an output of the tile renderer to rasterize the output of the tile renderer based on the hierarchical visibility structure. Other embodiments are disclosed and claimed.

Abstract: A viewing direction may define an angle/visual portion of a spherical video at which a viewing window is directed. A trajectory of viewing direction may include changes in viewing directions for playback of spherical video. Abrupt changes in the viewing directions may result in jerky or shaky views of the spherical video. Changes in the viewing directions may be stabilized to provide stabilized views of the spherical video. Amount of stabilization may be limited by a margin constraint.

Abstract: Disclosed is a computer-implemented method of determining a hypersurface image from a tomographic image data set describing a tomographic image of an anatomical body part. The method encompasses a locally depth-of-view-corrected reconstruction of a volumetric data set (pre-operative image data, like CT or MRI image data), in order to e.g. augment volumetric image data onto e.g. a microscope view, or in the head-up display of the microscope. For the depth correction, a surface model of the actual anatomical surface of the anatomical body part is used which encompasses a hypersurface reconstruction pf the volumetric data set. Thus, the correct information related to the tissue at the current visible surface is overlaid.

Abstract: A method for generating an augmented set of images involves data collection, data processing, and data augmentation processing performed to merge images. The data collection comprises the steps of choosing objects as selected objects, choosing configurations for imaging of the selected objects, and taking pictures with a second camera to create a set of raw object images. The data processing comprises the steps of performing dynamic range adjustment on the raw object image, performing color correction for corrected images, and removing uniform background from the corrected images to result in object images. The data augmentation processing performed to merge images comprises the steps of performing range simulation or magnification for resampled images, adding blur to the resampled images, adding noise to create final object images, and merging the final object images to the field images of a first camera to create a set of augmented images.

Abstract: The present disclosure concerns a methodology that allows a user to “orbit” around a model on a specific axis of rotation and view an orthographic floor plan of the model. A user may view and “walk through” the model while staying at a specific height above the ground with smooth transitions between orbiting, floor plan, and walking modes.

Abstract: A rendering process and system that may be used to composite virtual objects in panoramic video to provide a virtual reality and augmented reality experience. The process includes receiving low dynamic range (LDR) video data, e.g. 360° video; generating radiance maps, such as diffuse and specular maps, from the LDR data; inverse tone mapping the LDR data of the maps to generate high dynamic range (HDR) data for the maps; and receiving at least one virtual object and applying image based lighting (IBL) to the virtual object using the HDR data of the maps. A perceptually based threshold is also applied to radiance maps to detect prominent pixels, and using the prominent pixels as salient lights for image based shadowing (IBS) associated the virtual object. Objects are composited 360° video in real time using IBL and IBS without precomputation allowing user interaction with the objects.

Abstract: Described herein are techniques for reducing control flow divergence. The method includes identifying two or more shader programs having commonalities, generating a merged shader program that implements functionality of the identified two or more shader programs, wherein the functionality implemented includes a first execution option for a first shader program of the two or more shader programs and a second execution option for a second shader program of the two or more shader programs, modifying shader programs that call the first shader program to instead call the merged shader program and select the first execution option, modifying shader programs that call the second shader program to instead call the merged shader program and select the second execution option.

Abstract: An exemplary lightmap management system is implemented by a multi-access edge compute (“MEC”) server. The system accesses graphics data representative of a 3D model of an object included at a first location within a 3D scene, and identifies a second location from which a light source illuminates the object. The system generates or updates a dynamic lightmap for the object represented by the 3D model based on raytracing of virtual light rays from the second location to the first location. The system provides the dynamic lightmap to a media player device communicatively coupled to the MEC server and configured to apply the dynamic lightmap to the 3D model. For example, the applying of the dynamic lightmap may be performed as part of a rendering of an image that depicts the 3D scene from a viewpoint selected by a user to whom the media player device is presenting the 3D scene.

Abstract: This technology relates to rendering content from discrete applications. In this regard, one or more computing devices may receive a global scene graph containing resources provided by two or more discrete processes, wherein the global scene graph is instantiated by a first process of the two or more discrete processes. The one or more computing devices may render and output for display, the global scene graph in accordance with the resources contained there.

Abstract: The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure can determine at least some shading data for each of a plurality of patches. Further, aspects of the present disclosure can store the at least some shading data for each of the plurality of patches in a GMEM. Additionally, aspects of the present disclosure can communicate the at least some shading data for each of the plurality of patches. In some aspects, the present disclosure can configure the GMEM for storing the at least some shading data for each of a plurality of patches. Aspects of the present disclosure can also calculate when the GMEM has stored a maximum amount of shading data. Moreover, aspects of the present disclosure can divide each of the plurality of patches into one or more sub-patches when the GMEM has stored the maximum amount of shading data.

Abstract: A bounce light map for a scene is determined for use in rendering the scene in a graphics processing system. Initial lighting indications representing lighting within the scene are determined. For a texel position of the bounce light map, the initial lighting indications are sampled using an importance sampling technique to identify positions within the scene. Sampling rays are traced between a position in the scene corresponding to the texel position of the bounce light map and the respective identified positions with the scene. A lighting value is determined for the texel position of the bounce light map using results of the tracing of the sampling rays. By using the importance sampling method described herein, the rays which are traced are more likely to be directed towards more important regions of the scene which contribute more to the lighting of a texel.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for image rendering. One of the methods includes receiving data describing a scene, wherein the scene comprises one or more light sources and one or more objects having different surface optical properties; receiving a request to render an image of the scene using a multiple importance sampling method that combines a plurality of sampling techniques, wherein each sampling technique uses a different probability distribution to sample a respective fraction of total number of samples; modifying a particular one of the probability distributions to reduce a variance of the multiple importance sampling while holding the respective fractions and the other probability distributions fixed; rendering the scene using the multiple importance sampling using the modified particular probability distribution and the other probability distributions; and outputting the rendered scene in response to the request.

Abstract: A natural image with a three-dimensional effect is generated after correcting a high-luminance area of an image. The image processing apparatus of the present invention includes a first acquisition unit configured to acquire normal information corresponding to an image, an estimation unit configured to estimate a real illumination parameter based on a high-luminance area of an object included in the image, a first setting unit configured to set a virtual illumination parameter based on the real illumination parameter, and a lighting processing unit configured to perform lighting processing for the image based on the normal information and the virtual illumination parameter.