Lighting/shading Patents (Class 345/426)
  • Patent number: 11030800
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for image rendering. One of the methods includes receiving data describing a scene, wherein the scene comprises one or more light sources and one or more objects having different surface optical properties; receiving a request to render an image of the scene using a multiple importance sampling method that combines a plurality of sampling techniques, wherein each sampling technique uses a different probability distribution to sample a respective fraction of total number of samples; modifying a particular one of the probability distributions to reduce a variance of the multiple importance sampling while holding the respective fractions and the other probability distributions fixed; rendering the scene using the multiple importance sampling using the modified particular probability distribution and the other probability distributions; and outputting the rendered scene in response to the request.
    Type: Grant
    Filed: November 8, 2019
    Date of Patent: June 8, 2021
    Assignee: Chaos Software Ltd.
    Inventors: Ondr̆ej Karlík, Martin {hacek over (S)}ik, Petr Vévoda, Tomás̆ Sk{hacek over (r)}ivan, Jaroslav Kr̆ivánek
  • Patent number: 11030794
    Abstract: A bounce light map for a scene is determined for use in rendering the scene in a graphics processing system. Initial lighting indications representing lighting within the scene are determined. For a texel position of the bounce light map, the initial lighting indications are sampled using an importance sampling technique to identify positions within the scene. Sampling rays are traced between a position in the scene corresponding to the texel position of the bounce light map and the respective identified positions with the scene. A lighting value is determined for the texel position of the bounce light map using results of the tracing of the sampling rays. By using the importance sampling method described herein, the rays which are traced are more likely to be directed towards more important regions of the scene which contribute more to the lighting of a texel.
    Type: Grant
    Filed: May 1, 2019
    Date of Patent: June 8, 2021
    Assignee: Imagination Technologies Limited
    Inventors: Luke T. Peterson, Justin P. DeCell, Jens Fursund
  • Patent number: 11030799
    Abstract: A natural image with a three-dimensional effect is generated after correcting a high-luminance area of an image. The image processing apparatus of the present invention includes a first acquisition unit configured to acquire normal information corresponding to an image, an estimation unit configured to estimate a real illumination parameter based on a high-luminance area of an object included in the image, a first setting unit configured to set a virtual illumination parameter based on the real illumination parameter, and a lighting processing unit configured to perform lighting processing for the image based on the normal information and the virtual illumination parameter.
    Type: Grant
    Filed: September 25, 2019
    Date of Patent: June 8, 2021
    Assignee: Canon Kabushiki Kaisha
    Inventor: Yuichi Nakada
  • Patent number: 11017590
    Abstract: A lighting visualization system and methods for visualizing lighting scenarios for an object is provided. The system includes a graphic user interface for displaying a rendered image of the object, the rendered image representing a selected lighting scenario for the object. The system includes a control panel for indicating a value of parameters associated with the selected lighting scenario, each parameter being associated with at least one light source. The control panel includes a means for adjusting at least one parameter associated with at least one light source, thereby changing the selected lighting scenario. Upon changing the selected lighting scenario, the rendered image is modified and/or replaced. Each rendered image is rendered using a three-dimensional model of the object, one or more high-quality image of the object being utilized for creating the three-dimensional model.
    Type: Grant
    Filed: August 9, 2018
    Date of Patent: May 25, 2021
    Assignee: DURACOMM CORPORATION
    Inventors: Richard Welnowski, Jay Garcia, Benny Lee
  • Patent number: 11010930
    Abstract: A method for pre-processing point clouds comprising large amounts of point data. The method comprises converting the points' coordinates to Morton indices, sorting the Morton indices and sequentially determining intervals based on predefined criteria, which intervals define the leaf nodes and form the basis and starting point for the generation of a tree index structure comprising the leaf nodes, nodes, branches and nodes connecting the branches. Point data contained within a node or sub-trees of a node are quantizable.
    Type: Grant
    Filed: June 7, 2019
    Date of Patent: May 18, 2021
    Assignee: MY VIRTUAL REALITY SOFTWARE AS
    Inventors: Joris Constant Gaston Verschaeve, Roy Morten Brenna, Jørgen Lind, Rasmus Bording
  • Patent number: 11010963
    Abstract: A water surface mesh is determined for a scene to be rendered. This water surface mesh includes a grouping of geometric shapes such as triangles that represents the surface of the water. This water surface mesh is then used to create a refracted or reflected mesh. The refracted or reflected mesh shows an effect produced by the water surface's refraction or reflection of light. The relationship between the water surface mesh and the refracted or reflected mesh is then used to determine how to illuminate elements within the scene. This eliminates some previously necessary steps during rendering, and enables an accurate depiction of caustics within a scene that can be performed in real-time.
    Type: Grant
    Filed: April 2, 2019
    Date of Patent: May 18, 2021
    Assignee: NVIDIA CORPORATION
    Inventor: Holger Heinrich Gruen
  • Patent number: 11010931
    Abstract: A method of processing point cloud data at a decoder can include receiving three dimensional (3D) coordinates of a set of points of a point cloud including first points and a current point. Each of the first points can be associated with a reconstructed attribute value. A group of neighboring points of the current point can be determined from the first points. A first index is received and the first index indicates a reconstructed attribute value selected from the reconstructed attribute values of the plurality of neighboring points. The reconstructed attributed value indicated by the first index is determined based on a rate-distortion decision function. An attribute distance for each of the neighboring points can be determined based on the reconstructed attribute values of the neighboring points.
    Type: Grant
    Filed: September 4, 2019
    Date of Patent: May 18, 2021
    Assignee: Tencent America LLC
    Inventors: Sehoon Yea, Shan Liu
  • Patent number: 11004254
    Abstract: A ray (e.g., a traced path of light, etc.) is generated from an originating pixel within a scene being rendered. Additionally, one or more shadow map lookups are performed for the originating pixel to estimate an intersection of the ray with alpha-tested geometry within the scene. A shadow map stores the distance of geometry as seen from the point of view of the light, and alpha-tested geometry includes objects within the scene being rendered that have a determined texture and opacity. Further, the one or more shadow map lookups are performed to determine a visibility value for the pixel (e.g., that identifies whether the originating pixel is in a shadow) and a distance value for the pixel (e.g., that identifies how far the pixel is from the light). Further still, the visibility value and the distance value for the pixel are passed to a denoiser.
    Type: Grant
    Filed: July 25, 2019
    Date of Patent: May 11, 2021
    Assignee: NVIDIA CORPORATION
    Inventors: Jon Story, Holger Heinrich Gruen
  • Patent number: 10984584
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for image rendering. One of the methods includes receiving data describing a scene, wherein the scene comprises one or more light sources and one or more objects having different surface optical properties; receiving a request to render an image of the scene using a multiple importance sampling method that combines a plurality of sampling techniques, wherein each sampling technique uses a different probability distribution to sample a respective fraction of total number of samples; modifying a particular one of the probability distributions to reduce a variance of the multiple importance sampling while holding the respective fractions and the other probability distributions fixed; rendering the scene using the multiple importance sampling using the modified particular probability distribution and the other probability distributions; and outputting the rendered scene in response to the request.
    Type: Grant
    Filed: November 8, 2019
    Date of Patent: April 20, 2021
    Assignee: Chaos Software Ltd.
    Inventors: Ondr̆ej Karlík, Martin {hacek over (S)}ik, Petr Vévoda, Tomás̆ Sk{hacek over (r)}ivan, Jaroslav Kr̆ivánek
  • Patent number: 10970802
    Abstract: An imaging device includes: an image sensor that images an object to be imaged through polarizing plates arranged to have a different polarization direction for each pixel in a pixel group that includes a plurality of pixels corresponding to each of points of the object to be imaged; a pixel selecting unit that selects a pixel having a lowest brightness for each of the pixel group corresponding to each of the points; and an image output unit that outputs a captured image of the object to be imaged that is generated from pixels selected by the pixel selecting unit.
    Type: Grant
    Filed: August 1, 2018
    Date of Patent: April 6, 2021
    Assignee: FUJITSU FRONTECH LIMITED
    Inventors: Kozo Yamazaki, Isao Iwaguchi
  • Patent number: 10964090
    Abstract: Methods and ray tracing units are provided for performing intersection testing for use in rendering an image of a 3-D scene. A hierarchical acceleration structure may be traversed by: traversing one or more upper levels of nodes of the hierarchical acceleration structure according to a first traversal technique, the first traversal technique being a depth-first traversal technique; and traversing one or more lower levels of nodes of the hierarchical acceleration structure according to a second traversal technique, the second traversal technique not being a depth-first traversal technique. Results of traversing the hierarchical acceleration structure are used for rendering the image of the 3-D scene. The upper levels of the acceleration structure may be defined according to a spatial subdivision structure, whereas the lower levels of the acceleration structure may be defined according to a bounding volume structure.
    Type: Grant
    Filed: July 9, 2018
    Date of Patent: March 30, 2021
    Assignee: Imagination Technologies Limited
    Inventors: Gregory Clark, John W. Howson, Justin DeCell, Steven J. Clohset
  • Patent number: 10957098
    Abstract: For three-dimensional rendering, a machine-learnt model is trained to generate representation vectors for rendered images formed with different rendering parameter settings. The distances between representation vectors of the images to a reference are used to select the rendered image and corresponding rendering parameters that provides a consistency with the reference. In an additional or different embodiment, optimized pseudo-random sequences are used for physically-based rendering. The random number generator seed is selected to improve the convergence speed of the renderer and to provide higher quality images, such as providing images more rapidly for training compared to using non-optimized seed selection.
    Type: Grant
    Filed: February 13, 2020
    Date of Patent: March 23, 2021
    Assignee: Siemens Healthcare GmbH
    Inventors: Kaloian Petkov, Chen Liu, Shun Miao, Sandra Sudarsky, Daphne Yu, Tommaso Mansi
  • Patent number: 10950039
    Abstract: Provided is an image processing apparatus that: acquires a depth map that includes information that indicates a distance up to a subject in an actual space, the depth map including, for each of one or a plurality of areas in the depth map, information regarding the distance up to a subject portion that appears in the area and regarding a color component of the subject portion; and generates a composite image in which a virtual object is arranged in a scene image that represents a scene of the actual space. The image processing apparatus determines a display color of the virtual object on the basis of the distance up to the subject portion that appears in the depth map and the color component thereof.
    Type: Grant
    Filed: June 7, 2017
    Date of Patent: March 16, 2021
    Assignee: Sony Interactive Entertainment Inc.
    Inventor: Shinya Wada
  • Patent number: 10950034
    Abstract: In one embodiment for generating passthrough, a computing system may compute, based on an image of a physical environment, depth measurements of at least one physical object. The system may generate a first model of the physical object using the depth measurements. The system may identify first pixels in the image that depict the physical object and associate them with a first representative depth value computed using the first model. The system may determine, for a pixel of an output image, that a portion of the first model and a portion of a second model of a virtual object are visible. The system may determine that the portion of the first model is associated with the plurality of first pixels and determine occlusion at the pixel based on a comparison between the first representative depth value and a depth value associated with the portion of the second model.
    Type: Grant
    Filed: January 27, 2020
    Date of Patent: March 16, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Alberto Garcia Garcia, Gioacchino Noris, Gian Diego Tipaldi
  • Patent number: 10949956
    Abstract: Systems and methods for enhanced specular reflections are provided. An example method may include determining a first portion of a specular reflection associated with a computer-generated object based on a first contribution from an environment map component at a shading point of the computer-generated object and determining a second portion of the specular reflection associated with the computer-generated object based on a second contribution from a camera feed component at an intersection point of a camera feed and a reflection vector associated with the environment map component. The example method may further include determining the specular reflection, at the shading point, associated with the computer-generated object based on a blending of the first and second portions of the specular reflection.
    Type: Grant
    Filed: April 3, 2019
    Date of Patent: March 16, 2021
    Assignee: GOOGLE LLC
    Inventors: Ivan Neulander, Jeremy Chernobieff
  • Patent number: 10952302
    Abstract: In an example, an expected sky condition is calculated for a geographic location, a time of day, and a date based on a mathematical model. A predicted distribution of direct and interreflected solar radiation within the environment is calculated based on the expected sky condition. Measurement data from one or more photosensors is obtained that provides measurements of an initial distribution of direct and interreflected radiation within the environment, including radiation from solar and electrical lighting sources. A target distribution of direct and interreflected artificial electromagnetic radiation produced by electrical lighting is determined, based on the measurement data and the predicted distribution of direct and interreflected solar radiation, to achieve the target distribution of direct and interreflected radiation within the environment.
    Type: Grant
    Filed: March 15, 2019
    Date of Patent: March 16, 2021
    Inventors: Ian Ashdown, Wallace Jay Scott
  • Patent number: 10950030
    Abstract: The present invention teaches a real-time hybrid ray tracing method for non-planar specular reflections. The non-planar surface is handled by multiple small planar surfaces. Advantage is taken of the planar nature of triangles that comprise building blocks of a non-planar surface. All secondary rays bouncing from a given surface triangle toward object triangles keep a close direction to each other. A collective control of secondary rays is enabled by this closeness and by decoupling secondary rays from primary rays. The result is high coherence of secondary rays.
    Type: Grant
    Filed: June 23, 2020
    Date of Patent: March 16, 2021
    Assignee: ADSHIR LTD.
    Inventors: Reuven Bakalash, Ron Weitzman
  • Patent number: 10943387
    Abstract: This disclosure presents a technique for utilizing ray tracing to produce a high quality visual scene that includes shadows while minimizing computing costs. Since the scene quality and computing cost is directly proportional to the number of rays used, this technique can lower the number of rays needed for shadow region rendering while maintaining a targeted visual quality for the scene. The process includes generating a complex pixel mask based on depth boundary testing, and generating a penumbra mask based on the shadow regions. These masks can use distance/depth data to cull certain pixels from their respective analysis to reduce processing time. A penumbra area can then be denoised using the two masks and the distance/depth data. Finally, the depth boundary pixel computations, i.e., complex pixels, can be resolved. From these processes, a final shadow mask can be generated and sent to the rendering process to complete the scene rendering.
    Type: Grant
    Filed: May 28, 2019
    Date of Patent: March 9, 2021
    Assignee: Nvidia Corporation
    Inventor: Jon Story
  • Patent number: 10943385
    Abstract: When a renderer of a graphics processor is to perform a graphics rendering operation that uses one or more texture layers in accordance with contribution control data that controls the contribution that each texture layer makes to the rendering operation for a group of fragments, the renderer determines contribution control data for each fragment in the group, and based on the determined contribution control data, either: fetches and uses the texture data values for a texture layer for each fragment in the group from memory, or does not fetch texture data values for a texture layer for each fragment in the group from memory and instead uses a dummy value for the texture layer for each fragment in the group for the graphics rendering operation.
    Type: Grant
    Filed: July 5, 2019
    Date of Patent: March 9, 2021
    Assignee: Arm Limited
    Inventors: Peter William Harris, Edvard Fielding
  • Patent number: 10943390
    Abstract: Methods and systems related to dynamic visualization of a representation of a three dimensional object are provided. In some embodiments, a computer system accesses a volumetric dataset representing the object. The computer system applies a set of one or more light rays/vectors to the volumetric data-set representing the object and generates a plurality of intermediate images, each intermediate image of the plurality of intermediate images corresponding to each of the one or more light vectors applied to the volumetric data-set representing an object. The computer system generates an accumulated image based on the plurality of intermediate images. The computer system determines whether a user has manipulated the accumulated image within a predetermined amount of time and in accordance with a determination that the user has manipulated the accumulated image within a predetermined amount of time, updates the accumulated image based on a user manipulation.
    Type: Grant
    Filed: November 19, 2018
    Date of Patent: March 9, 2021
    Assignee: FOVIA, INC.
    Inventors: Georgiy Buyanovskiy, Kevin Kreeger
  • Patent number: 10932860
    Abstract: Systems, methods, and media for presenting medical imaging data in an interactive virtual environment reality are provided. In some embodiments, a system comprises a head mounted display (HMD) that: determines that a transparent 3D object overlaps a 3D model of a portion of anatomy based on a medical imaging scan; sets values for pixels corresponding to portions of the 3D model not occluded by the transparent object by performing a shading operation; sets values for pixels corresponding to portions of the 3D model at the boundary of the transparent object to intensity values taken from the medical imaging data; displays the 3D model with an exterior surface shaded to evoke a 3D object, and internal surfaces at the boundaries of the transparent object not shaded to preserve details embedded in the medical imaging data.
    Type: Grant
    Filed: April 30, 2018
    Date of Patent: March 2, 2021
    Assignee: THE BRIGHAM AND WOMEN'S HOSPITAL, INC.
    Inventors: Konstantin Kovtun, Christopher Williams
  • Patent number: 10937245
    Abstract: An augmented reality-based lighting design method includes displaying, by an augmented reality device, a real-time image of a target physical area on a display screen. The method further includes displaying, by the augmented reality device, a lighting fixture 3-D model on the display screen in response to a user input, wherein the lighting fixture 3-D model is overlaid on the real-time image of the target physical area. The method also includes determining, by the augmented reality device, illuminance values for locations in the target physical area, where the illuminance values indicate illuminance levels of a light to be provided by a lighting fixture represented by the lighting fixture 3-D model. The method further includes displaying, by the augmented reality device, the illuminance values on the display screen overlaid on the real-time image of the target physical area.
    Type: Grant
    Filed: November 19, 2018
    Date of Patent: March 2, 2021
    Assignee: SIGNIFY HOLDING B.V.
    Inventors: Nam Chin Cho, Parth Joshi, William Thomas Cook, Debasmita Mukherjee, Aarathi Ramesh, Vinay Mahantgauda Patil
  • Patent number: 10930246
    Abstract: A display device includes a display, a sensor module which senses a color temperature of light incident to the sensor module from an outside of the display device, and a processor electrically connected to the display and the sensor module. The processor operates in a first mode or a second mode or is turned off, depending on a user input, outputs a moving picture to the display in the first mode, outputs a still image to the display if detecting a user, turns off the display if failing to detect the user, in the second mode, adjusts a color temperature of the still image based on the color temperature sensed by using the sensor module in outputting the still image, and outputs the still image having the adjusted color temperature to the display.
    Type: Grant
    Filed: November 27, 2019
    Date of Patent: February 23, 2021
    Assignee: SAMSUNG ELECTRONICS CO, LTD.
    Inventors: Sang Un Yun, Sung Wan Koo, Su Ji Kim, Ha Yeon Kil
  • Patent number: 10926176
    Abstract: A system and method for facilitating moving particles in a virtual space use depth information that reflects distances of objects in a virtual space to either a ground level or a viewpoint. The depth information may be stored in a set of information. The system detects collisions of particles and other objects based on the depth information, on a frame-by-frame basis.
    Type: Grant
    Filed: March 28, 2017
    Date of Patent: February 23, 2021
    Assignee: Kabam, Inc.
    Inventors: Cody Snyder, Russell Warneboldt
  • Patent number: 10930050
    Abstract: Described herein is a technique for performing ray tracing. According to this technique, instead of executing intersection and/or any hit shaders during traversal of an acceleration structure to determine the closest hit for a ray, an acceleration structure is fully traversed in an invocation of a shader program, and the closest intersection with a triangle is recorded in a data structure associated with the material of the triangle. Later, a scheduler launches waves by grouping together multiple data items associated with the same material. The rays processed by that wave are processed with a continuation ray, rather than the full original ray. A continuation ray starts from the previous point of intersection and extends in the direction of the original ray. These steps help counter divergence that would occur if a single shader program that inlined the intersection and any hit shaders were executed.
    Type: Grant
    Filed: December 13, 2018
    Date of Patent: February 23, 2021
    Assignee: Advanced Micro Devices, Inc.
    Inventor: Skyler Jonathon Saleh
  • Patent number: 10929656
    Abstract: In a minimally invasive surgical system, a plurality of video images is acquired. Each image includes a hand pose image. Depth data for the hand pose image is also acquired or synthesized. The hand pose image is segmented from the image using the depth data. The segmented image is combined with an acquired surgical site image using the depth data. The combined image is displayed to a person at a surgeon's console of the minimally invasive surgical system. Processing each of the video images in the plurality video images in this way reproduces the hand gesture overlaid on the video of the surgical site in the display.
    Type: Grant
    Filed: July 31, 2019
    Date of Patent: February 23, 2021
    Assignee: Intuitive Surgical Operations, Inc.
    Inventors: Brandon D. Itkowitz, Simon P. DiMaio, Tao Zhao
  • Patent number: 10918940
    Abstract: Embodiments of the present application disclose a method and device for downloading and reconstructing game data, and relate to the technical field of Internet.
    Type: Grant
    Filed: August 9, 2016
    Date of Patent: February 16, 2021
    Assignee: JINGCAI ONLINE TECHNOLOGY (DALIAN) CO., LTD.
    Inventors: Changsong Jiang, Jianbin Fang, Jiqing Zhang, Ming Yang
  • Patent number: 10922878
    Abstract: Systems and methods for lighting inserted content are provided. For example, the inserted content may include augmented reality content that is inserted into an image of a physical space. An example system and method may include determining a location within an image to insert content. For example, the image may be captured by a camera device. The example system and method may also include identifying a region of the image based on the determined location to insert the content, determining at least one lighting parameter based on the identified region, and rendering the content using the determined at least one lighting parameter.
    Type: Grant
    Filed: October 3, 2018
    Date of Patent: February 16, 2021
    Assignee: GOOGLE LLC
    Inventors: Ivan Neulander, Chloe LeGendre, Paul Debevec
  • Patent number: 10922790
    Abstract: Apparatus and method for denoising of images generated by a rendering engine such as a ray tracing engine. For example, one embodiment of a system or apparatus comprises: A system comprising: a plurality of nodes to perform ray tracing operations; a dispatcher node to dispatch graphics work to the plurality of nodes, each node to perform ray tracing to render a region of an image frame; at least a first node of the plurality comprising: a ray-tracing renderer to perform ray tracing to render a first region of the image frame; and a denoiser to perform denoising of the first region using a combination of data associated with the first region and data associated with a region outside of the first region, at least some of the data associated with the region outside of the first region to be retrieved from at least one other node.
    Type: Grant
    Filed: December 21, 2018
    Date of Patent: February 16, 2021
    Assignee: Intel Corporation
    Inventors: Carson Brownlee, Ingo Wald, Attila Afra, Johannes Guenther, Jefferson Amstutz, Carsten Benthin
  • Patent number: 10916051
    Abstract: Implementations described herein relate to methods, systems, and computer-readable media to relight a video. In some implementations, a computer-implemented method includes receiving a plurality of frames of a video. Each video frame includes depth data and color data for a plurality of pixels. The method further includes segmenting each frame based on the depth data to classify each pixel as a foreground pixel or a background pixel. The method further includes setting depth value of each background pixel to a fixed depth value and applying a Gaussian filter to smooth depth value for the plurality of pixels. The method further includes calculating surface normals based on the depth values of the plurality of pixels. The method further includes rendering a relighted frame by adding a virtual light based on the surface normals and the color data.
    Type: Grant
    Filed: July 8, 2019
    Date of Patent: February 9, 2021
    Assignee: Google LLC
    Inventors: Guangyu Zhou, Dillon Cower
  • Patent number: 10916052
    Abstract: Methods, systems and apparatuses may provide for technology that determines the size of a graphics primitive, renders pixels associated with the graphics primitive on a per tile basis if the size exceeds a threshold, and renders the pixels associated with the graphics primitive in a mesh order if the size does not exceed the threshold. In one example, the technology discards state data associated with the graphics primitive in response to a completion of rendering the pixels associated with the graphics primitive in the mesh order.
    Type: Grant
    Filed: April 26, 2019
    Date of Patent: February 9, 2021
    Assignee: Intel Corporation
    Inventors: Justin DeCell, Saurabh Sharma, Subramaniam Maiyuran, Raghavendra Miyar, Jorge Garcia Pabon
  • Patent number: 10909747
    Abstract: The disclosed computer-implemented method may include (i) associating a planar element within a virtual environment with an alternate virtual environment and (ii) rendering the virtual environment such that (A) the alternate virtual environment is framed within the planar element, (B) at least one rendered element of the alternate virtual environment renders at a distal depth beyond a proximal depth of the planar element within the virtual environment, and (C) altering a viewpoint within the virtual environment relative to a position of the planar element within the virtual environment produces a parallax effect between the virtual environment and the rendered element of the alternate virtual environment due at least in part to the rendered element of the alternate virtual environment being rendered at the distal depth beyond the proximal depth of the planar element within the virtual environment. Various other methods, systems, and computer-readable media are also disclosed.
    Type: Grant
    Filed: March 28, 2018
    Date of Patent: February 2, 2021
    Assignee: Facebook Technologies, LLC
    Inventors: Agatha Yick Jun Yu, Matthew Alan Scott
  • Patent number: 10909423
    Abstract: Data representing a scene is received. The scene includes labeled elements such as walls, a floor, a ceiling, and objects placed at various locations in the scene. The original received scene may be modified in different ways to create new scenes that are based on the original scene. These modifications include adding clutter to the scene, moving one or more elements of the scene, swapping one or more elements of the scene with different labeled elements, changing the size, color, or materials associated with one or more of the elements of the scene, and changing the lighting used in the scene. Each new scene may be used to generate labeled training data for a classifier by placing a virtual sensor (e.g., a camera) in the new scene, and generating sensor output data for the virtual sensor based on its placement in the new scene.
    Type: Grant
    Filed: June 7, 2018
    Date of Patent: February 2, 2021
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Trebor Lee Connell, Emanuel Shalev, Michael John Ebstyne, Don Dongwoo Kim
  • Patent number: 10902665
    Abstract: Images are rendered from deeply learned raytracing parameters. Active learning, via a machine learning (ML) model (e.g., implemented by a deep neural network), is used to automatically determine, infer, and/or predict optimized, or at least somewhat optimized, values for parameters used in raytracing methods. Utilizing deep learning to determine optimized, or at least somewhat optimized, values for raytracing parameters is in contrast to conventional methods, which require users to rely of heuristics for parameter value setting. In various embodiments, one or more parameters regarding the termination and splitting of traced light paths in stochastic-based (e.g., Monte Carlo) raytracing are determined via active learning. In some embodiments, one or more parameters regarding the sampling rate of shadow rays are also determined.
    Type: Grant
    Filed: March 28, 2019
    Date of Patent: January 26, 2021
    Assignee: ADOBE INC.
    Inventors: Xin Sun, Nathan Aaron Carr, Alexandr Kuznetsov
  • Patent number: 10902265
    Abstract: One embodiment provides a method, including: determining, using a processor, a location of an information handling device; providing, using a camera of the information handling device, a camera view encompassing at least a portion of the location, wherein the portion encompassed by the camera view comprises at least one object; accessing a data store comprising depth information for the at least one object in the location; and leveraging the depth information to produce an imaging effect on the at least one object in the camera view of the portion. Other aspects are described and claimed.
    Type: Grant
    Filed: March 27, 2019
    Date of Patent: January 26, 2021
    Assignee: Lenovo (Singapore) Pte. Ltd.
    Inventors: John Weldon Nicholson, Daryl Cromer, Howard Locker
  • Patent number: 10897514
    Abstract: A target data processing method comprises: in response to receiving, from a client, a request for processing target data, obtaining, at a server and from a storage device, the target data, the target data being predetermined to be obtained by the client from the storage device and a first set of operations being predetermined to be performed on the target data; and processing, at the server, the target data by performing the first set of operations on the target data such that at least one dedicated processing resource of the server performs, on the processed target data, a second set of operations associated with the first set of operations. As such, the data moving path may be shortened significantly, the network traffic pressure may be reduced significantly, the efficiency of the target data processing system may be improved, and moreover, the number of dedicated processing resources in the server may be easily scaled.
    Type: Grant
    Filed: May 24, 2019
    Date of Patent: January 19, 2021
    Assignee: EMC IP Holding Company LLC
    Inventors: Junping Zhao, Kun Wang
  • Patent number: 10891709
    Abstract: The present disclosure relates to methods and apparatus of operation of a GPU. The apparatus can determine a plurality of attributes including one or more vertex attributes and one or more instance attributes. The apparatus can also send the plurality of attributes to at least one processing unit. Additionally, the apparatus can store at least one of the plurality of attributes in a buffer of the at least one processing unit. In some aspects, the at least one of the plurality of attributes can include the one or more instance attributes. The apparatus can also retrieve the at least one of the plurality of attributes from the buffer when the at least one of the plurality of attributes is stored in the buffer. Also, the apparatus can store at least some of the plurality of attributes in a cache of the at least one processing unit.
    Type: Grant
    Filed: March 27, 2019
    Date of Patent: January 12, 2021
    Assignee: QUALCOMM Incorporated
    Inventors: Kyusik Chung, Li Shen, SeokHoon Kim
  • Patent number: 10891772
    Abstract: A rendering apparatus includes: a fragmentation unit configured to divide a three-dimensional model into a plurality of fragments; a calculation unit configured to calculate first radiance for each of the plurality of fragments, the first radiance being radiance in a direction from each of the plurality of fragments to every other of the plurality of fragments; and a rendering unit configured to calculate second radiance for each fragment visible from a viewpoint among the plurality of fragments based on the first radiance, the second radiance being radiance in a direction from the fragment to the viewpoint, project each fragment visible having the second radiance on a rendering screen relevant to the viewpoint, and generate a rendered image.
    Type: Grant
    Filed: March 9, 2017
    Date of Patent: January 12, 2021
    Assignee: NEC CORPORATION
    Inventor: Arihiro Yoshida
  • Patent number: 10891775
    Abstract: A method, computer readable medium, and system are disclosed for implementing automatic level-of-detail for physically-based materials. The method includes the steps of identifying a declarative representation of a material to be rendered, creating a reduced complexity declarative representation of the material by applying one or more term rewriting rules to the declarative representation of the material, and returning the reduced complexity declarative representation of the material.
    Type: Grant
    Filed: October 24, 2019
    Date of Patent: January 12, 2021
    Assignee: NVIDIA CORPORATION
    Inventors: Lutz Kettner, Jan Jordan
  • Patent number: 10885699
    Abstract: Systems can identify visible surfaces for pixels in an image (portion) to be rendered. A sampling pattern of ray directions is applied to the pixels, so that the sampling pattern of ray directions repeats, and with respect to any pixel, the same ray direction can be found in the same relative position, with respect to that pixel, as for other pixels. Rays are emitted from visible surfaces in the respective ray direction supplied from the sampling pattern. Ray intersections can cause shaders to execute and contribute results to a sample buffer. With respect to shading of a given pixel, ray results from a selected subset of the pixels are used; the subset is selected by identifying a set of pixels, collectively from which rays were traced for the ray directions in the pattern, and requiring that surfaces from which rays were traced for those pixels satisfy a similarity criteria.
    Type: Grant
    Filed: March 16, 2020
    Date of Patent: January 5, 2021
    Assignee: Imagination Technologies Limited
    Inventors: Gareth Morgan, Luke T. Peterson
  • Patent number: 10885701
    Abstract: An augmented reality (AR) application allows for the specification of various lighting conditions, or lighting states, that can be simulated for an augmented reality view of a scene. This can include, for example, specifying a time of day to cause the view to be rendered with a specific amount of sunlight, or specifying a state of various light sources that can impact the illumination of the scene. A light map and lighting state data can be determined for the scene. A change to the lighting state data can result in a filter being applied to the live view of the scene to cause the overall appearance to correspond to the target lighting state. Augmented reality content displayed with respect to the live view can be rendered to have an appearance that corresponds to the altered live view. AR content can also be used to update the appearance of shadows.
    Type: Grant
    Filed: December 8, 2017
    Date of Patent: January 5, 2021
    Assignee: Amazon Technologies, Inc.
    Inventor: Pratik Patel
  • Patent number: 10867434
    Abstract: Methods and devices for rendering graphics in a computer system include a graphical processing unit (GPU) with a flexible, dynamic, application-directed mechanism for varying the rate at which fragment shading is performed for rendering an image to a display. In particular, the described aspects include determining, at a rasterization stage, map coordinates based on coarse scan converting a primitive of an object, the map coordinates indicating a location on a sampling rate parameter (SRP) map of a fragment within the primitive of the object, and identifying a lookup value for the fragment within the primitive of the object based at least on map coordinates, and calculating a respective fragment variable SRP value for the fragment within the primitive of the object based at least on the lookup value.
    Type: Grant
    Filed: December 30, 2019
    Date of Patent: December 15, 2020
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Ivan Nevraev, Martin J. I. Fuller, Mark S. Grossman, Jason M. Gould
  • Patent number: 10867431
    Abstract: The present disclosure relates to methods and devices for operation of a GPU. The device can determine a first subset of primitives associated with a set of objects within an image. The first subset of primitives can be based on a first viewpoint with respect to the set of objects. The device can also determine, for a second viewpoint with respect to the set of objects, a second subset of primitives excluding the first subset of primitives. In some aspects, the second subset of primitives can have a difference in depth with respect to the first subset of primitives that is less than a threshold depth. Additionally, the device can mark the first subset of primitives and the second subset of primitives as visible. Further, the device can generate graphical content based on the marked first subset of primitives and the marked second subset of primitives.
    Type: Grant
    Filed: December 17, 2018
    Date of Patent: December 15, 2020
    Assignee: QUALCOMM Technologies, Inc.
    Inventors: Mark Dokter, Dieter Schmalstieg, Markus Steinberger
  • Patent number: 10860624
    Abstract: A system that displays geographic data is disclosed. The system obtains polygons that define a set of geographic regions. Then, the system projects rays from endpoints of the line segments that define the polygons onto a reference line to form intersection points. For each interval between pairs of consecutive intersection points on the reference line, the system keeps track of open line segments that project onto the interval. For each data point in a set of data points, the system identifies a relevant interval on the reference line that the data point projects onto, and performs a crossing number operation to identify polygons that the data point falls into, and the system increments a count for each polygon that the data point falls into. Finally, the system displays the set of geographic regions in a manner that indicates a number of data points that fall into each geographic region.
    Type: Grant
    Filed: September 4, 2019
    Date of Patent: December 8, 2020
    Assignee: SPLUNK Inc.
    Inventor: Geoffrey R. Hendrey
  • Patent number: 10861124
    Abstract: Methods, systems, and computer-readable media for rendering graphics using a graphics pipeline for emulating a fixed-function graphics pipeline, including selecting a first pre-compiled shader chunk from a set of pre-compiled shader chunks, selecting at least one second pre-compiled shader chunk from the set of pre-compiled shader chunks, and, concatenating the first shader chunk with the at least one second shader chunk, wherein each shader chunk corresponds to a fixed-function graphics function.
    Type: Grant
    Filed: February 22, 2019
    Date of Patent: December 8, 2020
    Assignee: Channel One Holdings Inc.
    Inventor: Aidan Fabius
  • Patent number: 10861230
    Abstract: A graphics processing pipeline includes three architectural features that allow a fragment shader to efficiently calculate per-sample attribute values using barycentric coordinates and per-vertex attributes. The first feature is barycentric coordinate injection to provide barycentric coordinates to the fragment shader. The second feature is an attribute qualifier that allows an attribute of a graphics primitive to be processed without conventional fixed-function interpolation. The third feature is a direct access path from the fragment shader to triangle data storage hardware resources where vertex attribute data and/or plane equation coefficients are stored. Allowing the fragment shader to calculate per-sample attribute values in this way advantageously increases system flexibility while reducing workload associated with triangle plane equation setup.
    Type: Grant
    Filed: February 6, 2019
    Date of Patent: December 8, 2020
    Assignee: NVIDIA Corporation
    Inventors: David Patrick, Dale L. Kirkland, Henry Packard Moreton, Ziyad Sami Hakura, Yury Uralsky
  • Patent number: 10861219
    Abstract: A system for performing object identification combines pose determination, EO/IR sensor data, and novel computer graphics rendering techniques. A first module extracts the orientation and distance of a target in a truth chip given that the target type is known. A second is a module identifies the vehicle within a truth chip given the known distance and elevation angle from camera to target. Image matching is based on synthetic image and truth chip image comparison, where the synthetic image is rotated and moved through a 3-Dimensional space. To limit the search space, it is assumed that the object is positioned on relatively flat ground and that the camera roll angle stays near zero. This leaves three dimensions of motion (distance, heading, and pitch angle) to define the space in which the synthetic target is moved. A graphical user interface (GUI) front end allows the user to manually adjust the orientation of the target within the synthetic images.
    Type: Grant
    Filed: November 1, 2016
    Date of Patent: December 8, 2020
    Assignee: Cybernet Systems Corp.
    Inventors: Douglas Haanpaa, Charles J. Cohen, Glenn J. Beach, Charles J. Jacobus
  • Patent number: 10853996
    Abstract: Efficiently generating terrain openness involves a digital elevation model comprising a texture representing a first geographic area and at least part of a plurality of mipmap levels representing geographic areas bordering the first geographic area. The texture and mipmap levels include pixels encoding elevation values for locations of geographic areas. For each pixel of the texture, derivatives are determined, as well as an openness factor based at least in part on the elevations at one or more pixels of the mipmap levels. The derivatives and openness factor are added to the texture. A hill shading factor is determined for each pixel based at least in part on the derivatives. An electronic map of the first geographic area is rendered using the openness and hill shading factors of each pixel of the texture. The rendering is sent for display.
    Type: Grant
    Filed: October 4, 2019
    Date of Patent: December 1, 2020
    Assignee: Mapbox, Inc.
    Inventors: Konstantin Friedrich Käfer, Ansis Brammanis
  • Patent number: 10854017
    Abstract: A three-dimensional virtual image display method is provided for a terminal. The method includes obtaining a first model map and a second model map of a three-dimensional virtual image, and determining a target region of the three-dimensional virtual image. The three-dimensional virtual image comprises a first model and a second model, and the target region is a joining region of the first model and the second model. The method also includes setting brightness of each pixel point in the target region in a shader to be a same value, the shader being configured to set illumination environmental data of each pixel point in the three-dimensional virtual image and the illumination environmental data at least comprising the brightness; and rendering the first model map and the second model map through the shader, so that a brightness of each pixel point in the target region displayed after rendering is equal.
    Type: Grant
    Filed: August 15, 2019
    Date of Patent: December 1, 2020
    Assignee: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
    Inventors: Jinhui Guo, Hui Chen, Bin Li
  • Patent number: 10847117
    Abstract: An augmented reality display alternates between displaying color frames and alpha frames to generate composite images for transparency control. A display processor includes two sets of buffers, each set including a color frame buffer and an alpha frame buffer. One set of buffers is used to provide data to the display while new video data is being written into the other set of buffers. The display processor switches which set of buffers is providing data and which set of buffers is having new video data written into them. By switching the alpha and color frame buffers in sets, a pair of color frame buffer and alpha frame buffer are always the current set of buffers to provide data as the output. If a frame drop occurs and the next color frame or alpha frame is not ready, the set of buffers is not switched to avoid disturbing flashing visual artifacts.
    Type: Grant
    Filed: May 13, 2019
    Date of Patent: November 24, 2020
    Assignee: Adobe Inc.
    Inventors: Tenell Glen Rhodes, Jr., Gavin Stuart Peter Miller, Li-Yi Wei, Qi Sun