Patents Assigned to NVIDIA Corporation
  • Patent number: 11189075
    Abstract: Methods and systems are described in some examples for changing the traversal of an acceleration data structure in a highly dynamic query-specific manner, with each query specifying test parameters, a test opcode and a mapping of test results to actions. In an example ray tracing implementation, traversal of a bounding volume hierarchy by a ray is performed with the default behavior of the traversal being changed in accordance with results of a test performed using the test opcode and test parameters specified in the ray data structure and another test parameter specified in a node of the bounding volume hierarchy. In an example implementation a traversal coprocessor is configured to perform the traversal of the bounding volume hierarchy.
    Type: Grant
    Filed: June 4, 2020
    Date of Patent: November 30, 2021
    Assignee: NVIDIA CORPORATION
    Inventors: Samuli Laine, Timo Aila, Tero Karras, Gregory Muthler, William P. Newhall, Jr., Ronald C. Babich, Jr., Craig Kolb, Ignacio Llamas, John Burgess
  • Patent number: 11188442
    Abstract: Memory, used by a computer to store data, is generally prone to faults, including permanent faults (i.e. relating to a lifetime of the memory hardware), and also transient faults (i.e. relating to some external cause) which are otherwise known as soft errors. Since soft errors can change the state of the data in the memory and thus cause errors in applications reading and processing the data, there is a desire to characterize the degree of vulnerability of the memory to soft errors. In particular, once the vulnerability for a particular memory to soft errors has been characterized, cost/reliability trade-offs can be determined, or soft error detection mechanisms (e.g. parity) may be selectively employed for the memory. In some cases, memory faults can be diagnosed by redundant execution and a diagnostic coverage may be determined.
    Type: Grant
    Filed: April 15, 2020
    Date of Patent: November 30, 2021
    Assignee: NVIDIA Corporation
    Inventors: Richard Gavin Bramley, Philip Payman Shirvani, Nirmal R. Saxena
  • Patent number: 11182649
    Abstract: Training deep neural networks requires a large amount of labeled training data. Conventionally, labeled training data is generated by gathering real images that are manually labelled which is very time-consuming. Instead of manually labelling a training dataset, domain randomization technique is used generate training data that is automatically labeled. The generated training data may be used to train neural networks for object detection and segmentation (labelling) tasks. In an embodiment, the generated training data includes synthetic input images generated by rendering three-dimensional (3D) objects of interest in a 3D scene. In an embodiment, the generated training data includes synthetic input images generated by rendering 3D objects of interest on a 2D background image. The 3D objects of interest are objects that a neural network is trained to detect and/or label.
    Type: Grant
    Filed: December 11, 2020
    Date of Patent: November 23, 2021
    Assignee: NVIDIA Corporation
    Inventors: Jonathan Tremblay, Aayush Prakash, Mark A. Brophy, Varun Jampani, Cem Anil, Stanley Thomas Birchfield, Thang Hong To, David Jesus Acuna Marrero
  • Patent number: 11182598
    Abstract: The present disclosure provides various approaches for smart area monitoring suitable for parking garages or other areas. These approaches may include ROI-based occupancy detection to determine whether particular parking spots are occupied by leveraging image data from image sensors, such as cameras. These approaches may also include multi-sensor object tracking using multiple sensors that are distributed across an area that leverage both image data and spatial information regarding the area, to provide precise object tracking across the sensors. Further approaches relate to various architectures and configurations for smart area monitoring systems, as well as visualization and processing techniques. For example, as opposed to presenting video of an area captured by cameras, 3D renderings may be generated and played from metadata extracted from sensors around the area.
    Type: Grant
    Filed: March 26, 2019
    Date of Patent: November 23, 2021
    Assignee: NVIDIA Corporation
    Inventors: Parthasarathy Sriram, Ratnesh Kumar, Farzin Aghdasi, Arman Toorians, Milind Naphade, Sujit Biswas, Vinay Kolar, Bhanu Pisupati, Aaron Bartholomew
  • Patent number: 11182309
    Abstract: Fabric Attached Memory (FAM) provides a pool of memory that can be accessed by one or more processors, such as a graphics processing unit(s) (GPU)(s), over a network fabric. In one instance, a technique is disclosed for using imperfect processors as memory controllers to allow memory, which is local to the imperfect processors, to be accessed by other processors as fabric attached memory. In another instance, memory address compaction is used within the fabric elements to fully utilize the available memory space.
    Type: Grant
    Filed: November 4, 2019
    Date of Patent: November 23, 2021
    Assignee: NVIDIA Corporation
    Inventors: John Feehrer, Denis Foley, Mark Hummel, Vyas Venkataraman, Ram Gummadi, Samuel H. Duncan, Glenn Dearth, Brian Kelleher
  • Patent number: 11182916
    Abstract: In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.
    Type: Grant
    Filed: December 27, 2019
    Date of Patent: November 23, 2021
    Assignee: NVIDIA Corporation
    Inventors: Yilin Yang, Bala Siva Sashank Jujjavarapu, Pekka Janis, Zhaoting Ye, Sangmin Oh, Minwoo Park, Daniel Herrera Castro, Tommi Koivisto, David Nister
  • Patent number: 11182884
    Abstract: The various embodiments of the present disclosure are directed towards methods for tone mapping High-Dynamic-Range (HDR) image data, as well as controlling the brightness of the image encoded by HDR the image data and/or the tone-mapped image data. HDR image is captured. A tone mapping function for the HDR image data is generated. To generate the tone mapping function, control points are dynamically determined based on an analysis of the HDR image data. The tone mapping function is fit to the control points. The tone mapping function is a non-linear function, and is described by a curve in a plane. The shape of the curve is constrained by a line generated from a portion of the control points. The tone mapping function is applied to the HDR image data. A color-compression is applied to the tone mapped image data to generate Standard Dynamic Range or Low Dynamic Range image data.
    Type: Grant
    Filed: July 30, 2019
    Date of Patent: November 23, 2021
    Assignee: NVIDIA Corporation
    Inventors: Yining Deng, Eric Dujardin, Hamidreza Mirzaei Domabi, Sung Hyun Hwang
  • Patent number: 11182207
    Abstract: Techniques are disclosed for reducing the latency between the completion of a producer task and the launch of a consumer task dependent on the producer task. Such latency exists when the information needed to launch the consumer task is unavailable when the producer task completes. Thus, various techniques are disclosed, where a task management unit initiates the retrieval of the information needed to launch the consumer task from memory in parallel with the producer task being launched. Because the retrieval of such information is initiated in parallel with the launch of the producer task, the information is often available when the producer task completes, thus allowing for the consumer task to be launched without delay. The disclosed techniques, therefore, enable the latency between completing the producer task and launching the consumer task to be reduced.
    Type: Grant
    Filed: June 24, 2019
    Date of Patent: November 23, 2021
    Assignee: NVIDIA CORPORATION
    Inventors: Gentaro Hirota, Brian Pharris, Jeff Tuckey, Robert Overman, Stephen Jones
  • Patent number: 11176682
    Abstract: In various examples, optical flow estimate (OFE) quality is improved when employing a hint-based algorithm in multi-level hierarchical motion estimation by using different scan orders at different resolution levels. A scan of an image performed with a scan order may initially leverage OFEs from a previous scan of the image, where the previous scan was performed using a different scan order. The OFEs leveraged from the previous scan are more likely to be of high accuracy until sufficient spatial hints are available to the hint-based algorithm for the scan to reduce the impact of potentially lower quality OFEs resulting from the different scan order of the previous scan.
    Type: Grant
    Filed: November 27, 2019
    Date of Patent: November 16, 2021
    Assignee: NVIDIA Corporation
    Inventors: Dong Zhang, Jianjun Chen, Manindra Parhy
  • Patent number: 11176967
    Abstract: In various examples, recordings of gameplay sessions are enhanced by the application of special effects to relatively high(er) and/or low(er) interest durations of the gameplay sessions. Durations of relatively high(er) or low(er) predicted interest in a gameplay session are identified, for instance, based upon level of activity engaged in by a gamer during a particular gameplay session duration. Once identified, different variations of video characteristic(s) are applied to at least a portion of the identified durations for implementation during playback. The recordings may be generated and/or played back in real-time with a live gameplay session, or after completion of the gameplay session. Further, video data of the recordings themselves may be modified to include the special effects and/or indications of the durations and/or variations may be included in metadata and used for playback.
    Type: Grant
    Filed: July 14, 2020
    Date of Patent: November 16, 2021
    Assignee: NVIDIA Corporation
    Inventors: Prabindh Sundareson, Prakash Yadav, Himanshu Bhat
  • Patent number: 11170299
    Abstract: In various examples, a deep neural network (DNN) is trained—using image data alone—to accurately predict distances to objects, obstacles, and/or a detected free-space boundary. The DNN may be trained with ground truth data that is generated using sensor data representative of motion of an ego-vehicle and/or sensor data from any number of depth predicting sensors—such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. The DNN may be trained using two or more loss functions each corresponding to a particular portion of the environment that depth is predicted for, such that—in deployment—more accurate depth estimates for objects, obstacles, and/or the detected free-space boundary are computed by the DNN. In some embodiments, a sampling algorithm may be used to sample depth values corresponding to an input resolution of the DNN from a predicted depth map of the DNN at an output resolution of the DNN.
    Type: Grant
    Filed: March 9, 2020
    Date of Patent: November 9, 2021
    Assignee: NVIDIA CORPORATION
    Inventors: Junghyun Kwon, Yilin Yang, Bala Siva Sashank Jujjavarapu, Zhaoting Ye, Sangmin Oh, Minwoo Park, David Nister
  • Patent number: 11170566
    Abstract: One aspect of the disclosure provides a method for rendering an image. The method includes: placing primitives of the image in a screen space; binning the primitives into tiles of the screen space that the primitives touch; and rasterizing the tiles. The aforementioned rasterizing includes shading a subset of the primitives binned to one of the tiles over multiple passes at multiple shading rates, each of the shading rates is based at least on a frequency at which a color being shaded at each pass changes across the screen space, and the subset of the primitives are cached in an on-chip memory of a processor rendering the image between the passes.
    Type: Grant
    Filed: April 28, 2020
    Date of Patent: November 9, 2021
    Assignee: Nvidia Corporation
    Inventor: Rahul Sathe
  • Patent number: 11169858
    Abstract: The disclosure is directed to a method of operating a game system where virtual machines (VM) supporting the game session can be reused by a second user after a first user ends their game session. In another aspect, the VM can be shut down if the number of VMs exceeds a target number of VMs or if an abnormality is detected in the VM. In another aspect, VMs can be instantiated in order to meet a target number of VMs. In another aspect, a software application is disclosed to execute the methods described herein. In another aspect, a game services system is disclosed that can operate a VM group, maintain a status parameter of the instantiated VMs, and service user requests for game sessions.
    Type: Grant
    Filed: June 15, 2018
    Date of Patent: November 9, 2021
    Assignee: Nvidia Corporation
    Inventor: Andrew Fear
  • Patent number: 11170740
    Abstract: A technique for selecting locations of tear lines when displaying visual content. The technique includes receiving coordinates for one or more portions of a display where a tear is permitted and determining if a frame transition is to occur while rendered content is being scanned out for display within the one or more portions of the display where tear is permitted. If the frame transition is to occur while the scanline for the display is in the one or more portions of the display where tear is permitted, then the technique further includes allowing the frame transition to occur. If the frame transition is to occur while the scanline for the display is not in the one or more portions of the display where tear is permitted, then the technique further includes delaying the frame transition until at least when the scanline for the display is in the one or more portions of the display where tear is permitted.
    Type: Grant
    Filed: July 6, 2018
    Date of Patent: November 9, 2021
    Assignee: NVIDIA Corporation
    Inventors: Radhika Ranjan Soni, Gaurav Singh
  • Patent number: 11170263
    Abstract: A technique utilizing speculative execution and rollback for performing data parallel training of a neural network model is disclosed. Activations for a layer of the neural network model are normalized during a speculative normalization operation using estimated normalization parameters associated with a partial population of a set of training data allocated to a particular processor. Normalization parameters associated with the total population of the set of training data are generated by a distributed reduce operation in parallel with the speculative normalization operation. An optional rollback operation can revert the activations to a pre-normalization state if the estimated normalization parameters for the partial population are subsequently determined to be inaccurate compared to the normalization parameters for the population of the set of training data distributed across a plurality of processors.
    Type: Grant
    Filed: October 31, 2019
    Date of Patent: November 9, 2021
    Assignee: NVIDIA Corporation
    Inventors: Larry Robert Dennison, Benjamin Klenk
  • Patent number: 11170471
    Abstract: A game-agnostic event detector can be used to automatically identify game events. Game-specific configuration data can be used to specify types of pre-processing to be performed on media for a game session, as well as types of detectors to be used to detect events for the game. Event data for detected events can be written to an event log in a form that is both human- and process-readable. The event data can be used for various purposes, such as to generate highlight videos or provide player performance feedback. The event data may be determined based upon output from detectors such as optical character recognition (OCR) engines, and the regions may be upscaled and binarized before OCR processing.
    Type: Grant
    Filed: January 20, 2020
    Date of Patent: November 9, 2021
    Assignee: NVIDIA CORPORATION
    Inventors: Nathan Otterness, Jonathan White, Dave Clark, Jim van Welzen
  • Patent number: 11171798
    Abstract: A network device configured to perform scalable, in-network computations is described. The network device is configured to process pull requests and/or push requests from a plurality of endpoints connected to the network. A collective communication primitive from a particular endpoint can be received at a network device. The collective communication primitive is associated with a multicast region of a shared global address space and is mapped to a plurality of participating endpoints. The network device is configured to perform an in-network computation based on information received from the participating endpoints before forwarding a response to the collective communication primitive back to one or more of the participating endpoints. The endpoints can inject pull requests (e.g., load commands) and/or push requests (e.g., store commands) into the network. A multicast capability enables tasks, such as a reduction operation, to be offloaded to hardware in the network device.
    Type: Grant
    Filed: July 24, 2020
    Date of Patent: November 9, 2021
    Assignee: NVIDIA Corporation
    Inventors: Benjamin Klenk, Nan Jiang, Larry Robert Dennison, Gregory M. Thorson
  • Patent number: 11164360
    Abstract: A hardware-based traversal coprocessor provides acceleration of tree traversal operations searching for intersections between primitives represented in a tree data structure and a ray. The primitives may include opaque and alpha triangles used in generating a virtual scene. The hardware-based traversal coprocessor is configured to determine primitives intersected by the ray, and return intersection information to a streaming multiprocessor for further processing. The hardware-based traversal coprocessor is configured to provide a deterministic result of intersected triangles regardless of the order that the memory subsystem returns triangle range blocks for processing, while opportunistically eliminating alpha intersections that lie further along the length of the ray than closer opaque intersections.
    Type: Grant
    Filed: July 2, 2020
    Date of Patent: November 2, 2021
    Assignee: NVIDIA Corporation
    Inventors: Samuli Laine, Tero Karras, Greg Muthler, William Parsons Newhall, Ronald Charles Babich, Ignacio Llamas, John Burgess
  • Patent number: 11163859
    Abstract: A computer system comprising a processor and a memory for storing instructions, that when executed by the processor performs a copy protection method. The copy protection method comprises executing a software loop of a first software application in a first operating system. A first call is executed in the software loop to a code portion. A decrypted code portion of the first software application is executed in a second operating system in response to the first call. The code portion is decrypted in response to a successful validation of the first software application.
    Type: Grant
    Filed: November 8, 2016
    Date of Patent: November 2, 2021
    Assignee: NVIDIA Corporation
    Inventors: Anthony Michael Tamasi, Timothy Paul Lottes, Bojan Skaljak, Fedor Fomichev, Andrew Leighton Edelsten, Jay Huang, Ashutosh Gajanan Rege, Keith Brian Galocy
  • Patent number: 11165848
    Abstract: A technique for evaluating qualitative streaming experience using session performance metadata is disclosed herein. A pipeline of a streaming service can be adapted to collect metadata, such as timestamps, from various components of the pipeline. The metadata can then be analyzed to calculate an objective quality metric for each streaming session using weighted scores derived from the metadata for a plurality of different components including, but not limited to, stutter, latency, and/or picture quality. The quality metric is designed to have high correlation with subjective measures of quality by users of the streaming service, but provides dense data samples compared to typical sparse responses collected from user feedback (e.g., user surveys). The objective quality metric can be utilized to quickly adjust, either manually or automatically, the streaming service parameters to improve the quality of the streaming service due to changes in, e.g., streaming content.
    Type: Grant
    Filed: August 21, 2020
    Date of Patent: November 2, 2021
    Assignee: NVIDIA Corporation
    Inventor: Alan Daniel Larson