Abstract: In various examples, object detections of a machine learning model are leveraged to automatically generate new ground truth data for images captured at different perspectives. The machine learning model may generate a prediction of a detected object at the different perspective, and an object tracking algorithm may be used to track the object through other images in a sequence of images where the machine learning model may not have detected the object. New ground truth data may be generated as a result of the object tracking algorithms outputs, and the new ground truth data may be used to retrain or update the machine learning model, train a different machine learning model, or increase the robustness of a ground truth data set that may be used for training machine learning models from various perspectives.
Abstract: A parallel processing unit (PPU) can be divided into partitions. Each partition is configured to operate similarly to how the entire PPU operates. A given partition includes a subset of the computational and memory resources associated with the entire PPU. Software that executes on a CPU partitions the PPU for an admin user. A guest user is assigned to a partition and can perform processing tasks within that partition in isolation from any other guest users assigned to any other partitions. Because the PPU can be divided into isolated partitions, multiple CPU processes can efficiently utilize PPU resources.
Type:
Grant
Filed:
September 5, 2019
Date of Patent:
February 15, 2022
Assignee:
NVIDIA CORPORATION
Inventors:
Jerome F. Duluk, Jr., Gregory Scott Palmer, Jonathon Stuart Ramsey Evans, Shailendra Singh, Samuel H. Duncan, Wishwesh Anil Gandhi, Lacky V. Shah, Eric Rock, Feiqi Su, James Leroy Deming, Alan Menezes, Pranav Vaidya, Praveen Joginipally, Timothy John Purcell, Manas Mandal
Abstract: A generative adversarial neural network (GAN) learns a particular task by being shown many examples. In one scenario, a GAN may be trained to generate new images including specific objects, such as human faces, bicycles, etc. Rather than training a complex GAN having a predetermined topology of features and interconnections between the features to learn the task, the topology of the GAN is modified as the GAN is trained for the task. The topology of the GAN may be simple in the beginning and become more complex as the GAN learns during the training, eventually evolving to match the predetermined topology of the complex GAN. In the beginning the GAN learns large-scale details for the task (bicycles have two wheels) and later, as the GAN becomes more complex, learns smaller details (the wheels have spokes).
Type:
Grant
Filed:
October 10, 2018
Date of Patent:
February 15, 2022
Assignee:
NVIDIA Corporation
Inventors:
Tero Tapani Karras, Timo Oskari Aila, Samuli Matias Laine, Jaakko T. Lehtinen
Abstract: Various techniques for adaptive rendering of images with noise reduction are described. More specifically, the present disclosure relates to approaches for rendering and denoising images—such as ray-traced images—in an iterative process that distributes computational efforts to pixels where denoised output is predicted with higher uncertainty. In some embodiments, an input image may be fed into a deep neural network (DNN) to jointly predict a denoised image and an uncertainty map. The uncertainty map may be used to create a distribution of additional samples (e.g., for one or more samples per pixel on average), and the additional samples may be used with the input image to adaptively render a higher quality image. This process may be repeated in a loop, until some criterion is satisfied, for example, when the denoised image converges to a designated quality, a time or sampling budget is satisfied, or otherwise.
Abstract: Many computing systems process data organized in a matrix format. For example, artificial neural networks (ANNs) perform numerous computations on data organized into matrices using conventional matrix arithmetic operations. One such operation, which is commonly performed, is the transpose operation. Additionally, many such systems need to process many matrices and/or matrices that are large in size. For sparse matrices that hold few significant values and many values that can be ignored, transmitting and processing all the values in such matrices is wasteful. Thus, techniques are introduced for storing a sparse matrix in a compressed format that allows for a matrix transpose operation to be performed on the compressed matrix without having to first decompress the compressed matrix. By utilizing the introduced techniques, more matrix operations can be performed than conventional systems.
Type:
Grant
Filed:
October 19, 2020
Date of Patent:
February 15, 2022
Assignee:
Nvidia Corporation
Inventors:
Jorge Albericio Latorre, Jeff Pool, David Garcia
Abstract: A method, computer readable medium, and system are disclosed for training a neural network model. The method includes the step of selecting an input vector from a set of training data that includes input vectors and sparse target vectors, where each sparse target vector includes target data corresponding to a subset of samples within an output vector of the neural network model. The method also includes the steps of processing the input vector by the neural network model to produce output data for the samples within the output vector and adjusting parameter values of the neural network model to reduce differences between the output vector and the sparse target vector for the subset of the samples.
Type:
Grant
Filed:
January 26, 2018
Date of Patent:
February 8, 2022
Assignee:
NVIDIA Corporation
Inventors:
Carl Jacob Munkberg, Jon Niklas Theodor Hasselgren, Jaakko T. Lehtinen, Timo Oskari Aila
Abstract: The disclosure relates to the transfer of visuals (e.g., window visuals) over virtual frames that may be stored in any number of video frames of one or more video streams. The visuals may be split into two-dimensional (2D) pages of a virtual frame, with each of the 2D pages being a fraction of the size of video frames of the video stream(s). The virtual frame may be encoded to the video frames of the video stream(s) and later reconstructed in accordance with a page table.
Type:
Grant
Filed:
November 27, 2019
Date of Patent:
February 8, 2022
Assignee:
NVIDIA Corporation
Inventors:
Johannes Zimmermann, Andrija Bosnjakovic, Ashley Reid
Abstract: Disclosed approaches provide for interactions of secondary rays of light transport paths in a virtual environment to share lighting contributions when determining lighting conditions for a light transport path. Interactions may be shared based on similarities in characteristics (e.g., hit locations), which may define a region in which interactions may share lighting condition data. The region may correspond to a texel of a texture map and lighting contribution data for interactions may be accumulated to the texel spatially and/or temporally, then used to compute composite lighting contribution data that estimates radiance at an interaction. Approaches are also provided for reprojecting lighting contributions of interactions to pixels to share lighting contribution data from secondary bounces of light transport paths while avoiding potential over blurring.
Abstract: A display controller within a display device includes a serial peripheral interface (SPI) that coordinates the updating of current settings for groups of light-emitting diodes (LEDs). The SPI controller operates in synchrony with a liquid-crystal display (LCD) vertical scan position in order to update the current settings for rows of LEDs in parallel with the updating of nearby rows of LCD pixels. When updating a row of LEDs, the SPI controller executes one or more SPI transactions included in an SPI program to write current settings for multiple LEDs nearly simultaneously. A compiler generates the SPI program based on the topology of LEDs included in the display device.
Abstract: Apparatuses, systems, and techniques to identify a shape or camera pose of a three-dimensional object from a two-dimensional image of the object. In at least one embodiment, objects are identified in an image using one or more neural networks that have been trained on objects of a similar category and a three-dimensional mesh template.
Type:
Grant
Filed:
April 15, 2020
Date of Patent:
February 1, 2022
Assignee:
NVIDIA Corporation
Inventors:
Xueting Li, Sifei Liu, Kihwan Kim, Shalini De Mello, Varun Jampani, Jan Kautz
Abstract: A genetic algorithm is utilized to generate routing candidates to which a reinforcement learning model is applied to correct the design rule constraint violations incrementally. A design rule checker provides feedback on the violations to the reinforcement learning model and the model learns how to fix the violations. A layout device placer based upon a simulated annealing method may also be utilized.
Abstract: The computational scaling challenges of holographic displays are mitigated by techniques for generating holograms that introduce foveation into a wave front recording planes approach to hologram generation. Spatial hashing is applied to organize the points or polygons of a display object into keys and values.
Type:
Application
Filed:
July 23, 2020
Publication date:
January 27, 2022
Applicant:
NVIDIA Corp.
Inventors:
Jui-Hsien Wang, Ward Lopes, Rachel Anastasia Brown, Peter Shirley
Abstract: Warp sharding techniques to switch execution between divergent shards on instructions that trigger a long stall, thereby interleaving execution between diverged threads within a warp instead of across warps. The technique may be applied to mitigate pipeline stalls in applications with low warp occupancy and high divergence. Warp data cache locality may also be improved by concentrating memory accesses within a warp rather than spreading them across warps.
Type:
Application
Filed:
February 24, 2021
Publication date:
January 27, 2022
Applicant:
NVIDIA Corp.
Inventors:
Sana Damani, Mark Stephenson, Ram Rangan, Daniel Robert Johnson, Rishkul Kulkarni
Abstract: Approaches presented herein can reduce temporal lag that may be introduced in a generated image sequence that utilizes temporal accumulation for denoising in dynamic scenes. A fast historical frame can be generated along with a full historical frame generated for a denoising process, with the fast historical frame being accumulated using an exponential moving average with a significantly higher blend weight. This fast history frame can be used to determine a clamping window that can be used to clamp a corresponding full historical value before, or after, reprojection. The fast historical blend weight can be adjusted to control the amount of noise versus temporal lag in an image sequence. In some embodiments, differences between fast and full historical values can also be used to determine an amount of spatial filtering to be applied.
Abstract: Integrated circuits (ICs)—depending on a current workload—may exceed thermal cooling budgets. As a result, ICs often implement thermal sensors to measure temperatures at junctions or hot spots along the IC. Due to a distance between the thermal sensors and the various junctions, a thermal offset may be added to the temperature readings from the thermal sensors to more accurately estimate the temperature at the junctions. To account for different workload distributions—e.g., asymmetric or symmetric—the systems and methods described herein may dynamically adjust the thermal offsets. As a result, the efficiency of the IC may be increased as thermal settings for the IC may take into account the ability of the thermal cooling budget to effectively cool the IC under a current operating condition—thereby reducing premature throttling back or shutting down of power to the IC.
Type:
Grant
Filed:
July 21, 2020
Date of Patent:
January 25, 2022
Assignee:
NVIDIA Corporation
Inventors:
Jun Gu, Tao Li, Chad Plummer, Brian Lawrence Smith
Abstract: Introduced herein is a routing technique that, for example, routes a transaction to a destination port over a network that supports link aggregation and multi-port connection. In one embodiment, two tables that can be searched based on the target and supplemental routing IDs of the transaction are utilized to route the transaction to the proper port of the destination endpoint. In an embodiment, the first table provides a list of available ports at each hop/route point that can route the transaction to the destination endpoint, and the second table provides a supplemental routing ID that can select a specific group of ports from the first table that can correctly route the transaction to the proper port.
Abstract: A content management system may maintain a scene description that represents a 3D virtual environment and a publish/subscribe model in which clients subscribe to content items that correspond to respective portions of the shared scene description. When changes are made to content, the changes may be served to subscribing clients. Rather than transferring entire descriptions of assets to propagate changes, differences between versions of content may be exchanged, which may be used construct updated versions of the content. Portions of scene description may reference other content items and clients may determine whether to request and load these content items for lazy loading. Content items may be identified by Uniform Resource Identifiers (URIs) used to reference the content items. The content management system may maintain states for client connections including for authentication, for the set of subscriptions in the publish/subscribe model, and for their corresponding version identifiers.
Type:
Grant
Filed:
March 22, 2020
Date of Patent:
January 18, 2022
Assignee:
NVIDIA Corporation
Inventors:
Rev Lebaredian, Michael Kass, Brian Harris, Andrey Shulzhenko, Dmitry Duka
Abstract: A gaming cloud gaming system and a method of initiating a gaming session. One embodiment of the gaming cloud gaming system includes a computing system having: (1) an entry point operable to receive a game session request and generate instructions for establishing a connection between a client and a game server, and (2) a dynamically configurable reverse proxy operable to proxy for the game server and configured to employ the instructions to create a route to a randomly selected port on the game server through which the connection is makeable.
Abstract: In various examples, the present disclosure relates to using temporal filters for automated real-time classification. The technology described herein improves the performance of a multiclass classifier that may be used to classify a temporal sequence of input signals—such as input signals representative of video frames. A performance improvement may be achieved, at least in part, by applying a temporal filter to an output of the multiclass classifier. For example, the temporal filter may leverage classifications associated with preceding input signals to improve the final classification given to a subsequent signal. In some embodiments, the temporal filter may also use data from a confusion matrix to correct for the probable occurrence of certain types of classification errors. The temporal filter may be a linear filter, a nonlinear filter, an adaptive filter, and/or a statistical filter.
Abstract: In a self-driving autonomous vehicle, a controller architecture includes multiple processors within the same box. Each processor monitors the others and takes appropriate safe action when needed. Some processors may run dormant or low priority redundant functions that become active when another processor is detected to have failed. The processors are independently powered and independently execute redundant algorithms from sensor data processing to actuation commands using different hardware capabilities (GPUs, processing cores, different input signals, etc.). Intentional hardware and software diversity improves fault tolerance. The resulting fault-tolerant/fail-operational system meets ISO26262 ASIL D specifications based on a single electronic controller unit platform that can be used for self-driving vehicles.
Type:
Grant
Filed:
January 26, 2018
Date of Patent:
January 4, 2022
Assignee:
NVIDIA Corporation
Inventors:
Mohammed Abdulla Yousuf, T. Y. Chan, Ram Ganapathi, Ashok Srinivasan, Mike Truog