Patents Assigned to NVIDIA Corporation
  • Patent number: 11514293
    Abstract: In various examples, historical trajectory information of objects in an environment may be tracked by an ego-vehicle and encoded into a state feature. The encoded state features for each of the objects observed by the ego-vehicle may be used—e.g., by a bi-directional long short-term memory (LSTM) network—to encode a spatial feature. The encoded spatial feature and the encoded state feature for an object may be used to predict lateral and/or longitudinal maneuvers for the object, and the combination of this information may be used to determine future locations of the object. The future locations may be used by the ego-vehicle to determine a path through the environment, or may be used by a simulation system to control virtual objects—according to trajectories determined from the future locations—through a simulation environment.
    Type: Grant
    Filed: September 9, 2019
    Date of Patent: November 29, 2022
    Assignee: NVIDIA Corporation
    Inventors: Ruben Villegas, Alejandro Troccoli, Iuri Frosio, Stephen Tyree, Wonmin Byeon, Jan Kautz
  • Patent number: 11514637
    Abstract: A method, computer readable medium, and system are disclosed for implementing automatic level-of-detail for physically-based materials. The method includes the steps of identifying a declarative representation of a material to be rendered, creating a reduced complexity declarative representation of the material by applying one or more term rewriting rules to the declarative representation of the material, and returning the reduced complexity declarative representation of the material.
    Type: Grant
    Filed: December 22, 2020
    Date of Patent: November 29, 2022
    Assignee: NVIDIA CORPORATION
    Inventors: Lutz Kettner, Jan Jordan
  • Patent number: 11513686
    Abstract: Accesses between a processor and its external memory is reduced when the processor internally maintains a compressed version of values stored in the external memory. The processor can then refer to the compressed version rather than access the external memory. One compression technique involves maintaining a dictionary on the processor mapping portions of a memory to values. When all of the values of a portion of memory are uniform (e.g., the same), the value is stored in the dictionary for that portion of memory. Thereafter, when the processor needs to access that portion of memory, the value is retrieved from the dictionary rather than from external memory. Techniques are disclosed herein to extend, for example, the capabilities of such dictionary-based compression so that the amount of accesses between the processor and its external memory are further reduced.
    Type: Grant
    Filed: May 5, 2020
    Date of Patent: November 29, 2022
    Assignee: NVIDIA Corporation
    Inventors: Ram Rangan, Suryakant Patidar, Praveen Krishnamurthy, Wishwesh Anil Gandhi
  • Patent number: 11513814
    Abstract: Diagnostics and boot up for AV hardware and software of a computer system of an autonomous vehicle may be performed based at least on receiving a shutdown or power off indication, then a computing state of the computer system may be suspended with the computer system entering a low-power mode. The suspended computing state can be rapidly restored without requiring a reboot and diagnostics for key-on. To ensure the integrity of the saved computing state, the computer system may exit the low-power mode, rerun the diagnostics, reload the programs, and then reenter the low-power mode. Restoring the suspended computing state may be triggered by a user inserting an ignition key, pressing a button to turn on the vehicle, opening a door to the vehicle, remotely unlocking the vehicle, remotely starting the vehicle, etc.
    Type: Grant
    Filed: June 28, 2021
    Date of Patent: November 29, 2022
    Assignee: NVIDIA Corporation
    Inventors: Mitchell Darren Luban, Krishna Sitaraman, Bhavesh Parekh, Michael Truog, Hari Krishnan, Karl Friedrich Greb
  • Patent number: 11514682
    Abstract: According to one or more embodiments, operations may comprise obtaining a first point cloud. The operations also comprise performing segmentation of the first point cloud, the segmentation generating one or more clusters of points of the point cloud. The operations also comprise determining, for each respective cluster of the plurality of clusters, a respective geometric feature of a corresponding object that corresponds to the respective cluster. The operations also comprise obtaining a second point cloud. The operations also comprise assigning a plurality of weights that comprises assigning a respective weight to each respective cluster based on the respective geometric feature that corresponds to the respective cluster. The operations also comprise obtaining a second point cloud and aligning the first point cloud with the second point cloud based on the plurality of weights.
    Type: Grant
    Filed: June 24, 2020
    Date of Patent: November 29, 2022
    Assignee: NVIDIA CORPORATION
    Inventor: Derik Schroeter
  • Patent number: 11512964
    Abstract: According to an aspect of an embodiment, operations may comprise obtaining a pose graph that comprises a plurality of nodes. The operations may also comprise dividing the pose graph into a plurality of pose subgraphs, each pose subgraph comprising one or more respective pose subgraph interior nodes and one or more respective pose subgraph boundary nodes. The operations may also comprise generating one or more boundary subgraphs based on the plurality of pose subgraphs, each of the one or more boundary subgraphs comprising one or more respective boundary subgraph boundary nodes and comprising one or more respective boundary subgraph interior nodes. The operations may also comprise obtaining an optimized pose graph by performing a pose graph optimization. The pose graph optimization may comprise performing a pose subgraph optimization of the plurality of pose subgraphs and performing a boundary subgraph optimization of the plurality of boundary subgraphs.
    Type: Grant
    Filed: March 5, 2020
    Date of Patent: November 29, 2022
    Assignee: NVIDIA CORPORATION
    Inventor: Chen Chen
  • Patent number: 11508113
    Abstract: Recurrent blurring may be used to render frames of a virtual environment, where the radius of a filter for a pixel is based on a number of successfully accumulated frames that correspond to that pixel. To account for rejections of accumulated samples for the pixel, ray-traced samples from a lower resolution version of a ray-traced render may be used to increase the effective sample count for the pixel. Parallax may be used to control the accumulation speed along with an angle between a view vector that corresponds to the pixel. A magnitude of one or more dimensions of a filter applied to the pixel may be based on an angle of a view vector that corresponds to the pixel to cause reflections to elongate along an axis under glancing angles. The dimension(s) may be based on a direction of a reflected specular lobe associated with the pixel.
    Type: Grant
    Filed: March 16, 2021
    Date of Patent: November 22, 2022
    Assignee: NVIDIA Corporation
    Inventors: Dmitriy Zhdan, Evgeny Makarov
  • Patent number: 11508112
    Abstract: Techniques are disclosed for improving the throughput of ray intersection or visibility queries performed by a ray tracing hardware accelerator. Throughput is improved, for example, by releasing allocated resources before ray visibility query results are reported by the hardware accelerator. The allocated resources are released when the ray visibility query results can be stored in a compressed format outside of the allocated resources. When reporting the ray visibility query results, the results are reconstructed based on the results stored in the compressed format. The compressed format storage can be used for ray visibility queries that return no intersections or terminate on any hit ray visibility query. One or more individual components of allocated resources can also be independently deallocated based on the type of data to be returned and/or results of the ray visibility query.
    Type: Grant
    Filed: June 18, 2020
    Date of Patent: November 22, 2022
    Assignee: NVIDIA Corporation
    Inventors: Gregory Muthler, John Burgess, Ronald Charles Babich, Jr., William Parsons Newhall, Jr.
  • Patent number: 11507846
    Abstract: Artificial neural networks (ANNs) are computing systems that imitate a human brain by learning to perform tasks by considering examples. By representing an artificial neural network utilizing individual paths each connecting an input of the ANN to an output of the ANN, a complexity of the ANN may be reduced, and the ANN may be trained and implemented in a much faster manner when compared to an implementation using fully connected ANN graphs.
    Type: Grant
    Filed: March 13, 2019
    Date of Patent: November 22, 2022
    Assignee: NVIDIA CORPORATION
    Inventors: Alexander Keller, Gonçalo Felipe Torcato Mordido, Noah Jonathan Gamboa, Matthijs Jules Van Keirsbilck
  • Patent number: 11508049
    Abstract: In various examples, a deep neural network (DNN) is trained for sensor blindness detection using a region and context-based approach. Using sensor data, the DNN may compute locations of blindness or compromised visibility regions as well as associated blindness classifications and/or blindness attributes associated therewith. In addition, the DNN may predict a usability of each instance of the sensor data for performing one or more operations—such as operations associated with semi-autonomous or autonomous driving. The combination of the outputs of the DNN may be used to filter out instances of the sensor data—or to filter out portions of instances of the sensor data determined to be compromised—that may lead to inaccurate or ineffective results for the one or more operations of the system.
    Type: Grant
    Filed: September 13, 2019
    Date of Patent: November 22, 2022
    Assignee: NVIDIA Corporation
    Inventors: Hae-Jong Seo, Abhishek Bajpayee, David Nister, Minwoo Park, Neda Cvijetic
  • Patent number: 11508076
    Abstract: A neural network model receives color data for a sequence of images corresponding to a dynamic scene in three-dimensional (3D) space. Motion of objects in the image sequence results from a combination of a dynamic camera orientation and motion or a change in the shape of an object in the 3D space. The neural network model generates two components that are used to produce a 3D motion field representing the dynamic (non-rigid) part of the scene. The two components are information identifying dynamic and static portions of each image and the camera orientation. The dynamic portions of each image contain motion in the 3D space that is independent of the camera orientation. In other words, the motion in the 3D space (estimated 3D scene flow data) is separated from the motion of the camera.
    Type: Grant
    Filed: January 22, 2021
    Date of Patent: November 22, 2022
    Assignee: NVIDIA Corporation
    Inventors: Zhaoyang Lv, Kihwan Kim, Deqing Sun, Alejandro Jose Troccoli, Jan Kautz
  • Patent number: 11498007
    Abstract: A technique for analyzing data in order to detect issues within a cloud-based service is disclosed. Host computing devices in a data center launch virtual machines, where at least some virtual machines run a pipelined stack for a streaming service. Virtual machines in the host computing devices generate event data including timestamps. Metadata generated by the pipelined stack during each streaming session is analyzed to identify deadzones in the corresponding host computing device, and the event data is processed to identify potential root causes of the corresponding deadzones. The event data can be generated by the virtual machine hosting the streaming service or by different virtual machines on the same host computing device. A distribution of events of each event type relative to the identified deadzones is determined and an operation of the host computing device can be adjusted based on the distribution.
    Type: Grant
    Filed: December 17, 2020
    Date of Patent: November 15, 2022
    Assignee: NVIDIA Corporation
    Inventors: Alan Daniel Larson, Bipin Todur
  • Patent number: 11501467
    Abstract: A remote device utilizes ray tracing to compute a light field for a scene to be rendered, where the light field includes information about light reflected off surfaces within the scene. This light field is then compressed utilizing lossless or lossy compression and one or more video compression techniques that implement temporal reuse, such that only differences between the light field for the scene and a light field for a previous scene are compressed. The compressed light field data is then sent to a client device that decompresses the light field data and uses such data to obtain the light field for the scene at the client device. This light field is then used by the client device to compute global illumination for the scene. The global illumination may be used to accurately render the scene at the mobile device, resulting in a realistic scene that is presented by the mobile device.
    Type: Grant
    Filed: May 5, 2021
    Date of Patent: November 15, 2022
    Assignee: NVIDIA CORPORATION
    Inventors: Michael Stengel, Alexander Majercik, Ben Boudaoud, Morgan McGuire, Dawid Stanislaw Pajak
  • Patent number: 11501572
    Abstract: In various examples, a set of object trajectories may be determined based at least in part on sensor data representative of a field of view of a sensor. The set of object trajectories may be applied to a long short-term memory (LSTM) network to train the LSTM network. An expected object trajectory for an object in the field of view of the sensor may be computed by the LSTM network based at least in part an observed object trajectory. By comparing the observed object trajectory to the expected object trajectory, a determination may be made that the observed object trajectory is indicative of an anomaly.
    Type: Grant
    Filed: March 25, 2019
    Date of Patent: November 15, 2022
    Assignee: NVIDIA Corporation
    Inventors: Milind Naphade, Shuo Wang
  • Patent number: 11502867
    Abstract: A network device configured to perform scalable, in-network computations is described. The network device is configured to process pull requests and/or push requests from a plurality of endpoints connected to the network. A collective communication primitive from a particular endpoint can be received at a network device. The collective communication primitive is associated with a multicast region of a shared global address space and is mapped to a plurality of participating endpoints. The network device is configured to perform an in-network computation based on information received from the participating endpoints before forwarding a response to the collective communication primitive back to one or more of the participating endpoints.
    Type: Grant
    Filed: July 24, 2020
    Date of Patent: November 15, 2022
    Assignee: NVIDIA Corporation
    Inventors: Benjamin Klenk, Nan Jiang, Larry Robert Dennison
  • Patent number: 11494265
    Abstract: In general, data is susceptible to errors caused by faults in hardware (i.e. permanent faults), such as faults in the functioning of memory and/or communication channels. To detect errors in data caused by hardware faults, the error correcting code (ECC) was introduced, which essentially provides a sort of redundancy to the data that can be used to validate that the data is free from errors caused by hardware faults. In some cases, the ECC can also be used to correct errors in the data caused by hardware faults. However, the ECC itself is also susceptible to errors, including specifically errors caused by faults in the ECC logic. A method, computer readable medium, and system are thus provided for securing against errors in an ECC.
    Type: Grant
    Filed: December 31, 2020
    Date of Patent: November 8, 2022
    Assignee: NVIDIA Corporation
    Inventor: Nirmal R. Saxena
  • Patent number: 11494879
    Abstract: A neural network architecture is disclosed for restoring noisy data. The neural network is a blind-spot network that can be trained according to a self-supervised framework. In an embodiment, the blind-spot network includes a plurality of network branches. Each network branch processes a version of the input data using one or more layers associated with kernels that have a receptive field that extends in a particular half-plane relative to the output value. In one embodiment, the versions of the input data are offset in a particular direction and the convolution kernels are rotated to correspond to the particular direction of the associated network branch. In another embodiment, the versions of the input data are rotated and the convolution kernel is the same for each network branch. The outputs of the network branches are composited to de-noise the image. In some embodiments, Bayesian filtering is performed to de-noise the input data.
    Type: Grant
    Filed: December 20, 2019
    Date of Patent: November 8, 2022
    Assignee: NVIDIA Corporation
    Inventors: Samuli Matias Laine, Tero Tapani Karras, Jaakko T. Lehtinen, Timo Oskari Aila
  • Patent number: 11496773
    Abstract: A method, computer readable medium, and system are disclosed for identifying residual video data. This data describes data that is lost during a compression of original video data. For example, the original video data may be compressed and then decompressed, and this result may be compared to the original video data to determine the residual video data. This residual video data is transformed into a smaller format by means of encoding, binarizing, and compressing, and is sent to a destination. At the destination, the residual video data is transformed back into its original format and is used during the decompression of the compressed original video data to improve a quality of the decompressed original video data.
    Type: Grant
    Filed: June 18, 2021
    Date of Patent: November 8, 2022
    Assignee: NVIDIA CORPORATION
    Inventors: Yi-Hsuan Tsai, Ming-Yu Liu, Deqing Sun, Ming-Hsuan Yang, Jan Kautz
  • Patent number: 11494976
    Abstract: Approaches are presented for training an inverse graphics network. An image synthesis network can generate training data for an inverse graphics network. In turn, the inverse graphics network can teach the synthesis network about the physical three-dimensional (3D) controls. Such an approach can provide for accurate 3D reconstruction of objects from 2D images using the trained inverse graphics network, while requiring little annotation of the provided training data. Such an approach can extract and disentangle 3D knowledge learned by generative models by utilizing differentiable renderers, enabling a disentangled generative model to function as a controllable 3D “neural renderer,” complementing traditional graphics renderers.
    Type: Grant
    Filed: March 5, 2021
    Date of Patent: November 8, 2022
    Assignee: Nvidia Corporation
    Inventors: Wenzheng Chen, Yuxuan Zhang, Sanja Fidler, Huan Ling, Jun Gao, Antonio Torralba Barriuso
  • Patent number: 11494370
    Abstract: Latency of in-system test (IST) execution for a hardware component of an in-field (deployed) computing platform may be reduced when a value of a physical operating parameter can be changed without rebooting the computing platform. A test (e.g., patterns or vectors) is executed for varying values of the physical operating parameter (e.g., supply voltage, clock speed, temperature, noise magnitude/duration, operating current, and the like), providing the ability to detect faults in the hardware components.
    Type: Grant
    Filed: March 20, 2020
    Date of Patent: November 8, 2022
    Assignee: NVIDIA Corporation
    Inventors: Sreedhar Narayanaswamy, Shantanu K. Sarangi, Hemalkumar Chandrakant Doshi, Hari Unni Krishnan, Gunaseelan Ponnuvel, Brian Lawrence Smith