Patents by Inventor Aaron Eliot Lefohn
Aaron Eliot Lefohn has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240378759Abstract: In computer graphics, texture refers to a type of surface, including the material characteristics, that can be applied to an object in an image. A texture may be defined using numerous parameters, such as color(s), roughness, glossiness, etc. In some implementations, a texture may be represented as an image that can be placed on a three-dimensional (3D) model of an object to give surface details to the 3D object. To reduce a size of textures (e.g. for storage and transmission), textures in a texture set may be compressed together. The present disclosure provides for decompression of at least a portion of a single texture representation of a set of textures into at least a portion of a plurality of textures included in the set of textures.Type: ApplicationFiled: May 10, 2024Publication date: November 14, 2024Inventors: Karthik Vaidyanathan, Marco Salvi, Bartlomiej Wronski, Tomas Akenine-Moller, Johan Pontus Ebelin, Aaron Eliot Lefohn, John Matthew Burgess, Steven James Heinrich, Michael Alan Fetterman, Shirish Gadre, Mark Alan Gebhart, Yury Uralsky
-
Publication number: 20240378792Abstract: In computer graphics, texture refers to a type of surface, including the material characteristics, that can be applied to an object in an image. A texture may be defined using numerous parameters, such as color(s), roughness, glossiness, etc. In some implementations, a texture may be represented as an image that can be placed on a three-dimensional (3D) model of an object to give surface details to the 3D object. To reduce a size of textures (e.g. for storage and transmission), textures in a texture set may be compressed together. The present disclosure provides for transcoding a compressed texture set to textures with a hardware-supported compression format.Type: ApplicationFiled: May 10, 2024Publication date: November 14, 2024Inventors: Karthik Vaidyanathan, Marco Salvi, Bartlomiej Wronski, Tomas Akenine-Moller, Johan Pontus Ebelin, Aaron Eliot Lefohn, John Matthew Burgess, Steven James Heinrich, Michael Alan Fetterman, Shirish Gadre, Mark Alan Gebhart, Yury Uralsky
-
Publication number: 20240257437Abstract: Embodiments of the present disclosure relate to real-time neural appearance models. Using a neural decoder, scenes are rendered in real-time with complex material appearance previously reserved for offline use. Learned hierarchical textures representing the material properties are encoded as latent codes. When a ray is cast and intersects with geometry in the scene, the intersection point is mapped to one of the latent codes. The latent code is interpreted using neural decoders, which produce reflectance values and importance-sampled directions that can be used to determine a pixel color.Type: ApplicationFiled: January 22, 2024Publication date: August 1, 2024Inventors: Karthik Vaidyanathan, Alex John Bauld Evans, Jan Novák, Andrea Weidlich, Fabrice Pierre Armand Rousselle, Aaron Eliot Lefohn, Franz Petrik Clarberg, Benedikt Bitterli, Tizian Lucien Zeltner
-
Publication number: 20240257405Abstract: In computer graphics, texture refers to a type of surface, including the material characteristics, that can be applied to an object in an image. A texture may be defined using numerous parameters, such as color(s), roughness, glossiness, etc. In some implementations, a texture may be represented as an image that can be placed on a three-dimensional (3D) model of an object to give surface details to the 3D object. To reduce a size of textures (e.g. for storage and transmission), the present disclosure provides, in one embodiment, for compression of a texture set using a non-linear function and quantization. In another embodiment, the disclosure provides for compression of one or more textures using a non-linear function configured to compress textures with an arbitrary number of channels and/or an arbitrary ordering of channels.Type: ApplicationFiled: January 23, 2024Publication date: August 1, 2024Inventors: Karthik Vaidyanathan, Marco Salvi, Bartlomiej Wronski, Tomas Akenine-Moller, Johan Pontus Ebelin, Aaron Eliot Lefohn, John Matthew Burgess, Steven James Heinrich, Michael Alan Fetterman, Shirish Gadre, Mark Alan Gebhart
-
Patent number: 11925860Abstract: This application discloses techniques for generating and querying projective hash maps. More specifically, projective hash maps can be used for spatial hashing of data related to N-dimensional points. Each point is projected onto a projection surface to convert the three-dimensional (3D) coordinates for the point to two-dimensional (2D) coordinates associated with the projection surface. Hash values based on the 2D coordinates are then used as an index to store data in the projective hash map. Utilizing the 2D coordinates rather than the 3D coordinates allows for more efficient searches to be performed to locate points in the 3D space. In particular, projective hash maps can be utilized by graphics applications for generating images, and the improved efficiency can, for example, enable a game streaming application on a server to render images transmitted to a user device via a network at faster frame rates.Type: GrantFiled: June 9, 2021Date of Patent: March 12, 2024Assignee: NVIDIA CorporationInventors: Marco Salvi, Jacopo Pantaleoni, Aaron Eliot Lefohn, Christopher Ryan Wyman, Pascal Gautron
-
Patent number: 11861811Abstract: A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.Type: GrantFiled: September 8, 2022Date of Patent: January 2, 2024Assignee: NVIDIA CorporationInventors: Carl Jacob Munkberg, Jon Niklas Theodor Hasselgren, Anjul Patney, Marco Salvi, Aaron Eliot Lefohn, Donald Lee Brittain
-
Publication number: 20230410375Abstract: A method, computer readable medium, and system are disclosed for temporally stable data reconstruction. A sequence of input data including artifacts is received. A first input data frame is processed using layers of a neural network model to produce external state including a reconstructed first data frame that approximates the first input data frame without artifacts. Hidden state generated during processing of the first input data is not provided as an input to the layer to process second input data. The external state is warped, using difference data corresponding to changes between input data frames, to produce warped external state more closely aligned with the second input data frame. The second input data frame is processed, based on the warped external state, using the layers of the neural network model to produce a reconstructed second data frame that approximates the second data frame without artifacts.Type: ApplicationFiled: July 24, 2023Publication date: December 21, 2023Inventors: Marco Salvi, Anjul Patney, Aaron Eliot Lefohn, Donald Lee Brittain
-
Patent number: 11836597Abstract: Motivated by the ability of humans to quickly and accurately detect visual artifacts in images, a neural network model is trained to identify and locate visual artifacts in a sequence of rendered images without comparing the sequence of rendered images against a ground truth reference. Examples of visual artifacts include aliasing, blurriness, mosaicking, and overexposure. The neural network model provides a useful fully-automated tool for evaluating the quality of images produced by rendering systems. The neural network model may be trained to evaluate the quality of images for video processing, encoding, and/or compression techniques. In an embodiment, the sequence includes at least four images corresponding to a video or animation.Type: GrantFiled: April 29, 2019Date of Patent: December 5, 2023Assignee: NVIDIA CorporationInventors: Anjul Patney, Aaron Eliot Lefohn
-
Publication number: 20230014245Abstract: A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.Type: ApplicationFiled: September 8, 2022Publication date: January 19, 2023Inventors: Carl Jacob Munkberg, Jon Niklas Theodor Hasselgren, Anjul Patney, Marco Salvi, Aaron Eliot Lefohn, Donald Lee Brittain
-
Patent number: 11557022Abstract: A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.Type: GrantFiled: December 18, 2019Date of Patent: January 17, 2023Assignee: NVIDIA CorporationInventors: Carl Jacob Munkberg, Jon Niklas Theodor Hasselgren, Anjul Patney, Marco Salvi, Aaron Eliot Lefohn, Donald Lee Brittain
-
Publication number: 20220395748Abstract: This application discloses techniques for generating and querying projective hash maps. More specifically, projective hash maps can be used for spatial hashing of data related to N-dimensional points. Each point is projected onto a projection surface to convert the three-dimensional (3D) coordinates for the point to two-dimensional (2D) coordinates associated with the projection surface. Hash values based on the 2D coordinates are then used as an index to store data in the projective hash map. Utilizing the 2D coordinates rather than the 3D coordinates allows for more efficient searches to be performed to locate points in the 3D space. In particular, projective hash maps can be utilized by graphics applications for generating images, and the improved efficiency can, for example, enable a game streaming application on a server to render images transmitted to a user device via a network at faster frame rates.Type: ApplicationFiled: June 9, 2021Publication date: December 15, 2022Inventors: Marco Salvi, Jacopo Pantaleoni, Aaron Eliot Lefohn, Christopher Ryan Wyman, Pascal Gautron
-
Patent number: 11475542Abstract: A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.Type: GrantFiled: December 17, 2019Date of Patent: October 18, 2022Assignee: NVIDIA CorporationInventors: Carl Jacob Munkberg, Jon Niklas Theodor Hasselgren, Anjul Patney, Marco Salvi, Aaron Eliot Lefohn, Donald Lee Brittain
-
Publication number: 20220198746Abstract: A global illumination data structure (e.g., a data structure created to store global illumination information for geometry within a scene to be rendered) is computed for the scene. Additionally, reservoir-based spatiotemporal importance resampling (RESTIR) is used to perform illumination gathering, utilizing the global illumination data structure. The illumination gathering includes identifying light values for points within the scene, where one or more points are selected within the scene based on the light values in order to perform ray tracing during the rendering of the scene.Type: ApplicationFiled: March 11, 2022Publication date: June 23, 2022Inventors: Christopher Ryan Wyman, Morgan McGuire, Peter Schuyler Shirley, Aaron Eliot Lefohn
-
Patent number: 11315310Abstract: A global illumination data structure (e.g., a data structure created to store global illumination information for geometry within a scene to be rendered) is computed for the scene. Additionally, reservoir-based spatiotemporal importance resampling (RESTIR) is used to perform illumination gathering, utilizing the global illumination data structure. The illumination gathering includes identifying light values for points within the scene, where one or more points are selected within the scene based on the light values in order to perform ray tracing during the rendering of the scene.Type: GrantFiled: January 19, 2021Date of Patent: April 26, 2022Assignee: NVIDIA CORPORATIONInventors: Christopher Ryan Wyman, Morgan McGuire, Peter Schuyler Shirley, Aaron Eliot Lefohn
-
Publication number: 20210287096Abstract: The disclosed microtraining techniques improve accuracy of trained neural networks by performing iterative refinement at low learning rates using a relatively short series microtraining steps. A neural network training framework receives the trained neural network along with a second training dataset and set of hyperparameters. The neural network training framework produces a microtrained neural network by adjusting one or more weights of the trained neural network using a lower learning rate to facilitate incremental accuracy improvements without substantially altering the computational structure of the trained neural network. The microtrained neural network may be assessed for changes in accuracy and/or quality. Additional microtraining sessions may be performed on the microtrained neural network to further improve accuracy or quality.Type: ApplicationFiled: March 13, 2020Publication date: September 16, 2021Inventors: Anjul Patney, Brandon Lee Rowlett, Yinghao Xu, Andrew Leighton Edelsten, Aaron Eliot Lefohn
-
Publication number: 20210287426Abstract: A global illumination data structure (e.g., a data structure created to store global illumination information for geometry within a scene to be rendered) is computed for the scene. Additionally, reservoir-based spatiotemporal importance resampling (RESTIR) is used to perform illumination gathering, utilizing the global illumination data structure. The illumination gathering includes identifying light values for points within the scene, where one or more points are selected within the scene based on the light values in order to perform ray tracing during the rendering of the scene.Type: ApplicationFiled: January 19, 2021Publication date: September 16, 2021Inventors: Christopher Ryan Wyman, Morgan McGuire, Peter Schuyler Shirley, Aaron Eliot Lefohn
-
Patent number: 11113800Abstract: A method, computer readable medium, and system are disclosed for performing spatiotemporal filtering. The method includes identifying image data to be rendered, reconstructing the image data to create reconstructed image data, utilizing a filter including a neural network having one or more skip connections and one or more recurrent layers, and returning the reconstructed image data.Type: GrantFiled: January 16, 2018Date of Patent: September 7, 2021Assignee: NVIDIA CORPORATIONInventors: Anton S. Kaplanyan, Chakravarty Reddy Alla Chaitanya, Timo Oskari Aila, Aaron Eliot Lefohn, Marco Salvi
-
Patent number: 10922876Abstract: A method, computer readable medium, and system are disclosed for redirecting a user's movement through a physical space while the user views a virtual environment. A temporary visual suppression event is detected when a user's eyes move relative to the user's head while viewing a virtual scene displayed on a display device, an orientation of the virtual scene relative to the user is modified to direct the user to physically move along a planned path through a virtual environment corresponding to the virtual scene, and the virtual scene is displayed on the display device according to the modified orientation.Type: GrantFiled: January 2, 2020Date of Patent: February 16, 2021Assignee: NVIDIA CorporationInventors: Qi Sun, Anjul Patney, Omer Shapira, Morgan McGuire, Aaron Eliot Lefohn, David Patrick Luebke
-
Publication number: 20200160590Abstract: A method, computer readable medium, and system are disclosed for redirecting a user's movement through a physical space while the user views a virtual environment. A temporary visual suppression event is detected when a user's eyes move relative to the user's head while viewing a virtual scene displayed on a display device, an orientation of the virtual scene relative to the user is modified to direct the user to physically move along a planned path through a virtual environment corresponding to the virtual scene, and the virtual scene is displayed on the display device according to the modified orientation.Type: ApplicationFiled: January 2, 2020Publication date: May 21, 2020Inventors: Qi Sun, Anjul Patney, Omer Shapira, Morgan McGuire, Aaron Eliot Lefohn, David Patrick Luebke
-
Publication number: 20200126192Abstract: A neural network-based rendering technique increases temporal stability and image fidelity of low sample count path tracing by optimizing a distribution of samples for rendering each image in a sequence. A sample predictor neural network learns spatio-temporal sampling strategies such as placing more samples in dis-occluded regions and tracking specular highlights. Temporal feedback enables a denoiser neural network to boost the effective input sample count and increases temporal stability. The initial uniform sampling step typically present in adaptive sampling algorithms is not needed. The sample predictor and denoiser operate at interactive rates to achieve significantly improved image quality and temporal stability compared with conventional adaptive sampling techniques.Type: ApplicationFiled: December 18, 2019Publication date: April 23, 2020Inventors: Carl Jacob Munkberg, Jon Niklas Theodor Hasselgren, Anjul Patney, Marco Salvi, Aaron Eliot Lefohn, Donald Lee Brittain