Patents by Inventor Jan NOVÁK
Jan NOVÁK has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 10789686Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: GrantFiled: January 6, 2020Date of Patent: September 29, 2020Assignees: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10706508Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: GrantFiled: July 31, 2018Date of Patent: July 7, 2020Assignees: Disney Enterprises, Inc., PixarInventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Patent number: 10699382Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: GrantFiled: July 31, 2018Date of Patent: June 30, 2020Assignees: Disney Enterprises, Inc., PixarInventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Publication number: 20200184313Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: ApplicationFiled: July 31, 2018Publication date: June 11, 2020Applicants: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Publication number: 20200184605Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: ApplicationFiled: February 12, 2020Publication date: June 11, 2020Applicants: PIXAR, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10672109Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: GrantFiled: July 31, 2018Date of Patent: June 2, 2020Assignees: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Publication number: 20200143522Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: ApplicationFiled: January 6, 2020Publication date: May 7, 2020Applicants: PIXAR, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10607319Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: GrantFiled: April 5, 2018Date of Patent: March 31, 2020Assignees: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10586310Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: GrantFiled: April 5, 2018Date of Patent: March 10, 2020Assignees: Pixar, Disney EnterprisesInventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10580194Abstract: Systems, methods and articles of manufacture for rendering three-dimensional virtual environments using reversible jumps are disclosed herein. In one embodiment, mappings from random numbers to light paths are modeled as an explicit iterative random walk. Inverses of path construction techniques are employed to turn light transport paths back into the random numbers that produced them. In particular, such inverses may be used to extend the Multiplexed Metropolis Light Transport (MMLT) technique to perform path-invariant perturbations that produce a new path sample using a different path construction technique but preserve the path's geometry.Type: GrantFiled: November 9, 2017Date of Patent: March 3, 2020Assignees: Disney Enterprises, Inc., ETH Zurich (Eidgenoessische Technische Hochschule Zurich)Inventors: Jan Novák, Wenzel A. Jakob, Wojciech Jarosz, Benedikt Martin Bitterli
-
Patent number: 10572979Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: GrantFiled: April 5, 2018Date of Patent: February 25, 2020Assignees: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10565685Abstract: According to one implementation, an image rendering system includes a computing platform having a hardware processor and a system memory storing an image denoising software code. The hardware processor executes the image denoising software code to receive an image file including multiple pixels, each pixel containing multiple depth bins, and to select a pixel including a noisy depth bin from among the pixels for denoising. The hardware processor further executes the image denoising software code to identify a plurality of reference depth bins from among the depth bins contained in one or more of the pixels, for use in denoising the noisy depth bin, and to denoise the noisy depth bin using an average of depth bin values corresponding respectively to each of the reference depth bins.Type: GrantFiled: June 27, 2017Date of Patent: February 18, 2020Assignee: Disney Enterprises, Inc.Inventors: David M. Adler, Delio Aleardo Vicini, Brent Burley, Jan Novak, Fabrice Pierre Armand Rousselle
-
Publication number: 20200035016Abstract: According to one implementation, a system includes a computing platform having a hardware processor and a system memory storing a software code including multiple artificial neural networks (ANNs). The hardware processor executes the software code to partition a multi-dimensional input vector into a first vector data and a second vector data, and to transform the second vector data using a first piecewise-polynomial transformation parameterized by one of the ANNs, based on the first vector data, to produce a transformed second vector data. The hardware processor further executes the software code to transform the first vector data using a second piecewise-polynomial transformation parameterized by another of the ANNs, based on the transformed second vector data, to produce a transformed first vector data, and to determine a multi-dimensional output vector based on an output from the plurality of ANNs.Type: ApplicationFiled: October 11, 2018Publication date: January 30, 2020Inventors: Thomas Muller, Brian McWilliams, Fabrice Pierre Armand Rousselle, Jan Novak
-
Publication number: 20200027198Abstract: Supervised machine learning using convolutional neural network (CNN) is applied to denoising images rendered by MC path tracing. The input image data may include pixel color and its variance, as well as a set of auxiliary buffers that encode scene information (e.g., surface normal, albedo, depth, and their corresponding variances). In some embodiments, a CNN directly predicts the final denoised pixel value as a highly non-linear combination of the input features. In some other embodiments, a kernel-prediction neural network uses a CNN to estimate the local weighting kernels, which are used to compute each denoised pixel from its neighbors. In some embodiments, the input image can be decomposed into diffuse and specular components. The diffuse and specular components are then independently preprocessed, filtered, and postprocessed, before recombining them to obtain a final denoised image.Type: ApplicationFiled: September 26, 2019Publication date: January 23, 2020Applicants: Disney Enterprises, Inc., ETH ZÜRICH (EIDGENÖSSISCHE TECHNISCHE HOCHSCHULE ZÜRICH)Inventors: Thijs Vogels, Jan Novák, Fabrice Rousselle, Brian McWilliams
-
Patent number: 10475165Abstract: Supervised machine learning using convolutional neural network (CNN) is applied to denoising images rendered by MC path tracing. The input image data may include pixel color and its variance, as well as a set of auxiliary buffers that encode scene information (e.g., surface normal, albedo, depth, and their corresponding variances). In some embodiments, a CNN directly predicts the final denoised pixel value as a highly non-linear combination of the input features. In some other embodiments, a kernel-prediction neural network uses a CNN to estimate the local weighting kernels, which are used to compute each denoised pixel from its neighbors. In some embodiments, the input image can be decomposed into diffuse and specular components. The diffuse and specular components are then independently preprocessed, filtered, and postprocessed, before recombining them to obtain a final denoised image.Type: GrantFiled: November 15, 2017Date of Patent: November 12, 2019Assignees: Disney Enterprises, Inc., ETH Zürich (Eidgenössische Technische Hochschule ZürichInventors: Thijs Vogels, Jan Novák, Fabrice Rousselle, Brian McWilliams
-
Publication number: 20190304069Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: ApplicationFiled: July 31, 2018Publication date: October 3, 2019Applicants: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill, David Adler
-
Publication number: 20190304067Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: ApplicationFiled: July 31, 2018Publication date: October 3, 2019Applicants: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill, David Adler
-
Publication number: 20190304068Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: ApplicationFiled: July 31, 2018Publication date: October 3, 2019Applicants: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Publication number: 20190139295Abstract: Systems, methods and articles of manufacture for rendering three-dimensional virtual environments using reversible jumps are disclosed herein. In one embodiment, mappings from random numbers to light paths are modeled as an explicit iterative random walk. Inverses of path construction techniques are employed to turn light transport paths back into the random numbers that produced them. In particular, such inverses may be used to extend the Multiplexed Metropolis Light Transport (MMLT) technique to perform path-invariant perturbations that produce a new path sample using a different path construction technique but preserve the path's geometry.Type: ApplicationFiled: November 9, 2017Publication date: May 9, 2019Inventors: Jan NOVÁK, Wenzel A. JAKOB, Wojciech JAROSZ, Benedikt Martin BITTERLI
-
Patent number: 10275934Abstract: A video rendering system includes a field-of-view detector, a display, and a computing platform including a hardware processor and a memory storing a multi-viewpoint video rendering software code. The hardware processor executes the multi-viewpoint video rendering software code to parameterize visible surfaces in a scene to define multiple texels for each visible surface, precompute one or more illumination value(s) for each texel of each visible surface, and for each texel of each visible surface, store the illumination value(s) in a cache assigned to the texel. In addition, the multi-viewpoint video rendering software code receives a perspective data from the field-of-view detector identifying one of multiple permissible perspectives for viewing the scene, and renders the scene on the display in real-time with respect to receiving the perspective data, based on the identified perspective and using one or more of the illumination value(s) precomputed for each texel of each visible surface.Type: GrantFiled: December 20, 2017Date of Patent: April 30, 2019Assignee: Disney Enterprises, Inc.Inventors: Jan Novak, Christopher Schroers, Fabrice Pierre Armand Rousselle, Matthias Fauconneau, Alexander Sorkine Hornung