Patents by Inventor Brian McWilliams
Brian McWilliams has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240086745Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining principal components of a data set using multi-agent interactions.Type: ApplicationFiled: February 7, 2022Publication date: March 14, 2024Inventors: Ian Michael Gemp, Brian McWilliams
-
Publication number: 20230083929Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: ApplicationFiled: November 9, 2022Publication date: March 16, 2023Applicants: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Patent number: 11532073Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: GrantFiled: July 31, 2018Date of Patent: December 20, 2022Assignees: Pixar, Disnev Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Publication number: 20210383227Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an embedding neural network based on score distributions. In one aspect, a system comprises: generating a first and second embedding of a data element, comprising: applying a first and second transformation to the data element to generate a respective first and second version of the data element and processing the respective versions using the embedding neural network to generate the respective first and second embeddings; generating, for the data element, a respective first and respective second score distribution, comprising: processing at least the first and the second embedding to generate the first and the second score distribution, respectively; and updating the current embedding network parameter values to optimize an objective function that is based on at least the first score distribution, that encourages a similarity between: (i) the first, and (ii) the second score distribution.Type: ApplicationFiled: June 4, 2021Publication date: December 9, 2021Inventors: Jovana Mitrovic, Brian McWilliams
-
Patent number: 11037274Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: GrantFiled: February 12, 2020Date of Patent: June 15, 2021Assignees: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10818080Abstract: According to one implementation, a system includes a computing platform having a hardware processor and a system memory storing a software code including multiple artificial neural networks (ANNs). The hardware processor executes the software code to partition a multi-dimensional input vector into a first vector data and a second vector data, and to transform the second vector data using a first piecewise-polynomial transformation parameterized by one of the ANNs, based on the first vector data, to produce a transformed second vector data. The hardware processor further executes the software code to transform the first vector data using a second piecewise-polynomial transformation parameterized by another of the ANNs, based on the transformed second vector data, to produce a transformed first vector data, and to determine a multi-dimensional output vector based on an output from the plurality of ANNs.Type: GrantFiled: October 11, 2018Date of Patent: October 27, 2020Assignees: Disney Enterprises, Inc., ETH Zürich (EIDGENÖSSISCHE TECHNISCHE HOCHSCHULE ZÜRICH)Inventors: Thomas Muller, Brian McWilliams, Fabrice Pierre Armand Rousselle, Jan Novak
-
Patent number: 10796414Abstract: Supervised machine learning using convolutional neural network (CNN) is applied to denoising images rendered by MC path tracing. The input image data may include pixel color and its variance, as well as a set of auxiliary buffers that encode scene information (e.g., surface normal, albedo, depth, and their corresponding variances). In some embodiments, a CNN directly predicts the final denoised pixel value as a highly non-linear combination of the input features. In some other embodiments, a kernel-prediction neural network uses a CNN to estimate the local weighting kernels, which are used to compute each denoised pixel from its neighbors. In some embodiments, the input image can be decomposed into diffuse and specular components. The diffuse and specular components are then independently preprocessed, filtered, and postprocessed, before recombining them to obtain a final denoised image.Type: GrantFiled: September 26, 2019Date of Patent: October 6, 2020Assignees: Disney Enterprises, Inc., ETH Zürich (Eidgenössische Technische Hochschule Zürich)Inventors: Thijs Vogels, Jan Novák, Fabrice Rousselle, Brian McWilliams
-
Patent number: 10789686Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: GrantFiled: January 6, 2020Date of Patent: September 29, 2020Assignees: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10706508Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: GrantFiled: July 31, 2018Date of Patent: July 7, 2020Assignees: Disney Enterprises, Inc., PixarInventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Patent number: 10699382Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: GrantFiled: July 31, 2018Date of Patent: June 30, 2020Assignees: Disney Enterprises, Inc., PixarInventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Publication number: 20200184313Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: ApplicationFiled: July 31, 2018Publication date: June 11, 2020Applicants: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Publication number: 20200184605Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: ApplicationFiled: February 12, 2020Publication date: June 11, 2020Applicants: PIXAR, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10672109Abstract: A modular architecture is provided for denoising Monte Carlo renderings using neural networks. The temporal approach extracts and combines feature representations from neighboring frames rather than building a temporal context using recurrent connections. A multiscale architecture includes separate single-frame or temporal denoising modules for individual scales, and one or more scale compositor neural networks configured to adaptively blend individual scales. An error-predicting module is configured to produce adaptive sampling maps for a renderer to achieve more uniform residual noise distribution. An asymmetric loss function may be used for training the neural networks, which can provide control over the variance-bias trade-off during denoising.Type: GrantFiled: July 31, 2018Date of Patent: June 2, 2020Assignees: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Jan Novak, Brian McWilliams, Mark Meyer, Alex Harvill
-
Publication number: 20200143522Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: ApplicationFiled: January 6, 2020Publication date: May 7, 2020Applicants: PIXAR, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10621695Abstract: According to one implementation, a video processing system includes a computing platform having a hardware processor and a system memory storing a software code including an artificial neural network (ANN). The hardware processor is configured to execute the software code to receive a first video sequence having a first display resolution, and to produce a second video sequence based on the first video sequence using the ANN. The second video sequence has a second display resolution higher than the first display resolution. The ANN is configured to provide sequential frames of the second video sequence that are temporally stable and consistent in color to reduce visual flicker and color shifting in the second video sequence.Type: GrantFiled: February 1, 2018Date of Patent: April 14, 2020Assignee: Disney Enterprises, Inc.Inventors: Christopher Schroers, Yifan Wang, Federico Perazzi, Brian McWilliams, Alexander Sorkine Hornung
-
Patent number: 10607319Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: GrantFiled: April 5, 2018Date of Patent: March 31, 2020Assignees: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10586310Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: GrantFiled: April 5, 2018Date of Patent: March 10, 2020Assignees: Pixar, Disney EnterprisesInventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Patent number: 10572979Abstract: Supervised machine learning using neural networks is applied to denoising images rendered by MC path tracing. Specialization of neural networks may be achieved by using a modular design that allows reusing trained components in different networks and facilitates easy debugging and incremental building of complex structures. Specialization may also be achieved by using progressive neural networks. In some embodiments, training of a neural-network based denoiser may use importance sampling, where more challenging patches or patches including areas of particular interests within a training dataset are selected with higher probabilities than others. In some other embodiments, generative adversarial networks (GANs) may be used for training a machine-learning based denoiser as an alternative to using pre-defined loss functions.Type: GrantFiled: April 5, 2018Date of Patent: February 25, 2020Assignees: Pixar, Disney Enterprises, Inc.Inventors: Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Mark Meyer, Jan Novak
-
Publication number: 20200035016Abstract: According to one implementation, a system includes a computing platform having a hardware processor and a system memory storing a software code including multiple artificial neural networks (ANNs). The hardware processor executes the software code to partition a multi-dimensional input vector into a first vector data and a second vector data, and to transform the second vector data using a first piecewise-polynomial transformation parameterized by one of the ANNs, based on the first vector data, to produce a transformed second vector data. The hardware processor further executes the software code to transform the first vector data using a second piecewise-polynomial transformation parameterized by another of the ANNs, based on the transformed second vector data, to produce a transformed first vector data, and to determine a multi-dimensional output vector based on an output from the plurality of ANNs.Type: ApplicationFiled: October 11, 2018Publication date: January 30, 2020Inventors: Thomas Muller, Brian McWilliams, Fabrice Pierre Armand Rousselle, Jan Novak
-
Publication number: 20200027198Abstract: Supervised machine learning using convolutional neural network (CNN) is applied to denoising images rendered by MC path tracing. The input image data may include pixel color and its variance, as well as a set of auxiliary buffers that encode scene information (e.g., surface normal, albedo, depth, and their corresponding variances). In some embodiments, a CNN directly predicts the final denoised pixel value as a highly non-linear combination of the input features. In some other embodiments, a kernel-prediction neural network uses a CNN to estimate the local weighting kernels, which are used to compute each denoised pixel from its neighbors. In some embodiments, the input image can be decomposed into diffuse and specular components. The diffuse and specular components are then independently preprocessed, filtered, and postprocessed, before recombining them to obtain a final denoised image.Type: ApplicationFiled: September 26, 2019Publication date: January 23, 2020Applicants: Disney Enterprises, Inc., ETH ZÜRICH (EIDGENÖSSISCHE TECHNISCHE HOCHSCHULE ZÜRICH)Inventors: Thijs Vogels, Jan Novák, Fabrice Rousselle, Brian McWilliams