Patents by Inventor Tero Tapani KARRAS

Tero Tapani KARRAS has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20210383241
    Abstract: Embodiments of the present disclosure relate to a technique for training neural networks, such as a generative adversarial neural network (GAN), using a limited amount of data. Training GANs using too little example data typically leads to discriminator overfitting, causing training to diverge and produce poor results. An adaptive discriminator augmentation mechanism is used that significantly stabilizes training with limited data providing the ability to train high-quality GANs. An augmentation operator is applied to the distribution of inputs to a discriminator used to train a generator, representing a transformation that is invertible to ensure there is no leakage of the augmentations into the images generated by the generator. Reducing the amount of training data that is needed to achieve convergence has the potential to considerably help many applications and may the increase use of generative models in fields such as medicine.
    Type: Application
    Filed: March 24, 2021
    Publication date: December 9, 2021
    Inventors: Tero Tapani Karras, Miika Samuli Aittala, Janne Johannes Hellsten, Samuli Matias Laine, Jaakko T. Lehtinen, Timo Oskari Aila
  • Publication number: 20210366177
    Abstract: In examples, a list of elements may be divided into spans and each span may be allocated a respective memory range for output based on a worst-case compression ratio of a compression algorithm that will be used to compress the span. Worker threads may output compressed versions of the spans to the memory ranges. To ensure placement constraints of a data structure will be satisfied, boundaries of the spans may be adjusted prior to compression. The size allocated to a span (e.g., each span) may be increased (or decreasing) to avoid padding blocks while allowing for the span's compressed data to use a block allocated to an adjacent span. Further aspects of the disclosure provide for compaction of the portions of compressed data in memory in order to free up space which may have been allocated to account for the memory gaps which may result from variable compression ratios.
    Type: Application
    Filed: May 21, 2020
    Publication date: November 25, 2021
    Inventors: Timo Tapani Viitanen, Tero Tapani Karras, Samuli Laine
  • Publication number: 20210329306
    Abstract: Apparatuses, systems, and techniques to perform compression of video data using neural networks to facilitate video streaming, such as video conferencing. In at least one embodiment, a sender transmits to a receiver a key frame from video data and one or more keypoints identified by a neural network from said video data, and a receiver reconstructs video data using said key frame and one or more received keypoints.
    Type: Application
    Filed: October 13, 2020
    Publication date: October 21, 2021
    Inventors: Ming-Yu Liu, Ting-Chun Wang, Arun Mohanray Mallya, Tero Tapani Karras, Samuli Matias Laine, David Patrick Luebke, Jaakko Lehtinen, Miika Samuli Aittala, Timo Oskari Aila
  • Publication number: 20210150354
    Abstract: A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.
    Type: Application
    Filed: January 7, 2021
    Publication date: May 20, 2021
    Inventors: Tero Tapani Karras, Samuli Matias Laine, David Patrick Luebke, Jaakko T. Lehtinen, Miika Samuli Aittala, Timo Oskari Aila, Ming-Yu Liu, Arun Mohanray Mallya, Ting-Chun Wang
  • Publication number: 20210150369
    Abstract: A style-based generative network architecture enables scale-specific control of synthesized output data, such as images. During training, the style-based generative neural network (generator neural network) includes a mapping network and a synthesis network. During prediction, the mapping network may be omitted, replicated, or evaluated several times. The synthesis network may be used to generate highly varied, high-quality output data with a wide variety of attributes. For example, when used to generate images of people's faces, the attributes that may vary are age, ethnicity, camera viewpoint, pose, face shape, eyeglasses, colors (eyes, hair, etc.), hair style, lighting, background, etc. Depending on the task, generated output data may include images, audio, video, three-dimensional (3D) objects, text, etc.
    Type: Application
    Filed: January 28, 2021
    Publication date: May 20, 2021
    Inventors: Tero Tapani Karras, Samuli Matias Laine, Jaakko T. Lehtinen, Miika Samuli Aittala, Janne Johannes Hellsten, Timo Oskari Aila
  • Publication number: 20210150357
    Abstract: A style-based generative network architecture enables scale-specific control of synthesized output data, such as images. During training, the style-based generative neural network (generator neural network) includes a mapping network and a synthesis network. During prediction, the mapping network may be omitted, replicated, or evaluated several times. The synthesis network may be used to generate highly varied, high-quality output data with a wide variety of attributes. For example, when used to generate images of people's faces, the attributes that may vary are age, ethnicity, camera viewpoint, pose, face shape, eyeglasses, colors (eyes, hair, etc.), hair style, lighting, background, etc. Depending on the task, generated output data may include images, audio, video, three-dimensional (3D) objects, text, etc.
    Type: Application
    Filed: January 28, 2021
    Publication date: May 20, 2021
    Inventors: Tero Tapani Karras, Samuli Matias Laine, Jaakko T. Lehtinen, Miika Samuli Aittala, Janne Johannes Hellsten, Timo Oskari Aila
  • Publication number: 20210150187
    Abstract: A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.
    Type: Application
    Filed: January 7, 2021
    Publication date: May 20, 2021
    Inventors: Tero Tapani Karras, Samuli Matias Laine, David Patrick Luebke, Jaakko T. Lehtinen, Miika Samuli Aittala, Timo Oskari Aila, Ming-Yu Liu, Arun Mohanray Mallya, Ting-Chun Wang
  • Publication number: 20210117795
    Abstract: A style-based generative network architecture enables scale-specific control of synthesized output data, such as images. During training, the style-based generative neural network (generator neural network) includes a mapping network and a synthesis network. During prediction, the mapping network may be omitted, replicated, or evaluated several times. The synthesis network may be used to generate highly varied, high-quality output data with a wide variety of attributes. For example, when used to generate images of people's faces, the attributes that may vary are age, ethnicity, camera viewpoint, pose, face shape, eyeglasses, colors (eyes, hair, etc.), hair style, lighting, background, etc. Depending on the task, generated output data may include images, audio, video, three-dimensional (3D) objects, text, etc.
    Type: Application
    Filed: December 28, 2020
    Publication date: April 22, 2021
    Inventors: Tero Tapani Karras, Timo Oskari Aila, Samuli Matias Laine
  • Patent number: 10984049
    Abstract: A method, computer readable medium, and system are disclosed for performing traversal stack compression. The method includes traversing a hierarchical data structure having more than two children per node, and during the traversing, creating at least one stack entry, utilizing a processor, where each stack entry contains a plurality of intersected nodes, and adding the at least one stack entry to a compressed traversal stack stored in a memory, utilizing the processor.
    Type: Grant
    Filed: January 25, 2018
    Date of Patent: April 20, 2021
    Assignee: NVIDIA CORPORATION
    Inventors: Henri Johannes Ylitie, Tero Tapani Karras, Samuli Matias Laine
  • Publication number: 20210049468
    Abstract: A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.
    Type: Application
    Filed: October 13, 2020
    Publication date: February 18, 2021
    Inventors: Tero Tapani Karras, Samuli Matias Laine, David Patrick Luebke, Jaakko T. Lehtinen, Miika Samuli Aittala, Timo Oskari Aila, Ming-Yu Liu, Arun Mohanray Mallya, Ting-Chun Wang
  • Publication number: 20210042503
    Abstract: A latent code defined in an input space is processed by the mapping neural network to produce an intermediate latent code defined in an intermediate latent space. The intermediate latent code may be used as appearance vector that is processed by the synthesis neural network to generate an image. The appearance vector is a compressed encoding of data, such as video frames including a person's face, audio, and other data. Captured images may be converted into appearance vectors at a local device and transmitted to a remote device using much less bandwidth compared with transmitting the captured images. A synthesis neural network at the remote device reconstructs the images for display.
    Type: Application
    Filed: October 13, 2020
    Publication date: February 11, 2021
    Inventors: Tero Tapani Karras, Samuli Matias Laine, David Patrick Luebke, Jaakko T. Lehtinen, Miika Samuli Aittala, Timo Oskari Aila, Ming-Yu Liu, Arun Mohanray Mallya, Ting-Chun Wang
  • Patent number: 10866990
    Abstract: An apparatus, computer readable medium, and method are disclosed for decompressing compressed geometric data stored in a lossless compression format. The compressed geometric data resides within a compression block sized according to a system cache line. An indirection technique maps a global identifier value in a linear identifier space to corresponding variable rate compressed data. The apparatus may include decompression circuitry within a graphics processing unit configured to perform ray-tracing.
    Type: Grant
    Filed: July 3, 2019
    Date of Patent: December 15, 2020
    Assignee: NVIDIA Corporation
    Inventors: Jaakko Lehtinen, Timo Oskari Aila, Tero Tapani Karras, Alexander Keller, Nikolaus Binder, Carsten Alexander Waechter, Samuli Matias Laine
  • Publication number: 20200242739
    Abstract: A neural network architecture is disclosed for restoring noisy data. The neural network is a blind-spot network that can be trained according to a self-supervised framework. In an embodiment, the blind-spot network includes a plurality of network branches. Each network branch processes a version of the input data using one or more layers associated with kernels that have a receptive field that extends in a particular half-plane relative to the output value. In one embodiment, the versions of the input data are offset in a particular direction and the convolution kernels are rotated to correspond to the particular direction of the associated network branch. In another embodiment, the versions of the input data are rotated and the convolution kernel is the same for each network branch. The outputs of the network branches are composited to de-noise the image. In some embodiments, Bayesian filtering is performed to de-noise the input data.
    Type: Application
    Filed: December 20, 2019
    Publication date: July 30, 2020
    Inventors: Samuli Matias Laine, Tero Tapani Karras, Jaakko T. Lehtinen, Timo Oskari Aila
  • Publication number: 20200151559
    Abstract: A style-based generative network architecture enables scale-specific control of synthesized output data, such as images. During training, the style-based generative neural network (generator neural network) includes a mapping network and a synthesis network. During prediction, the mapping network may be omitted, replicated, or evaluated several times. The synthesis network may be used to generate highly varied, high-quality output data with a wide variety of attributes. For example, when used to generate images of people's faces, the attributes that may vary are age, ethnicity, camera viewpoint, pose, face shape, eyeglasses, colors (eyes, hair, etc.), hair style, lighting, background, etc. Depending on the task, generated output data may include images, audio, video, three-dimensional (3D) objects, text, etc.
    Type: Application
    Filed: May 21, 2019
    Publication date: May 14, 2020
    Inventors: Tero Tapani Karras, Timo Oskari Aila, Samuli Matias Laine
  • Publication number: 20190324991
    Abstract: An apparatus, computer readable medium, and method are disclosed for decompressing compressed geometric data stored in a lossless compression format. The compressed geometric data resides within a compression block sized according to a system cache line. An indirection technique maps a global identifier value in a linear identifier space to corresponding variable rate compressed data. The apparatus may include decompression circuitry within a graphics processing unit configured to perform ray-tracing.
    Type: Application
    Filed: July 3, 2019
    Publication date: October 24, 2019
    Inventors: Jaakko Lehtinen, Timo Oskari Aila, Tero Tapani Karras, Alexander Keller, Nikolaus Binder, Carsten Alexander Waechter, Samuli Matias Laine
  • Patent number: 10331632
    Abstract: A system, method, and computer program product are provided for modifying a hierarchical tree data structure. An initial hierarchical tree data structure is received and treelets of node neighborhoods in the initial hierarchical tree data structure are formed. Each treelet includes n leaf nodes and n?1 internal nodes. The treelets are restructured, by a processor, to produce an optimized hierarchical tree data structure.
    Type: Grant
    Filed: August 19, 2013
    Date of Patent: June 25, 2019
    Assignee: NVIDIA Corporation
    Inventors: Tero Tapani Karras, Timo Oskari Aila
  • Publication number: 20190171936
    Abstract: A neural network learns a particular task by being shown many examples. In one scenario, a neural network may be trained to label an image, such as cat, dog, bicycle, chair, etc. In other scenario, a neural network may be trained to remove noise from videos or identify specific objects within images, such as human faces, bicycles, etc. Rather than training a complex neural network having a predetermined topology of features and interconnections between the features to learn the task, the topology of the neural network is modified as the neural network is trained for the task, eventually evolving to match the predetermined topology of the complex neural network. In the beginning the neural network learns large-scale details for the task (bicycles have two wheels) and later, as the neural network becomes more complex, learns smaller details (the wheels have spokes).
    Type: Application
    Filed: January 18, 2019
    Publication date: June 6, 2019
    Inventors: Tero Tapani Karras, Timo Oskari Aila, Samuli Matias Laine, Jaakko T. Lehtinen, Janne Hellsten
  • Publication number: 20190130278
    Abstract: A generative adversarial neural network (GAN) learns a particular task by being shown many examples. In one scenario, a GAN may be trained to generate new images including specific objects, such as human faces, bicycles, etc. Rather than training a complex GAN having a predetermined topology of features and interconnections between the features to learn the task, the topology of the GAN is modified as the GAN is trained for the task. The topology of the GAN may be simple in the beginning and become more complex as the GAN learns during the training, eventually evolving to match the predetermined topology of the complex GAN. In the beginning the GAN learns large-scale details for the task (bicycles have two wheels) and later, as the GAN becomes more complex, learns smaller details (the wheels have spokes).
    Type: Application
    Filed: October 10, 2018
    Publication date: May 2, 2019
    Inventors: Tero Tapani Karras, Timo Oskari Aila, Samuli Matias Laine, Jaakko T. Lehtinen
  • Patent number: 10242485
    Abstract: An apparatus, computer readable medium, and method are disclosed for performing an intersection query between a query beam and a target bounding volume. The target bounding volume may comprise an axis-aligned bounding box (AABB) associated with a bounding volume hierarchy (BVH) tree. An intersection query comprising beam information associated with the query beam and slab boundary information for a first dimension of a target bounding volume is received. Intersection parameter values are calculated for the first dimension based on the beam information and the slab boundary information and a slab intersection case is determined for the first dimension based on the beam information. A parametric variable range for the first dimension is assigned based on the slab intersection case and the intersection parameter values and it is determined whether the query beam intersects the target bounding volume based on at least the parametric variable range for the first dimension.
    Type: Grant
    Filed: December 28, 2016
    Date of Patent: March 26, 2019
    Assignee: NVIDIA CORPORATION
    Inventors: Tero Tapani Karras, Timo Oskari Aila, Samuli Matias Laine, John Erik Lindholm
  • Publication number: 20180373809
    Abstract: A method, computer readable medium, and system are disclosed for performing traversal stack compression. The method includes traversing a hierarchical data structure having more than two children per node, and during the traversing, creating at least one stack entry, utilizing a processor, where each stack entry contains a plurality of intersected nodes, and adding the at least one stack entry to a compressed traversal stack stored in a memory, utilizing the processor.
    Type: Application
    Filed: January 25, 2018
    Publication date: December 27, 2018
    Inventors: Henri Johannes Ylitie, Tero Tapani Karras, Samuli Matias Laine