Patents by Inventor Lucas Theis

Lucas Theis has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20230378975
    Abstract: A method of encoding data includes determining, by an encoder, a first data instance by corrupting the data with Gaussian noise. The method also includes determining, by the encoder, information representative of one or more conditional distributions. The method additionally includes determining, by the encoder, an index of a corrupted data instance of the sequence of progressively less corrupted data instances. The index corresponds with a conditional distribution of the one or more conditional distributions which causes the corrupted data instance to have a desired bit-rate. The method further includes transmitting, from the encoder to a decoder, the first data instance and the information representative of the one or more conditional distributions to enable the decoder to recover the corrupted data instance having the desired bit-rate and use the corrupted data instance to generate output data representative of the data.
    Type: Application
    Filed: May 18, 2023
    Publication date: November 23, 2023
    Inventor: Lucas Theis
  • Patent number: 11537869
    Abstract: Systems and methods provide a learned difference metric that operates in a wide artifact space. An example method includes initializing a committee of deep neural networks with labeled distortion pairs, iteratively actively learning a difference metric using the committee and psychophysics tasks for informative distortion pairs, and using the difference metric as an objective function in a machine-learned digital file processing task. Iteratively actively learning the difference metric can include providing an unlabeled distortion pair as input to each of the deep neural networks in the committee, a distortion pair being a base image and a distorted image resulting from application of an artifact applied to the base image, obtaining a plurality of difference metric scores for the unlabeled distortion pair from the deep neural networks, and identifying the unlabeled distortion pair as an informative distortion pair when the difference metric scores satisfy a diversity metric.
    Type: Grant
    Filed: December 27, 2017
    Date of Patent: December 27, 2022
    Assignee: Twitter, Inc.
    Inventors: Ferenc Huszar, Lucas Theis, Pietro Berkes
  • Patent number: 11386599
    Abstract: A method for developing visual data using source data, target data, and a hierarchical algorithm. According to a first aspect, there is provided a method for developing visual data from source data, target data and using a hierarchical algorithm, the method comprising the steps of: determining an alignment between the target data and the source data; and producing the visual data by transferring one or more features of the source data onto one or more features of the target data; wherein, the visual data is produced after the step of determining the alignment between the target data and the source data; and wherein the visual data is produced using the hierarchical algorithm.
    Type: Grant
    Filed: May 13, 2020
    Date of Patent: July 12, 2022
    Assignee: Twitter, Inc.
    Inventors: Lucas Theis, Zehan Wang, Robert David Bishop
  • Patent number: 11308361
    Abstract: An example system includes a processor and a memory. The system performs sub-pixel convolution that is free of checkerboard artifacts. In one example implementation, the system may execute a method that includes initializing one or more parameters of a sub-kernel of a kernel and copying the one or more parameters of the sub-kernel to other sub-kernels of the kernel. The method may further include performing convolution of an input image with the sub-kernels of the kernel and generating a plurality of first output images. A second output image is then generated based on the plurality of first output images.
    Type: Grant
    Filed: July 5, 2018
    Date of Patent: April 19, 2022
    Assignee: Twitter, Inc.
    Inventors: Andrew Aitken, Christian Ledig, Lucas Theis, Jose Caballero, Zehan Wang, Wenzhe Shi
  • Patent number: 11234006
    Abstract: Methods and systems for optimising the quality of visual data. Specifically, methods and systems for preserving visual information during compression and decompression. An example method for optimising visual data includes using a pre-processing neural network to optimise visual data prior to encoding the visual data in visual data processing; and using a post-processing neural network to enhance visual data following decoding visual data in visual data processing.
    Type: Grant
    Filed: May 13, 2020
    Date of Patent: January 25, 2022
    Assignee: Magic Pony Technology Limited
    Inventors: Zehan Wang, Robert David Bishop, Ferenc Huszar, Lucas Theis
  • Publication number: 20210264568
    Abstract: A neural network is trained to process received visual data to estimate a high-resolution version of the visual data using a training dataset and reference dataset. A set of training data is generated, and a generator convolutional neural network parameterized by first weights and biases is trained by comparing characteristics of the training data to characteristics of the reference dataset. The first network is trained to generate super-resolved image data from low-resolution image data and the training includes modifying first weights and biases to optimize processed visual data based on the comparison between the characteristics of the training data and the characteristics of the reference dataset.
    Type: Application
    Filed: May 5, 2021
    Publication date: August 26, 2021
    Inventors: Wenzhe Shi, Christian Ledig, Zehan Wang, Lucas Theis, Ferenc Huszar
  • Patent number: 11024009
    Abstract: A neural network is trained to process received visual data to estimate a high-resolution version of the visual data using a training dataset and reference dataset. A set of training data is generated and a generator convolutional neural network parameterized by first weights and biases is trained by comparing characteristics of the training data to characteristics of the reference dataset. The first network is trained to generate super-resolved image data from low-resolution image data and the training includes modifying first weights and biases to optimize processed visual data based on the comparison between the characteristics of the training data and the characteristics of the reference dataset.
    Type: Grant
    Filed: September 15, 2017
    Date of Patent: June 1, 2021
    Assignee: Twitter, Inc.
    Inventors: Wenzhe Shi, Christian Ledig, Zehan Wang, Lucas Theis, Ferenc Huszar
  • Patent number: 10909743
    Abstract: Generating texture maps for use in rendering visual output. According to a first aspect, there is provided a method for generating textures for use in rendering visual output, the method comprising the steps of: generating, using a first hierarchical algorithm, a first texture from one or more sets of initialisation data; and selectively refining the first texture, using one or more further hierarchical algorithms, to generate one or more further textures from at least a section of the first texture and one or more sets of further initialisation data; wherein at least a section of each of the one or more further textures differs from the first texture.
    Type: Grant
    Filed: December 28, 2017
    Date of Patent: February 2, 2021
    Assignee: Magic Pony Technology Limited
    Inventors: Lucas Theis, Zehan Wang, Robert David Bishop
  • Patent number: 10825138
    Abstract: Systems and methods for developing improved-fidelity visual data using fidelity data and using a hierarchical algorithm are provided. An example method includes receiving at least a plurality of neighbouring sections of visual data, selecting a plurality of input sections from the received plurality of neighbouring sections of visual data, extracting features from the plurality of input sections of visual data, and producing the improved-fidelity visual data by applying the fidelity data to the extracted features.
    Type: Grant
    Filed: December 28, 2017
    Date of Patent: November 3, 2020
    Assignee: Magic Pony Technology Limited
    Inventors: Zehan Wang, Robert David Bishop, Lucas Theis
  • Publication number: 20200280730
    Abstract: Methods and systems for optimising the quality of visual data. Specifically, methods and systems for preserving visual information during compression and decompression. An example method for optimising visual data includes using a pre-processing neural network to optimise visual data prior to encoding the visual data in visual data processing; and using a post-processing neural network to enhance visual data following decoding visual data in visual data processing.
    Type: Application
    Filed: May 13, 2020
    Publication date: September 3, 2020
    Inventors: Zehan Wang, Robert David Bishop, Ferenc Huszar, Lucas Theis
  • Publication number: 20200273224
    Abstract: A method for developing visual data using source data, target data, and a hierarchical algorithm. According to a first aspect, there is provided a method for developing visual data from source data, target data and using a hierarchical algorithm, the method comprising the steps of: determining an alignment between the target data and the source data; and producing the visual data by transferring one or more features of the source data onto one or more features of the target data; wherein, the visual data is produced after the step of determining the alignment between the target data and the source data; and wherein the visual data is produced using the hierarchical algorithm.
    Type: Application
    Filed: May 13, 2020
    Publication date: August 27, 2020
    Inventors: Lucas Theis, Zehan Wang, Robert David Bishop
  • Patent number: 10701394
    Abstract: A method includes selecting a plurality of low-resolution frames associated with a video, performing a first motion estimation between a first frame and a second frame, performing a second motion estimation between a third frame and the second frame, generating a high-resolution frame representing the second frame based on the first motion estimation, the second motion estimation and the second frame using a sub-pixel convolutional neural network.
    Type: Grant
    Filed: November 10, 2017
    Date of Patent: June 30, 2020
    Assignee: Twitter, Inc.
    Inventors: Jose Caballero, Christian Ledig, Andrew Aitken, Alfredo Alejandro Acosta Diaz, Lucas Theis, Ferenc Huszar, Johannes Totz, Zehan Wang, Wenzhe Shi
  • Patent number: 10699456
    Abstract: A method for developing visual data using source data, target data, and a hierarchical algorithm. According to a first aspect, there is provided a method for developing visual data from source data, target data and using a hierarchical algorithm, the method comprising the steps of: determining an alignment between the target data and the source data; and producing the visual data by transferring one or more features of the source data onto one or more features of the target data; wherein, the visual data is produced after the step of determining the alignment between the target data and the source data; and wherein the visual data is produced using the hierarchical algorithm.
    Type: Grant
    Filed: December 28, 2017
    Date of Patent: June 30, 2020
    Assignee: MAGIC PONY TECHNOLOGY LIMITED
    Inventors: Lucas Theis, Zehan Wang, Robert David Bishop
  • Patent number: 10681361
    Abstract: Methods and systems for optimising the quality of visual data. Specifically, methods and systems for preserving visual information during compression and decompression. An example method for optimising visual data includes using a pre-processing hierarchical algorithm to optimise visual data prior to encoding the visual data in visual data processing; and using a post-processing hierarchical algorithm to enhance visual data following decoding visual data in visual data processing.
    Type: Grant
    Filed: December 27, 2017
    Date of Patent: June 9, 2020
    Assignee: Magic Pony Technology Limited
    Inventors: Zehan Wang, Robert David Bishop, Ferenc Huszar, Lucas Theis
  • Patent number: 10666962
    Abstract: Disclosed is method for training a plurality of visual processing algorithms for processing visual data. The method includes using a pre-processing hierarchical algorithm to process the visual data prior to encoding the visual data in visual data processing, and using a post-processing hierarchical algorithm to further process the visual data following decoding visual data in visual data processing. The encoding and decoding are performed with respect to a predetermined visual data codec and may be content specific.
    Type: Grant
    Filed: September 18, 2017
    Date of Patent: May 26, 2020
    Assignee: Magic Pony Technology Limited
    Inventors: Zehan Wang, Robert David Bishop, Ferenc Huszar, Lucas Theis
  • Patent number: 10623775
    Abstract: A system (e.g., an auto-encoder system) includes an encoder, a decoder and a learning module. The encoder generates compressed video data using a lossy compression algorithm, the lossy compression algorithm being implemented using a trained neural network with at least one convolution, generate at least one first parameter based on the compressed video data, and communicate the compressed video data and the model to at least one device configured to decode the compressed video data using an inverse algorithm based on the lossy compression algorithm. The decoder generates decoded video data based on the compressed video data using the inverse algorithm and the model, and generate at least one second parameter based on the decoded video data. The learning module trains the model using the at least one first parameter and the at least one second parameter.
    Type: Grant
    Filed: November 6, 2017
    Date of Patent: April 14, 2020
    Assignee: Twitter, Inc.
    Inventors: Lucas Theis, Ferenc Huszar, Zehan Wang, Wenzhe Shi
  • Patent number: 10552977
    Abstract: Systems and methods generate a face-swapped image from a target image using a convolutional neural network trained to apply a source identity to the expression and pose of the target image. The convolutional neural network produces face-swapped images fast enough to transform a video stream. An example method includes aligning the face portion of a target image from an original view to a reference view to generate a target face and generating a swapped face by changing the target face to that of a source identity using a convolutional neural network trained to minimize loss of content from the target face and style from the source identity. The method also includes realigning the swapped face from the reference view to the original view and generating a swapped image by stitching the realigned swapped face with the remaining portion of the target image.
    Type: Grant
    Filed: April 18, 2017
    Date of Patent: February 4, 2020
    Assignee: Twitter, Inc.
    Inventors: Lucas Theis, Iryna Korshunova, Wenzhe Shi, Zehan Wang
  • Publication number: 20180240017
    Abstract: Systems and methods provide a learned difference metric that operates in a wide artifact space. An example method includes initializing a committee of deep neural networks with labeled distortion pairs, iteratively actively learning a difference metric using the committee and psychophysics tasks for informative distortion pairs, and using the difference metric as an objective function in a machine-learned digital file processing task. Iteratively actively learning the difference metric can include providing an unlabeled distortion pair as input to each of the deep neural networks in the committee, a distortion pair being a base image and a distorted image resulting from application of an artifact applied to the base image, obtaining a plurality of difference metric scores for the unlabeled distortion pair from the deep neural networks, and identifying the unlabeled distortion pair as an informative distortion pair when the difference metric scores satisfy a diversity metric.
    Type: Application
    Filed: December 27, 2017
    Publication date: August 23, 2018
    Inventors: Ferenc Huszar, Lucas Theis, Pietro Berkes
  • Publication number: 20180144526
    Abstract: A method for developing visual data using source data, target data, and a hierarchical algorithm. According to a first aspect, there is provided a method for developing visual data from source data, target data and using a hierarchical algorithm, the method comprising the steps of: determining an alignment between the target data and the source data; and producing the visual data by transferring one or more features of the source data onto one or more features of the target data; wherein, the visual data is produced after the step of determining the alignment between the target data and the source data; and wherein the visual data is produced using the hierarchical algorithm.
    Type: Application
    Filed: December 28, 2017
    Publication date: May 24, 2018
    Inventors: Lucas Theis, Zehan Wang, Robert David Bishop
  • Publication number: 20180139458
    Abstract: Methods and systems for optimising the quality of visual data. Specifically, methods and systems for preserving visual information during compression and decompression. An example method for optimising visual data includes using a pre-processing hierarchical algorithm to optimise visual data prior to encoding the visual data in visual data processing; and using a post-processing hierarchical algorithm to enhance visual data following decoding visual data in visual data processing.
    Type: Application
    Filed: December 27, 2017
    Publication date: May 17, 2018
    Inventors: Zehan Wang, Robert David Bishop, Ferenc Huszar, Lucas Theis