Patents by Inventor Arthur Mensch

Arthur Mensch has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20240119261
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence of discrete tokens using a diffusion model. In one aspect, a method includes generating, by using the diffusion model, a final latent representation of the sequence of discrete tokens that includes a determined value for each of a plurality of latent variables; applying a de-embedding matrix to the final latent representation of the output sequence of discrete tokens to generate a de-embedded final latent representation that includes, for each of the plurality of latent variables, a respective numeric score for each discrete token in a vocabulary of multiple discrete tokens; selecting, for each of the plurality of latent variables, a discrete token from among the multiple discrete tokens in the vocabulary that has a highest numeric score; and generating the output sequence of discrete tokens that includes the selected discrete tokens.
    Type: Application
    Filed: September 28, 2023
    Publication date: April 11, 2024
    Inventors: Robin Strudel, Rémi Leblond, Laurent Sifre, Sander Etienne Lea Dieleman, Nikolay Savinov, Will S. Grathwohl, Corentin Tallec, Florent Altché, Iaroslav Ganin, Arthur Mensch, Yilin Du
  • Publication number: 20230350936
    Abstract: A query processing system is described which receives a query input comprising an input token string and also at least one data item having a second, different modality, and generates a corresponding output token string.
    Type: Application
    Filed: April 28, 2023
    Publication date: November 2, 2023
    Inventors: Jean-Baptiste Alayrac, Jeffrey Donahue, Karel Lenc, Karen Simonyan, Malcolm Kevin Campbell Reynolds, Pauline Luc, Arthur Mensch, Iain Barr, Antoine Miech, Yana Elizabeth Hasson, Katherine Elizabeth Millican, Roman Ring
  • Publication number: 20230315532
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model to perform a machine learning task. In one aspect, a method performed by one or more computer is described. The method includes: obtaining data defining a compute budget that characterizes an amount of computing resources allocated for training a machine learning model to perform a machine learning task; processing the data defining the compute budget using an allocation mapping, in accordance with a set of allocation mapping parameters, to generate an allocation tuple defining: (i) a target model size for the machine learning model, and (ii) a target amount of training data for training the machine learning model; instantiating the machine learning model, where the machine learning model has the target model size; and obtaining the target amount of training data for training the machine learning model.
    Type: Application
    Filed: March 28, 2023
    Publication date: October 5, 2023
    Inventors: Jordan Hoffmann, Sebastian Borgeaud Dit Avocat, Laurent Sifre, Arthur Mensch
  • Publication number: 20230177309
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network having one or more conditional computation layers, where each conditional computation layer includes a gating sub-layer having multiple gating parameters and an expert sub-layer having multiple expert neural networks.
    Type: Application
    Filed: December 7, 2022
    Publication date: June 8, 2023
    Inventors: Aidan Clark, Arthur Mensch
  • Publication number: 20230177334
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a final output sequence. In one aspect, a method comprises: receiving a current output sequence comprising one or more current output segments; receiving a set of reference segments and a respective reference segment embedding of each reference segment that has been generated using an embedding neural network; for each current output segment: processing the current output segment using the embedding neural network to generate a current output segment embedding of the current output segment; and selecting k most similar reference segments to the current output segment using the reference segment embeddings and the current output segment embedding; and processing the current output sequence and the k most similar reference segments for each current output segment to generate an additional output segment that follows the current output sequence in the final output sequence.
    Type: Application
    Filed: December 7, 2022
    Publication date: June 8, 2023
    Inventors: Sebastian Borgeaud Dit Avocat, Laurent Sifre, Arthur Mensch, Jordan Hoffmann