Patents by Inventor Zoltan Csaki

Zoltan Csaki has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20250148276
    Abstract: Embodiments described herein provide systems and techniques for training large language models. In one aspect, a process for performing in-context training of a language model is disclosed. This process may begin by receiving a language model that includes a context window of a predetermined size, as well as receiving a set of in-context prompt/completion pairs prepared for a target task. The process then constructs a first token sequence based on the set of in-context prompt/completion pairs. Next, the process fits the first token sequence into the context window. The process subsequently performs a first in-context training pass using the first token sequence to train the language model to generate a next token in accordance with the target task.
    Type: Application
    Filed: November 6, 2023
    Publication date: May 8, 2025
    Applicant: SambaNova Systems, Inc.
    Inventors: Zoltan Csaki, Bo Li, Urmish Ajit Thakker, Venkat Krishna SRINIVASAN
  • Publication number: 20250148205
    Abstract: Embodiments described herein provide systems and techniques for training large language models. In one aspect, a process for performing in-context few-shot training for a transformer-based language model is disclosed. This process may begin by receiving the transformer-based language model having a context window of a predetermined size, as well as a training dataset comprising a set of prompt/completion examples. The process then constructs a training sequence based on the training dataset. Next, the process performs a single forward pass using the training sequence as input. The process subsequently performs a set of backward passes from a subset of examples in the training sequence, wherein each backward pass is conditioned on a selected subset of prompt/completion examples in the training sequence.
    Type: Application
    Filed: November 6, 2023
    Publication date: May 8, 2025
    Applicant: SambaNova Systems, Inc.
    Inventors: Zoltan Csaki, Bo Li, Urmish Ajit Thakker, Venkat Krishna SRINIVASAN