Patents by Inventor Sidharth Mudgal

Sidharth Mudgal has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Blockwise controlled decoding of natural language (NL) based output generated using a large language model (LLM) to reduce latency in rendering thereof
    Patent number: 12681964
    Abstract: Implementations relate to reducing latency in generating and/or rendering a given stream of natural language (NL) based output generated using a large language model (LLM). Processor(s) of a system can: receive NL based input associated with a client device, generate the stream of NL based output utilizing the LLM that is responsive to the NL based input and that is for a given dialog context of an ongoing dialog, and cause the stream of NL based output to be rendered at the client device. Notably, the processor(s) can employ attribute classifier(s) and a multi-objective scorer to implement a blockwise controlled decoding technique in generating the stream of NL based output utilizing the LLM. By implementing the blockwise controlled decoding technique in generating the stream of NL based output utilizing the LLM, the processor(s) can reduce latency in generating and/or of the stream of NL based output generated utilizing the LLM.
    Type: Grant
    Filed: July 25, 2023
    Date of Patent: July 14, 2026
    Assignee: GOOGLE LLC
    Inventors: Sidharth Mudgal, Ahmad Beirami, Jilin Chen, Alex Beutel, Harish Ganapathy, YaGuang Li, Tao Wang, Yanping Huang, Trevor Strohman
  • Streaming of natural language (NL) based output generated using a large language model (LLM) to reduce latency in rendering thereof
    Patent number: 12602408
    Abstract: Implementations relate to reducing latency in generating and/or rendering natural language (NL) output generated using a large language model (LLM). Processor(s) of a system can: receive NL based input associated with a client device, and generate the NL based output utilizing the LLM. The NL based output can be a stream of NL based output in that it includes a plurality of segments, and is generated on a segment-by-segment basis. In some implementations, a first segment of the stream of NL based output is selected for inclusion in the stream of NL based output as a second segment (and any subsequent segment) is being generated to reduce latency in evaluating the NL based output as a whole prior to rendering thereof. In some versions of those implementations, the first segment is rendered as the second segment (and any subsequent segment) is being generated to further reduce latency in rendering thereof.
    Type: Grant
    Filed: April 19, 2023
    Date of Patent: April 14, 2026
    Assignee: GOOGLE LLC
    Inventors: Martin Baeuml, Yanping Huang, Wenhao Jia, Chang Lan, Yuanzhong Xu, Junwhan Ahn, Alexander Bailey, Leif Schelin, Trevor Strohman, Emanuel Taropa, Sidharth Mudgal, Yanyan Zheng, Zhifeng Chen, Ahmad Beirami
  • BLOCKWISE CONTROLLED DECODING OF NATURAL LANGUAGE (NL) BASED OUTPUT GENERATED USING A LARGE LANGUAGE MODEL (LLM) TO REDUCE LATENCY IN RENDERING THEREOF
    Publication number: 20240330334
    Abstract: Implementations relate to reducing latency in generating and/or rendering a given stream of natural language (NL) based output generated using a large language model (LLM). Processor(s) of a system can: receive NL based input associated with a client device, generate the stream of NL based output utilizing the LLM that is responsive to the NL based input and that is for a given dialog context of an ongoing dialog, and cause the stream of NL based output to be rendered at the client device. Notably, the processor(s) can employ attribute classifier(s) and a multi-objective scorer to implement a blockwise controlled decoding technique in generating the stream of NL based output utilizing the LLM. By implementing the blockwise controlled decoding technique in generating the stream of NL based output utilizing the LLM, the processor(s) can reduce latency in generating and/or of the stream of NL based output generated utilizing the LLM.
    Type: Application
    Filed: July 25, 2023
    Publication date: October 3, 2024
    Inventors: Sidharth Mudgal, Ahmad Beirami, Jilin Chen, Alex Beutel, Harish Ganapathy, YaGuang Li, Tao Wang, Yanping Huang, Trevor Strohman
  • STREAMING OF NATURAL LANGUAGE (NL) BASED OUTPUT GENERATED USING A LARGE LANGUAGE MODEL (LLM) TO REDUCE LATENCY IN RENDERING THEREOF
    Publication number: 20240311402
    Abstract: Implementations relate to reducing latency in generating and/or rendering natural language (NL) output generated using a large language model (LLM). Processor(s) of a system can: receive NL based input associated with a client device, and generate the NL based output utilizing the LLM. The NL based output can be a stream of NL based output in that it includes a plurality of segments, and is generated on a segment-by-segment basis. In some implementations, a first segment of the stream of NL based output is selected for inclusion in the stream of NL based output as a second segment (and any subsequent segment) is being generated to reduce latency in evaluating the NL based output as a whole prior to rendering thereof. In some versions of those implementations, the first segment is rendered as the second segment (and any subsequent segment) is being generated to further reduce latency in rendering thereof.
    Type: Application
    Filed: April 19, 2023
    Publication date: September 19, 2024
    Inventors: Martin Baeuml, Yanping Huang, Wenhao Jia, Chang Lan, Yuanzhong Xu, Junwhan Ahn, Alexander Bailey, Leif Schelin, Trevor Strohman, Emanuel Taropa, Sidharth Mudgal, Yanyan Zheng, Zhifeng Chen, Ahmad Beirami