Patents by Inventor Saurabh M. KULKARNI
Saurabh M. KULKARNI has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11675654Abstract: Embodiments of the present disclosure include an error recovery method comprising detecting a computing error, restarting a first artificial intelligence processor of a plurality of artificial intelligence processors processing a data set, and loading a model in the artificial intelligence processor, wherein the model corresponds to a same model processed by the plurality of artificial intelligence processors during a previous processing iteration by the plurality of artificial intelligence processors on data from the data set.Type: GrantFiled: December 16, 2021Date of Patent: June 13, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Bharadwaj Pudipeddi, Maral Mesmakhosroshahi, Jinwen Xi, Saurabh M. Kulkarni, Marc Tremblay, Matthias Baenninger, Nuno Claudino Pereira Lopes
-
Patent number: 11663444Abstract: Systems and methods for pipelined neural network processing with continuous and asynchronous updates are described. A method for processing a neural network comprising L layers, where L is an integer greater than two, includes partitioning the L layers among a set of computing resources configured to process forward passes and backward passes associated with each of the L layers. The method further includes initiating processing of the forward passes and the backward passes using the set of computing resources. The method further includes upon completion of a first set of forward passes and a first set of backward passes associated with a first layer of the L layers, initiating update of parameters associated with the first layer when gradients are available for updating the parameters associated with the first layer without waiting to calculate gradients associated with any of remaining L layers.Type: GrantFiled: September 27, 2019Date of Patent: May 30, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Andy Wagner, Tiyasa Mitra, Saurabh M. Kulkarni, Marc Tremblay, Sujeeth S. Bharadwaj
-
Patent number: 11562047Abstract: A method of increasing computer hardware efficiency of a matrix computation. The method comprises receiving at a computer processing device, digital signals encoding one or more operations of the matrix computation, each operation including one or more operands. The method further comprises, responsive to determining, by a sparse data check device of the computer processing machine, that an operation of the matrix computation includes all dense operands, forwarding the operation to a dense computation device of the computer processing machine configured to perform the operation of the matrix computation based on the dense operands. The method further comprises, responsive to determining, by the sparse data check device, that an operation of the matrix computation includes one or more sparse operands, forwarding the operation to a sparse computation device configured to perform the operation of the matrix computation.Type: GrantFiled: April 29, 2020Date of Patent: January 24, 2023Assignee: Microsoft Technology Licensing, LLCInventors: Layali Rashid, Saurabh M. Kulkarni, Marc Tremblay
-
Patent number: 11449752Abstract: Methods for gradient accumulation with free momentum are performed by systems and devices during neural network model training. An accumulator that includes a processor circuit and a memory element generates free momentum between passes of a neural network model training process. The processor circuit receives a difference weight (gradient) and generates a first input by applying a weighting parameter thereto. The processor circuit obtains a prior weight from the memory element and generates a second input by applying another weighting parameter thereto. The processor circuit generates a filtered input with momentum by filtering the first and second input. The memory element generates a stored next pass weight by accumulating the filtered input with the prior weight. A computing resource then processes the next pass of the neural network model training using the stored next pass weight. The methods, systems, and devices are applicable to pipelined model parallelism training processes.Type: GrantFiled: March 31, 2020Date of Patent: September 20, 2022Assignee: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: Andrew Wagner, Marc Tremblay, Saurabh M. Kulkarni, Tiyasa Mitra, Sujeeth S. Bharadwaj
-
Publication number: 20220107864Abstract: Embodiments of the present disclosure include an error recovery method comprising detecting a computing error, restarting a first artificial intelligence processor of a plurality of artificial intelligence processors processing a data set, and loading a model in the artificial intelligence processor, wherein the model corresponds to a same model processed by the plurality of artificial intelligence processors during a previous processing iteration by the plurality of artificial intelligence processors on data from the data set.Type: ApplicationFiled: December 16, 2021Publication date: April 7, 2022Inventors: Bharadwaj PUDIPEDDI, Maral MESMAKHOSROSHAHI, Jinwen XI, Saurabh M. KULKARNI, Marc TREMBLAY, Matthias BAENNINGER, Nuno CLAUDINO PEREIRA LOPES
-
Patent number: 11226859Abstract: Embodiments of the present disclosure include an error recovery method comprising detecting a computing error, restarting a first artificial intelligence processor of a plurality of artificial intelligence processors processing a data set, and loading a model in the artificial intelligence processor, wherein the model corresponds to a same model processed by the plurality of artificial intelligence processors during a previous processing iteration by the plurality of artificial intelligence processors on data from the data set.Type: GrantFiled: March 27, 2020Date of Patent: January 18, 2022Assignee: Microsoft Technology Licensing, LLCInventors: Bharadwaj Pudipeddi, Maral Mesmakhosroshahi, Jinwen Xi, Saurabh M. Kulkarni, Marc Tremblay, Matthias Baenninger, Nuno Claudino Pereira Lopes
-
Patent number: 11163887Abstract: A bare metal resource includes a trusted portion and an untrusted portion. The trusted portion includes trusted hardware, an image repository, and a clearance manager. The clearance manager is executable during bootup of the bare metal resource to perform a clearance process on the untrusted portion, including deleting the BIOS in the untrusted portion and loading a trusted BIOS from the image repository on the untrusted hardware, to place the untrusted portion in a trusted state. The bare metal resource may be provisioned to a tenant of a cloud provider after being placed in the trusted state.Type: GrantFiled: December 28, 2018Date of Patent: November 2, 2021Assignee: MICROSOFT TECHNOLOGY LICENSING, LLCInventors: Bryan W. Tuttle, Carlos Jose Cela, Ho-Yuen Chau, Melur K. Raghuraman, Saurabh M. Kulkarni, Yimin Deng
-
Publication number: 20210303991Abstract: Methods for gradient accumulation with free momentum are performed by systems and devices during neural network model training. An accumulator that includes a processor circuit and a memory element generates free momentum between passes of a neural network model training process. The processor circuit receives a difference weight (gradient) and generates a first input by applying a weighting parameter thereto. The processor circuit obtains a prior weight from the memory element and generates a second input by applying another weighting parameter thereto. The processor circuit generates a filtered input with momentum by filtering the first and second input. The memory element generates a stored next pass weight by accumulating the filtered input with the prior weight. A computing resource then processes the next pass of the neural network model training using the stored next pass weight. The methods, systems, and devices are applicable to pipelined model parallelism training processes.Type: ApplicationFiled: March 31, 2020Publication date: September 30, 2021Inventors: Andrew Wagner, Marc Tremblay, Saurabh M. Kulkarni, Tiyasa Mitra, Sujeeth S. Bharadwaj
-
Publication number: 20210240797Abstract: A method of increasing computer hardware efficiency of a matrix computation. The method comprises receiving at a computer processing device, digital signals encoding one or more operations of the matrix computation, each operation including one or more operands. The method further comprises, responsive to determining, by a sparse data check device of the computer processing machine, that an operation of the matrix computation includes all dense operands, forwarding the operation to a dense computation device of the computer processing machine configured to perform the operation of the matrix computation based on the dense operands. The method further comprises, responsive to determining, by the sparse data check device, that an operation of the matrix computation includes one or more sparse operands, forwarding the operation to a sparse computation device configured to perform the operation of the matrix computation.Type: ApplicationFiled: April 29, 2020Publication date: August 5, 2021Applicant: Microsoft Technology Licensing, LLCInventors: Layali RASHID, Saurabh M. KULKARNI, Marc TREMBLAY
-
Publication number: 20210232451Abstract: Embodiments of the present disclosure include an error recovery method comprising detecting a computing error, restarting a first artificial intelligence processor of a plurality of artificial intelligence processors processing a data set, and loading a model in the artificial intelligence processor, wherein the model corresponds to a same model processed by the plurality of artificial intelligence processors during a previous processing iteration by the plurality of artificial intelligence processors on data from the data set.Type: ApplicationFiled: March 27, 2020Publication date: July 29, 2021Inventors: Bharadwaj PUDIPEDDI, Maral MESMAKHOSROSHAHI, Jinwen XI, Saurabh M. KULKARNI, Marc TREMBLAY, Matthias BAENNINGER, Nuno CLAUDINO PEREIRA LOPES
-
Publication number: 20210097366Abstract: Systems and methods for pipelined neural network processing with continuous and asynchronous updates are described. A method for processing a neural network comprising L layers, where L is an integer greater than two, includes partitioning the L layers among a set of computing resources configured to process forward passes and backward passes associated with each of the L layers. The method further includes initiating processing of the forward passes and the backward passes using the set of computing resources. The method further includes upon completion of a first set of forward passes and a first set of backward passes associated with a first layer of the L layers, initiating update of parameters associated with the first layer when gradients are available for updating the parameters associated with the first layer without waiting to calculate gradients associated with any of remaining L layers.Type: ApplicationFiled: September 27, 2019Publication date: April 1, 2021Inventors: Andy Wagner, Tiyasa Mitra, Saurabh M. Kulkarni, Marc Tremblay, Sujeeth S. Bharadwaj
-
Publication number: 20190251266Abstract: A bare metal resource includes a trusted portion and an untrusted portion. The trusted portion includes trusted hardware, an image repository, and a clearance manager. The clearance manager is executable during bootup of the bare metal resource to perform a clearance process on the untrusted portion, including deleting the BIOS in the untrusted portion and loading a trusted BIOS from the image repository on the untrusted hardware, to place the untrusted portion in a trusted state. The bare metal resource may be provisioned to a tenant of a cloud provider after being placed in the trusted state.Type: ApplicationFiled: December 28, 2018Publication date: August 15, 2019Applicant: Microsoft Technology Licensing, LLCInventors: Bryan W. TUTTLE, Carlos Jose CELA, Ho-Yuen CHAU, Melur K. RAGHURAMAN, Saurabh M. KULKARNI, Yimin DENG