Patents by Inventor Cheng Ran Li

Cheng Ran Li has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Pre-warming scheme to load machine learning models
    Patent number: 11562288
    Abstract: Techniques for hosting adding and warming a host are described. In some instances, a method of determining that at least one group of hosts is to be increased by adding an additional host to the group of hosts; sending a request to the group of hosts for a list of machine learning models loaded per host of the group of hosts; receiving, from each host, the list of loaded machine learning models; loading at least a proper subset of list of loaded machine learning models into random access memory of the at least one group; receiving a request to perform an inference; routing the request to the additional host of the group of hosts; performing an inference using the additional host of the group of hosts; and providing a result of the inference to an external entity is described.
    Type: Grant
    Filed: September 28, 2018
    Date of Patent: January 24, 2023
    Assignee: Amazon Technologies, Inc.
    Inventors: Enrico Sartorello, Stefano Stefani, Nikhil Kandoi, Rama Krishna Sandeep Pokkunuri, Kalpesh N. Sutaria, Navneet Sabbineni, Ganesh Kumar Gella, Cheng Ran Li
  • PRE-WARMING SCHEME TO LOAD MACHINE LEARNING MODELS
    Publication number: 20200104749
    Abstract: Techniques for hosting adding and warming a host are described. In some instances, a method of determining that at least one group of hosts is to be increased by adding an additional host to the group of hosts; sending a request to the group of hosts for a list of machine learning models loaded per host of the group of hosts; receiving, from each host, the list of loaded machine learning models; loading at least a proper subset of list of loaded machine learning models into random access memory of the at least one group; receiving a request to perform an inference; routing the request to the additional host of the group of hosts; performing an inference using the additional host of the group of hosts; and providing a result of the inference to an external entity is described.
    Type: Application
    Filed: September 28, 2018
    Publication date: April 2, 2020
    Inventors: Enrico Sartorello, Stefano Stefani, Nikhil Kandoi, Rama Krishna Sandeep Pokkunuri, Kalpesh N. Sutaria, Navneet Sabbineni, Ganesh Kumar Gella, Cheng Ran Li