Patents by Inventor Xiao Qi SHI

Xiao Qi SHI has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20230342619
    Abstract: Systems are methods are provided for training an automated agent. The automated agent maintains a reinforcement learning neural network and generates, according to outputs of the reinforcement learning neural network, signals for communicating resource task requests. First and second task data are received. The task data are processed to compute a first performance metric reflective of performance of the automated agent relative to other entities in a first time interval, and a second performance metric reflective of performance of the automated agent relative to other entities in a second time interval. A reward for the reinforcement learning neural network that reflects a difference between the second performance metric and the first performance metric is computed and provided to the reinforcement learning neural network to train the automated agent.
    Type: Application
    Filed: June 13, 2023
    Publication date: October 26, 2023
    Inventors: Hasham BURHANI, Shary MUDASSIR, Xiao Qi SHI, Connor LAWLESS, Weiguang DING
  • Patent number: 11715017
    Abstract: Systems are methods are provided for training an automated agent. The automated agent maintains a reinforcement learning neural network and generates, according to outputs of the reinforcement learning neural network, signals for communicating resource task requests. First and second task data are received. The task data are processed to compute a first performance metric reflective of performance of the automated agent relative to other entities in a first time interval, and a second performance metric reflective of performance of the automated agent relative to other entities in a second time interval. A reward for the reinforcement learning neural network that reflects a difference between the second performance metric and the first performance metric is computed and provided to the reinforcement learning neural network to train the automated agent.
    Type: Grant
    Filed: May 30, 2019
    Date of Patent: August 1, 2023
    Assignee: ROYAL BANK OF CANADA
    Inventors: Hasham Burhani, Shary Mudassir, Xiao Qi Shi, Connor Lawless, Weiguang Ding
  • Publication number: 20230066706
    Abstract: Systems, devices, and methods for training an automated agent are disclosed. Multiple automated agents are instantiated, each of the automated agents configured to train over a plurality of training cycles. For each resource, a dedicated portion of a memory device to store state data for the respective resource is allocated. The method includes receiving a request for state data for a particular resource from a subset of the automated agents; for each of the training cycles for the subset of the plurality of automated agents, storing updated state data for the particular resource in the dedicated portion of the memory device allocated to the particular resource; and transmitting an address of the dedicated portion of the memory device for the particular resource to the subset of the automated agents, to facilitate asynchronous reading of the stored state data for the particular resource during each training cycle.
    Type: Application
    Filed: August 25, 2021
    Publication date: March 2, 2023
    Inventors: Hasham BURHANI, Xiao Qi SHI, Kiarash JAMALI
  • Publication number: 20230063830
    Abstract: Systems, devices, and methods for automated generation of resource task requests are disclosed. A reinforcement learning neural network having an output layer with a plurality of policy heads is maintained. At least one reward is provided to the reinforcement learning neural network, the at least one reward corresponding to at least one prior resource task request generated based on outputs of the reinforcement learning neural network. State data are provided to the reinforcement learning neural network, the state data reflective of a current state of an environment in which resource task requests are made. A plurality of outputs is obtained, each from a corresponding policy head, the plurality of outputs including a first output defining a quantity of a resource and a second output defining a cost of the resource. A resource task request signal is generated based on the plurality of outputs from the plurality of policy heads.
    Type: Application
    Filed: August 23, 2022
    Publication date: March 2, 2023
    Inventors: Xiao Qi SHI, Hasham BURHANI, Daniel BALICKI
  • Publication number: 20230061752
    Abstract: Systems, devices, and methods for training an automated agent are disclosed. An automated agent is instantiated. The automated agent includes a reinforcement learning neural network that is trained over a plurality training cycles and provides a policy for generating resource task requests. A learning condition that is expected to impede training of the automated agent during a given training cycle of the plurality of training cycles is detected. In response to the detecting, a disable signal is generated to disable training of the automated agent for at least the given training cycle.
    Type: Application
    Filed: August 23, 2022
    Publication date: March 2, 2023
    Inventors: Xiao Qi Shi, Hasham Burhani
  • Publication number: 20230061206
    Abstract: Systems are methods are provided for training an automated agent. The automated agent maintains a reinforcement learning neural network and generates, according to outputs of the reinforcement learning neural network, signals for communicating resource task requests. The system includes a communication interface, a processor, memory, and software code stored in the memory. The software code, when executed, causes the system to: instantiate an automated agent that maintains the reinforcement learning neural network; receive current state data of a resource for a first task; receive historical state metrics of the resource computed based on a plurality of historical tasks; compute normalized state data based on the current state data; and provide the historical state metrics and the normalized state data to the reinforcement learning neural network of said automated agent for training.
    Type: Application
    Filed: August 25, 2021
    Publication date: March 2, 2023
    Inventors: Hasham BURHANI, Xiao Qi SHI
  • Publication number: 20230038434
    Abstract: Systems are methods are provided for training an automated agent. The automated agent maintains a reinforcement learning neural network and generates, according to outputs of the reinforcement learning neural network, signals for communicating resource task requests. The system includes a communication interface, a processor, memory, and software code stored in the memory.
    Type: Application
    Filed: August 9, 2021
    Publication date: February 9, 2023
    Inventors: Hasham BURHANI, Xiao Qi SHI
  • Publication number: 20190370649
    Abstract: Systems are methods are provided for training an automated agent. The automated agent maintains a reinforcement learning neural network and generates, according to outputs of the reinforcement learning neural network, signals for communicating resource task requests. First and second task data are received. The task data are processed to compute a first performance metric reflective of performance of the automated agent relative to other entities in a first time interval, and a second performance metric reflective of performance of the automated agent relative to other entities in a second time interval. A reward for the reinforcement learning neural network that reflects a difference between the second performance metric and the first performance metric is computed and provided to the reinforcement learning neural network to train the automated agent.
    Type: Application
    Filed: May 30, 2019
    Publication date: December 5, 2019
    Inventors: Hasham BURHANI, Shary MUDASSIR, Xiao Qi SHI, Connor LAWLESS