Patents by Inventor Jakub Konecny
Jakub Konecny has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20230376856Abstract: The present disclosure provides efficient communication techniques for transmission of model updates within a machine learning framework, such as, for example, a federated learning framework in which a high-quality centralized model is trained on training data distributed overt a large number of clients each with unreliable network connections and low computational power. In an example federated learning setting, in each of a plurality of rounds, each client independently updates the model based on its local data and communicates the updated model back to the server, where all the client-side updates are used to update a global model. The present disclosure provides systems and methods that reduce communication costs. In particular, the present disclosure provides at least: structured update approaches in which the model update is restricted to be small and sketched update approaches in which the model update is compressed before sending to the server.Type: ApplicationFiled: August 4, 2023Publication date: November 23, 2023Inventors: Hugh Brendan McMahan, Dave Morris Bacon, Jakub Konecny, Xinnan Yu
-
Patent number: 11763197Abstract: The present disclosure provides efficient communication techniques for transmission of model updates within a machine learning framework, such as, for example, a federated learning framework in which a high-quality centralized model is trained on training data distributed overt a large number of clients each with unreliable network connections and low computational power. In an example federated learning setting, in each of a plurality of rounds, each client independently updates the model based on its local data and communicates the updated model back to the server, where all the client-side updates are used to update a global model. The present disclosure provides systems and methods that reduce communication costs. In particular, the present disclosure provides at least: structured update approaches in which the model update is restricted to be small and sketched update approaches in which the model update is compressed before sending to the server.Type: GrantFiled: April 16, 2020Date of Patent: September 19, 2023Assignee: GOOGLE LLCInventors: Hugh Brendan McMahan, Dave Morris Bacon, Jakub Konecny, Xinnan Yu
-
Publication number: 20210382962Abstract: Systems and methods of determining a global model are provided. In particular, one or more local updates can be received from a plurality of user devices. Each local update can be determined by the respective user device based at least in part on one or more data examples stored on the user device. The one or more data examples stored on the plurality of user devices are distributed on an uneven basis, such that no user device includes a representative sample of the overall distribution of data examples. The local updates can then be aggregated to determine a global model.Type: ApplicationFiled: August 19, 2021Publication date: December 9, 2021Inventors: Hugh Brendan McMahan, Jakub Konecny, Eider Brantly Moore, Daniel Ramage, Blaise H. Aguera-Arcas
-
Patent number: 11120102Abstract: Systems and methods of determining a global model are provided. In particular, one or more local updates can be received from a plurality of user devices. Each local update can be determined by the respective user device based at least in part on one or more data examples stored on the user device. The one or more data examples stored on the plurality of user devices are distributed on an uneven basis, such that no user device includes a representative sample of the overall distribution of data examples. The local updates can then be aggregated to determine a global model.Type: GrantFiled: August 27, 2020Date of Patent: September 14, 2021Assignee: Google LLCInventors: Hugh Brendan McMahan, Jakub Konecny, Eider Brantly Moore, Daniel Ramage, Blaise H. Aguera-Arcas
-
Patent number: 11023561Abstract: Systems and methods of determining a global model are provided. In particular, one or more local updates can be received from a plurality of user devices. Each local update can be determined by the respective user device based at least in part on one or more data examples stored on the user device. The one or more data examples stored on the plurality of user devices are distributed on an uneven basis, such that no user device includes a representative sample of the overall distribution of data examples. The local updates can then be aggregated to determine a global model.Type: GrantFiled: September 3, 2019Date of Patent: June 1, 2021Assignee: Google LLCInventors: Hugh Brendan McMahan, Jakub Konecny, Eider Brantley Moore, Daniel R. Ramage, Blaise H. Aguera-Arcas
-
Publication number: 20210073639Abstract: A computing system and method can be used to implement a version of federated learning (FL) that incorporates adaptivity (e.g., leverages an adaptive learning rate). In particular, the present disclosure provides a general optimization framework in which (1) clients perform multiple epochs of training using a client optimizer to minimize loss on their local data and (2) a server system updates its global model by applying a gradient-based server optimizer to the average of the clients' model updates. This framework can seamlessly incorporate adaptivity by using adaptive optimizers as client and/or server optimizers. Building upon this general framework, the present disclosure also provides example specific adaptive optimization techniques for FL which use per-coordinate methods as server optimizers. By focusing on adaptive server optimization, the use of adaptive learning rates is enabled without increase in client storage or communication costs and compatibility with cross-device FL can be ensured.Type: ApplicationFiled: November 20, 2020Publication date: March 11, 2021Inventors: Sashank Jakkam Reddi, Sanjiv Kumar, Manzil Zaheer, Zachary Charles, Zach Garrett, Keith Rush, Jakub Konecny, Hugh Brendan McMahan
-
Publication number: 20200394253Abstract: Systems and methods of determining a global model are provided. In particular, one or more local updates can be received from a plurality of user devices. Each local update can be determined by the respective user device based at least in part on one or more data examples stored on the user device. The one or more data examples stored on the plurality of user devices are distributed on an uneven basis, such that no user device includes a representative sample of the overall distribution of data examples. The local updates can then be aggregated to determine a global model.Type: ApplicationFiled: August 27, 2020Publication date: December 17, 2020Inventors: Hugh Brendan McMahan, Jakub Konecny, Eider Brantly Moore, Daniel Ramage, Blaise H. Aguera-Arcas
-
Publication number: 20200242514Abstract: The present disclosure provides efficient communication techniques for transmission of model updates within a machine learning framework, such as, for example, a federated learning framework in which a high-quality centralized model is trained on training data distributed overt a large number of clients each with unreliable network connections and low computational power. In an example federated learning setting, in each of a plurality of rounds, each client independently updates the model based on its local data and communicates the updated model back to the server, where all the client-side updates are used to update a global model. The present disclosure provides systems and methods that reduce communication costs. In particular, the present disclosure provides at least: structured update approaches in which the model update is restricted to be small and sketched update approaches in which the model update is compressed before sending to the server.Type: ApplicationFiled: April 16, 2020Publication date: July 30, 2020Inventors: Hugh Brendan McMahan, Dave Morris Bacon, Jakub Konecny, Xinnan Yu
-
Patent number: 10657461Abstract: The present disclosure provides efficient communication techniques for transmission of model updates within a machine learning framework, such as, for example, a federated learning framework in which a high-quality centralized model is trained on training data distributed overt a large number of clients each with unreliable network connections and low computational power. In an example federated learning setting, in each of a plurality of rounds, each client independently updates the model based on its local data and communicates the updated model back to the server, where all the client-side updates are used to update a global model. The present disclosure provides systems and methods that reduce communication costs. In particular, the present disclosure provides at least: structured update approaches in which the model update is restricted to be small and sketched update approaches in which the model update is compressed before sending to the server.Type: GrantFiled: September 7, 2017Date of Patent: May 19, 2020Assignee: Google LLCInventors: Hugh Brendan McMahan, Dave Morris Bacon, Jakub Konecny, Xinnan Yu
-
Publication number: 20200004801Abstract: Systems and methods of determining a global model are provided. In particular, one or more local updates can be received from a plurality of user devices. Each local update can be determined by the respective user device based at least in part on one or more data examples stored on the user device. The one or more data examples stored on the plurality of user devices are distributed on an uneven basis, such that no user device includes a representative sample of the overall distribution of data examples. The local updates can then be aggregated to determine a global model.Type: ApplicationFiled: September 3, 2019Publication date: January 2, 2020Inventors: Hugh Brendan McMahan, Jakub Konecny, Eider Brantley Moore, Daniel R. Ramage, Blaise H. Aguera-Arcas
-
Publication number: 20190340534Abstract: The present disclosure provides efficient communication techniques for transmission of model updates within a machine learning framework, such as, for example, a federated learning framework in which a high-quality centralized model is trained on training data distributed overt a large number of clients each with unreliable network connections and low computational power. In an example federated learning setting, in each of a plurality of rounds, each client independently updates the model based on its local data and communicates the updated model back to the server, where all the client-side updates are used to update a global model. The present disclosure provides systems and methods that reduce communication costs. In particular, the present disclosure provides at least: structured update approaches in which the model update is restricted to be small and sketched update approaches in which the model update is compressed before sending to the server.Type: ApplicationFiled: September 7, 2017Publication date: November 7, 2019Inventors: Hugh Brendan McMahan, Dave Morris Bacon, Jakub Konecny, Xinnan Yu
-
Patent number: 10402469Abstract: Systems and methods of determining a global model are provided. In particular, one or more local updates can be received from a plurality of user devices. Each local update can be determined by the respective user device based at least in part on one or more data examples stored on the user device. The one or more data examples stored on the plurality of user devices are distributed on an uneven basis, such that no user device includes a representative sample of the overall distribution of data examples. The local updates can then be aggregated to determine a global model.Type: GrantFiled: February 17, 2016Date of Patent: September 3, 2019Assignee: Google LLCInventors: Hugh Brendan McMahan, Jakub Konecny, Eider Brantly Moore, Daniel R. Ramage, Blaise H. Aguera-Arcas
-
Publication number: 20170109322Abstract: Systems and methods of determining a global model are provided. In particular, one or more local updates can be received from a plurality of user devices. Each local update can be determined by the respective user device based at least in part on one or more data examples stored on the user device. The one or more data examples stored on the plurality of user devices are distributed on an uneven basis, such that no user device includes a representative sample of the overall distribution of data examples. The local updates can then be aggregated to determine a global model.Type: ApplicationFiled: February 17, 2016Publication date: April 20, 2017Inventors: Hugh Brendan McMahan, Jakub Konecny, Eider Brantly Moore, Daniel Ramage, Blaise H. Aguera-Arcas