Patents by Inventor Andrew Gerald Howard
Andrew Gerald Howard has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240119256Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: ApplicationFiled: October 13, 2023Publication date: April 11, 2024Inventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Patent number: 11922288Abstract: A neural network system is configured to receive an input image and to generate a classification output for the input image. The neural network system includes: a separable convolution subnetwork comprising a plurality of separable convolutional neural network layers arranged in a stack one after the other, in which each separable convolutional neural network layer is configured to: separately apply both a depthwise convolution and a pointwise convolution during processing of an input to the separable convolutional neural network layer to generate a layer output.Type: GrantFiled: February 27, 2023Date of Patent: March 5, 2024Assignee: Google LLCInventors: Francois Chollet, Andrew Gerald Howard
-
Patent number: 11823024Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: GrantFiled: July 22, 2021Date of Patent: November 21, 2023Assignee: GOOGLE LLCInventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Publication number: 20230297852Abstract: Example implementations of the present disclosure combine efficient model design and dynamic inference. With a standalone lightweight model, the unnecessary computation on easy examples is avoided and the information extracted by the lightweight model also guide the synthesis of a specialist network from the basis models. With extensive experiments on ImageNet it is shown that a proposed example BasisNet is particularly effective for image classification and a BasisNet-MV3 achieves 80.3% top-1 accuracy with 290 M MAdds without early termination.Type: ApplicationFiled: July 29, 2021Publication date: September 21, 2023Inventors: Li Zhang, Andrew Gerald Howard, Brendan Wesley Jou, Yukun Zhu, Mingda Zhang, Andrey Zhmoginov
-
Publication number: 20230267307Abstract: Systems and methods of the present disclosure are directed to a method for generating a machine-learned multitask model configured to perform tasks. The method can include obtaining a machine-learned multitask search model comprising candidate nodes. The method can include obtaining tasks and machine-learned task controller models associated with the tasks. As an example, for a task, the method can include using the task controller model to route a subset of the candidate nodes in a machine-learned task submodel for the corresponding task. The method can include inputting task input data to the task submodel to obtain a task output. The method can include generating, using the task output, a feedback value based on an objective function. The method can include adjusting parameters of the task controller model based on the feedback value.Type: ApplicationFiled: July 23, 2020Publication date: August 24, 2023Inventors: Qifei Wang, Junjie Ke, Grace Chu, Gabriel Mintzer Bender, Luciano Sbaiz, Feng Yang, Andrew Gerald Howard, Alec Michael Go, Jeffrey M. Gilbert, Peyman Milanfar, Joshua William Charles Greaves
-
Publication number: 20230267330Abstract: The present disclosure provides systems and methods that enable parameter-efficient transfer learning, multi-task learning, and/or other forms of model re-purposing such as model personalization or domain adaptation. In particular, as one example, a computing system can obtain a machine-learned model that has been previously trained on a first training dataset to perform a first task. The machine-learned model can include a first set of learnable parameters. The computing system can modify the machine-learned model to include a model patch, where the model patch includes a second set of learnable parameters. The computing system can train the machine-learned model on a second training dataset to perform a second task that is different from the first task, which may include learning new values for the second set of learnable parameters included in the model patch while keeping at least some (e.g., all) of the first set of parameters fixed.Type: ApplicationFiled: May 2, 2023Publication date: August 24, 2023Inventors: Mark Sandler, Andrew Gerald Howard, Andrey Zhmoginov, Pramod Kaushik Mudrakarta
-
Patent number: 11734545Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: GrantFiled: February 17, 2018Date of Patent: August 22, 2023Assignee: GOOGLE LLCInventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Publication number: 20230237314Abstract: A neural network system is configured to receive an input image and to generate a classification output for the input image. The neural network system includes: a separable convolution subnetwork comprising a plurality of separable convolutional neural network layers arranged in a stack one after the other, in which each separable convolutional neural network layer is configured to: separately apply both a depthwise convolution and a pointwise convolution during processing of an input to the separable convolutional neural network layer to generate a layer output.Type: ApplicationFiled: February 27, 2023Publication date: July 27, 2023Inventors: Francois Chollet, Andrew Gerald Howard
-
Patent number: 11676008Abstract: The present disclosure provides systems and methods that enable parameter-efficient transfer learning, multi-task learning, and/or other forms of model re-purposing such as model personalization or domain adaptation. In particular, as one example, a computing system can obtain a machine-learned model that has been previously trained on a first training dataset to perform a first task. The machine-learned model can include a first set of learnable parameters. The computing system can modify the machine-learned model to include a model patch, where the model patch includes a second set of learnable parameters. The computing system can train the machine-learned model on a second training dataset to perform a second task that is different from the first task, which may include learning new values for the second set of learnable parameters included in the model patch while keeping at least some (e.g., all) of the first set of parameters fixed.Type: GrantFiled: September 20, 2019Date of Patent: June 13, 2023Assignee: GOOGLE LLCInventors: Mark Sandler, Andrey Zhmoginov, Andrew Gerald Howard, Pramod Kaushik Mudrakarta
-
Publication number: 20230091374Abstract: The present disclosure is directed to object and/or character recognition for use in applications such as computer vision. Advantages of the present disclosure include lightweight functionality that can be used on devices such as smart phones. Aspects of the present disclosure include a sequential architecture where a lightweight machine-learned model can receive an image, detect whether an object is present in one or more regions of the image, and generate an output based on the detection. This output can be applied as a filter to remove image data that can be neglected for more memory intensive machine-learned models applied downstream.Type: ApplicationFiled: February 24, 2020Publication date: March 23, 2023Inventors: Qifei Wang, Alexander Kuznetsov, Alec Michael Go, Grace Chu, Eunyoung Kim, Feng Yang, Andrew Gerald Howard, Jeffrey M. Gilbert
-
Patent number: 11593614Abstract: A neural network system is configured to receive an input image and to generate a classification output for the input image. The neural network system includes: a separable convolution subnetwork comprising a plurality of separable convolutional neural network layers arranged in a stack one after the other, in which each separable convolutional neural network layer is configured to: separately apply both a depthwise convolution and a pointwise convolution during processing of an input to the separable convolutional neural network layer to generate a layer output.Type: GrantFiled: October 6, 2017Date of Patent: February 28, 2023Assignee: Google LLCInventors: Francois Chollet, Andrew Gerald Howard
-
Publication number: 20210350206Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: ApplicationFiled: July 22, 2021Publication date: November 11, 2021Inventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Patent number: 11157814Abstract: The present disclosure provides systems and methods to reduce computational costs associated with convolutional neural networks. In addition, the present disclosure provides a class of efficient models termed “MobileNets” for mobile and embedded vision applications. MobileNets are based on a straight-forward architecture that uses depthwise separable convolutions to build light weight deep neural networks. The present disclosure further provides two global hyper-parameters that efficiently trade-off between latency and accuracy. These hyper-parameters allow the entity building the model to select the appropriately sized model for the particular application based on the constraints of the problem. MobileNets and associated computational cost reduction techniques are effective across a wide range of applications and use cases.Type: GrantFiled: September 18, 2017Date of Patent: October 26, 2021Assignee: Google LLCInventors: Andrew Gerald Howard, Bo Chen, Dmitry Kalenichenko, Tobias Christoph Weyand, Menglong Zhu, Marco Andreetto, Weijun Wang
-
Patent number: 11157815Abstract: The present disclosure provides systems and methods to reduce computational costs associated with convolutional neural networks. In addition, the present disclosure provides a class of efficient models termed “MobileNets” for mobile and embedded vision applications. MobileNets are based on a straight-forward architecture that uses depthwise separable convolutions to build light weight deep neural networks. The present disclosure further provides two global hyper-parameters that efficiently trade-off between latency and accuracy. These hyper-parameters allow the entity building the model to select the appropriately sized model for the particular application based on the constraints of the problem. MobileNets and associated computational cost reduction techniques are effective across a wide range of applications and use cases.Type: GrantFiled: July 29, 2019Date of Patent: October 26, 2021Assignee: Google LLCInventors: Andrew Gerald Howard, Bo Chen, Dmitry Kalenichenko, Tobias Christoph Weyand, Menglong Zhu, Marco Andreetto, Weijun Wang
-
Publication number: 20210027140Abstract: A neural network system is configured to receive an input image and to generate a classification output for the input image. The neural network system includes: a separable convolution subnetwork comprising a plurality of separable convolutional neural network layers arranged in a stack one after the other, in which each separable convolutional neural network layer is configured to: separately apply both a depthwise convolution and a pointwise convolution during processing of an input to the separable convolutional neural network layer to generate a layer output.Type: ApplicationFiled: October 6, 2017Publication date: January 28, 2021Inventors: Francois Chollet, Andrew Gerald Howard
-
Publication number: 20200104706Abstract: The present disclosure provides systems and methods that enable parameter-efficient transfer learning, multi-task learning, and/or other forms of model re-purposing such as model personalization or domain adaptation. In particular, as one example, a computing system can obtain a machine-learned model that has been previously trained on a first training dataset to perform a first task. The machine-learned model can include a first set of learnable parameters. The computing system can modify the machine-learned model to include a model patch, where the model patch includes a second set of learnable parameters. The computing system can train the machine-learned model on a second training dataset to perform a second task that is different from the first task, which may include learning new values for the second set of learnable parameters included in the model patch while keeping at least some (e.g., all) of the first set of parameters fixed.Type: ApplicationFiled: September 20, 2019Publication date: April 2, 2020Inventors: Mark Sandler, Andrey Zhmoginov, Andrew Gerald Howard, Pramod Kaushik Mudrakarta
-
Publication number: 20190347537Abstract: The present disclosure provides systems and methods to reduce computational costs associated with convolutional neural networks. In addition, the present disclosure provides a class of efficient models termed “MobileNets” for mobile and embedded vision applications. MobileNets are based on a straight-forward architecture that uses depthwise separable convolutions to build light weight deep neural networks. The present disclosure further provides two global hyper-parameters that efficiently trade-off between latency and accuracy. These hyper-parameters allow the entity building the model to select the appropriately sized model for the particular application based on the constraints of the problem. MobileNets and associated computational cost reduction techniques are effective across a wide range of applications and use cases.Type: ApplicationFiled: July 29, 2019Publication date: November 14, 2019Inventors: Andrew Gerald Howard, Bo Chen, Dmitry Kalenichenko, Tobias Christoph Weyand, Menglong Zhu, Marco Andreetto, Weijun Wang
-
Publication number: 20190147318Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: ApplicationFiled: February 17, 2018Publication date: May 16, 2019Inventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Publication number: 20180137406Abstract: The present disclosure provides systems and methods to reduce computational costs associated with convolutional neural networks. In addition, the present disclosure provides a class of efficient models termed “MobileNets” for mobile and embedded vision applications. MobileNets are based on a straight-forward architecture that uses depthwise separable convolutions to build light weight deep neural networks. The present disclosure further provides two global hyper-parameters that efficiently trade-off between latency and accuracy. These hyper-parameters allow the entity building the model to select the appropriately sized model for the particular application based on the constraints of the problem. MobileNets and associated computational cost reduction techniques are effective across a wide range of applications and use cases.Type: ApplicationFiled: September 18, 2017Publication date: May 17, 2018Inventors: Andrew Gerald Howard, Bo Chen, Dmitry Kalenichenko, Tobias Christoph Weyand, Menglong Zhu, Marco Andreetto, Weijun Wang