Patents by Inventor Menglong Zhu
Menglong Zhu has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11961298Abstract: Systems and methods for detecting objects in a video are provided. A method can include inputting a video comprising a plurality of frames into an interleaved object detection model comprising a plurality of feature extractor networks and a shared memory layer. For each of one or more frames, the operations can include selecting one of the plurality of feature extractor networks to analyze the one or more frames, analyzing the one or more frames by the selected feature extractor network to determine one or more features of the one or more frames, determining an updated set of features based at least in part on the one or more features and one or more previously extracted features extracted from a previous frame stored in the shared memory layer, and detecting an object in the one or more frames based at least in part on the updated set of features.Type: GrantFiled: February 22, 2019Date of Patent: April 16, 2024Assignee: GOOGLE LLCInventors: Menglong Zhu, Mason Liu, Marie Charisse White, Dmitry Kalenichenko, Yinxiao Li
-
Publication number: 20240119256Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: ApplicationFiled: October 13, 2023Publication date: April 11, 2024Inventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Patent number: 11823024Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: GrantFiled: July 22, 2021Date of Patent: November 21, 2023Assignee: GOOGLE LLCInventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Patent number: 11734545Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: GrantFiled: February 17, 2018Date of Patent: August 22, 2023Assignee: GOOGLE LLCInventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Publication number: 20220189170Abstract: Systems and methods for detecting objects in a video are provided. A method can include inputting a video comprising a plurality of frames into an interleaved object detection model comprising a plurality of feature extractor networks and a shared memory layer. For each of one or more frames, the operations can include selecting one of the plurality of feature extractor networks to analyze the one or more frames, analyzing the one or more frames by the selected feature extractor network to determine one or more features of the one or more frames, determining an updated set of features based at least in part on the one or more features and one or more previously extracted features extracted from a previous frame stored in the shared memory layer, and detecting an object in the one or more frames based at least in part on the updated set of features.Type: ApplicationFiled: February 22, 2019Publication date: June 16, 2022Inventors: Menglong Zhu, Mason Liu, Marie Charisse White, Dmitry Kalenichenko, Yinxiao Li
-
Publication number: 20210350206Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: ApplicationFiled: July 22, 2021Publication date: November 11, 2021Inventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Patent number: 11157814Abstract: The present disclosure provides systems and methods to reduce computational costs associated with convolutional neural networks. In addition, the present disclosure provides a class of efficient models termed “MobileNets” for mobile and embedded vision applications. MobileNets are based on a straight-forward architecture that uses depthwise separable convolutions to build light weight deep neural networks. The present disclosure further provides two global hyper-parameters that efficiently trade-off between latency and accuracy. These hyper-parameters allow the entity building the model to select the appropriately sized model for the particular application based on the constraints of the problem. MobileNets and associated computational cost reduction techniques are effective across a wide range of applications and use cases.Type: GrantFiled: September 18, 2017Date of Patent: October 26, 2021Assignee: Google LLCInventors: Andrew Gerald Howard, Bo Chen, Dmitry Kalenichenko, Tobias Christoph Weyand, Menglong Zhu, Marco Andreetto, Weijun Wang
-
Patent number: 11157815Abstract: The present disclosure provides systems and methods to reduce computational costs associated with convolutional neural networks. In addition, the present disclosure provides a class of efficient models termed “MobileNets” for mobile and embedded vision applications. MobileNets are based on a straight-forward architecture that uses depthwise separable convolutions to build light weight deep neural networks. The present disclosure further provides two global hyper-parameters that efficiently trade-off between latency and accuracy. These hyper-parameters allow the entity building the model to select the appropriately sized model for the particular application based on the constraints of the problem. MobileNets and associated computational cost reduction techniques are effective across a wide range of applications and use cases.Type: GrantFiled: July 29, 2019Date of Patent: October 26, 2021Assignee: Google LLCInventors: Andrew Gerald Howard, Bo Chen, Dmitry Kalenichenko, Tobias Christoph Weyand, Menglong Zhu, Marco Andreetto, Weijun Wang
-
Patent number: 10713491Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing object detection. In one aspect, a method includes receiving multiple video frames. The video frames are sequentially processed using an object detection neural network to generate an object detection output for each video frame. The object detection neural network includes a convolutional neural network layer and a recurrent neural network layer. For each video frame after an initial video frame, processing the video frame using the object detection neural network includes generating a spatial feature map for the video frame using the convolutional neural network layer and generating a spatio-temporal feature map for the video frame using the recurrent neural network layer.Type: GrantFiled: July 27, 2018Date of Patent: July 14, 2020Assignee: Google LLCInventors: Menglong Zhu, Mason Liu
-
Publication number: 20200034627Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing object detection. In one aspect, a method includes receiving multiple video frames. The video frames are sequentially processed using an object detection neural network to generate an object detection output for each video frame. The object detection neural network includes a convolutional neural network layer and a recurrent neural network layer. For each video frame after an initial video frame, processing the video frame using the object detection neural network includes generating a spatial feature map for the video frame using the convolutional neural network layer and generating a spatio-temporal feature map for the video frame using the recurrent neural network layer.Type: ApplicationFiled: July 27, 2018Publication date: January 30, 2020Inventors: Menglong Zhu, Mason Liu
-
Publication number: 20190347537Abstract: The present disclosure provides systems and methods to reduce computational costs associated with convolutional neural networks. In addition, the present disclosure provides a class of efficient models termed “MobileNets” for mobile and embedded vision applications. MobileNets are based on a straight-forward architecture that uses depthwise separable convolutions to build light weight deep neural networks. The present disclosure further provides two global hyper-parameters that efficiently trade-off between latency and accuracy. These hyper-parameters allow the entity building the model to select the appropriately sized model for the particular application based on the constraints of the problem. MobileNets and associated computational cost reduction techniques are effective across a wide range of applications and use cases.Type: ApplicationFiled: July 29, 2019Publication date: November 14, 2019Inventors: Andrew Gerald Howard, Bo Chen, Dmitry Kalenichenko, Tobias Christoph Weyand, Menglong Zhu, Marco Andreetto, Weijun Wang
-
Publication number: 20190147318Abstract: The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.Type: ApplicationFiled: February 17, 2018Publication date: May 16, 2019Inventors: Andrew Gerald Howard, Mark Sandler, Liang-Chieh Chen, Andrey Zhmoginov, Menglong Zhu
-
Publication number: 20180137406Abstract: The present disclosure provides systems and methods to reduce computational costs associated with convolutional neural networks. In addition, the present disclosure provides a class of efficient models termed “MobileNets” for mobile and embedded vision applications. MobileNets are based on a straight-forward architecture that uses depthwise separable convolutions to build light weight deep neural networks. The present disclosure further provides two global hyper-parameters that efficiently trade-off between latency and accuracy. These hyper-parameters allow the entity building the model to select the appropriately sized model for the particular application based on the constraints of the problem. MobileNets and associated computational cost reduction techniques are effective across a wide range of applications and use cases.Type: ApplicationFiled: September 18, 2017Publication date: May 17, 2018Inventors: Andrew Gerald Howard, Bo Chen, Dmitry Kalenichenko, Tobias Christoph Weyand, Menglong Zhu, Marco Andreetto, Weijun Wang
-
Patent number: 8831290Abstract: Poses of a movable camera relative to an environment are obtained by determining point correspondences from a set of initial images and then applying 2-point motion estimation to the point correspondences to determine a set of initial poses of the camera. A point cloud is generated from the set of initial poses and the point correspondences. Then, for each next image, the point correspondences and corresponding poses are determined, while updating the point cloud.Type: GrantFiled: August 1, 2012Date of Patent: September 9, 2014Assignee: Mitsubishi Electric Research Laboratories, Inc.Inventors: Srikumar Ramalingam, Yuichi Taguchi, Menglong Zhu
-
Publication number: 20140037136Abstract: Poses of a movable camera relative to an environment are obtained by determining point correspondences from a set of initial images and then applying 2-point motion estimation to the point correspondences to determine a set of initial poses of the camera. A point cloud is generated from the set of initial poses and the point correspondences. Then, for each next image, the point correspondences and corresponding poses are determined, while updating the point cloud.Type: ApplicationFiled: August 1, 2012Publication date: February 6, 2014Inventors: Srikumar Ramalingam, Yuichi Taguchi, Menglong Zhu