Patents by Inventor Jiquan Ngiam
Jiquan Ngiam has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Patent number: 11941875Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for processing a perspective view range image generated from sensor measurements of an environment. The perspective view range image includes a plurality of pixels arranged in a two-dimensional grid and including, for each pixel, (i) features of one or more sensor measurements at a location in the environment corresponding to the pixel and (ii) geometry information comprising range features characterizing a range of the location in the environment corresponding to the pixel relative to the one or more sensors. The system processes the perspective view range image using a first neural network to generate an output feature representation. The first neural network comprises a first perspective point-set aggregation layer comprising a geometry-dependent kernel.Type: GrantFiled: July 27, 2021Date of Patent: March 26, 2024Assignee: Waymo LLCInventors: Yuning Chai, Pei Sun, Jiquan Ngiam, Weiyue Wang, Vijay Vasudevan, Benjamin James Caine, Xiao Zhang, Dragomir Anguelov
-
Patent number: 11774596Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing data generated by a sensing system that rotationally senses an environment. In one aspect, a method comprises partitioning a predetermined period of time into a plurality of sub-periods, wherein the predetermined period of time is a period of time for which data generated by the sensing system constitutes a complete rotational sensing of the environment; for each sub-period: receiving current data generated by the sensing system during the sub-period and characterizing a respective partial scene of the environment; processing the current data using an object detection neural network to generate a current object detection output that is specific to the respective partial scene of the environment.Type: GrantFiled: September 1, 2022Date of Patent: October 3, 2023Assignee: Google LLCInventors: Jonathon Shlens, Vijay Vasudevan, Jiquan Ngiam, Wei Han, Zhifeng Chen, Brandon Chauloon Yang, Benjamin James Caine, Zhengdong Zhang, Christoph Sprunk, Ouais Alsharif, Junhua Mao, Chen Wu
-
Patent number: 11670038Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using dynamic voxelization. When deployed within an on-board system of a vehicle, processing the point cloud data using dynamic voxelization can be used to make autonomous driving decisions for the vehicle with enhanced accuracy, for example by combining representations of point cloud data characterizing a scene from multiple views of the scene.Type: GrantFiled: November 1, 2021Date of Patent: June 6, 2023Assignee: Waymo LLCInventors: Yin Zhou, Pei Sun, Yu Zhang, Dragomir Anguelov, Jiyang Gao, Yu Ouyang, Zijian Guo, Jiquan Ngiam, Vijay Vasudevan
-
Publication number: 20220415042Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing data generated by a sensing system that rotationally senses an environment. In one aspect, a method comprises partitioning a predetermined period of time into a plurality of sub-periods, wherein the predetermined period of time is a period of time for which data generated by the sensing system constitutes a complete rotational sensing of the environment; for each sub-period: receiving current data generated by the sensing system during the sub-period and characterizing a respective partial scene of the environment; processing the current data using an object detection neural network to generate a current object detection output that is specific to the respective partial scene of the environment.Type: ApplicationFiled: September 1, 2022Publication date: December 29, 2022Inventors: Jonathon Shlens, Vijay Vasudevan, Jiquan Ngiam, Wei Han, Zhifeng Chen, Brandon Chauloon Yang, Benjamin James Caine, Zhengdong Zhang, Christoph Sprunk, Ouais Alsharif, Junhua Mao, Chen Wu
-
Publication number: 20220383076Abstract: A method for performing one or more tasks, wherein each of the one or more tasks includes predicting behavior of one or more agents in an environment, the method comprising: obtaining a three-dimensional (3D) input tensor representing behaviors of the one or more agents in the environment across a plurality of time steps; generating an encoded representation of the 3D input tensor by processing the 3D input tensor using an encoder neural network, wherein 3D input tensor comprises a plurality of observed cells and a plurality of masked cells; and processing the encoded representation of the 3D input tensor using a decoder neural network to generate a 4D output tensor.Type: ApplicationFiled: May 31, 2022Publication date: December 1, 2022Inventors: Jonathon Shlens, Vijay Vasudevan, Jiquan Ngiam, Benjamin James Caine, Zhengdong Zhang, Zhifeng Chen, Hao-Tien Chiang, David Joseph Weiss, Jeffrey Ling, Ashish Venugopal
-
Patent number: 11508147Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing data generated by a sensing system that rotationally senses an environment. In one aspect, a method comprises partitioning a predetermined period of time into a plurality of sub-periods, wherein the predetermined period of time is a period of time for which data generated by the sensing system constitutes a complete rotational sensing of the environment; for each sub-period: receiving current data generated by the sensing system during the sub-period and characterizing a respective partial scene of the environment; processing the current data using an object detection neural network to generate a current object detection output that is specific to the respective partial scene of the environment.Type: GrantFiled: March 6, 2020Date of Patent: November 22, 2022Assignee: Google LLCInventors: Jonathon Shlens, Vijay Vasudevan, Jiquan Ngiam, Wei Han, Zhifeng Chen, Brandon Chauloon Yang, Benjamin James Caine, Zhengdong Zhang, Christoph Sprunk, Ouais Alsharif, Junhua Mao, Chen Wu
-
Patent number: 11450120Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data representing a sensor measurement of a scene captured by one or more sensors to generate an object detection output that identifies locations of one or more objects in the scene. When deployed within an on-board system of a vehicle, the object detection output that is generated can be used to make autonomous driving decisions for the vehicle with enhanced accuracy.Type: GrantFiled: July 8, 2020Date of Patent: September 20, 2022Assignee: Waymo LLCInventors: Jonathon Shlens, Patrick An Phu Nguyen, Benjamin James Caine, Jiquan Ngiam, Wei Han, Brandon Chauloon Yang, Yuning Chai, Pei Sun, Yin Zhou, Xi Yi, Ouais Alsharif, Zhifeng Chen, Vijay Vasudevan
-
Publication number: 20220180193Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform 3D object detection. One of the methods includes training a student neural network to perform 3D object detection using pseudo-labels generated by a teacher neural network.Type: ApplicationFiled: December 9, 2021Publication date: June 9, 2022Inventors: Benjamin James Caine, Rebecca Dawn Roelofs, Jonathon Shlens, Zhifeng Chen, Jiquan Ngiam, Vijay Vasudevan
-
Publication number: 20220129740Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using neural networks that include one or more conditional convolutional layers. A conditional convolutional layer has a plurality of kernels and determines a respective input-dependent weight for each of the plurality of kernels and generates an input-dependent kernel by computing a weighted sum of the plurality of kernels in accordance with the respective input-dependent weights.Type: ApplicationFiled: January 23, 2020Publication date: April 28, 2022Inventors: Brandon Chauloon Yang, Quoc V. Le, Jiquan Ngiam, Gabriel Mintzer Bender
-
Publication number: 20220121945Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training giant neural networks. One of the methods includes obtaining data specifying a partitioning of the neural network into N composite layers that form a sequence of composite layers, wherein each composite layer comprises a distinct plurality of layers from the multiple network layers of the neural network; obtaining data assigning each of the N composite layers to one or more computing devices from a set of N computing devices; partitioning a mini-batch of training examples into a plurality of micro-batches; and training the neural network, comprising: performing a forward pass through the neural network until output activations have been computed for each micro-batch for a final composite layer in the sequence, and performing a backward pass through the neural network until output gradients have been computed for each micro-batch for the first composite layer in the sequence.Type: ApplicationFiled: January 3, 2022Publication date: April 21, 2022Inventors: Zhifeng Chen, Yanping Huang, Youlong Cheng, HyoukJoong Lee, Dehao Chen, Jiquan Ngiam
-
Publication number: 20220058858Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using dynamic voxelization. When deployed within an on-board system of a vehicle, processing the point cloud data using dynamic voxelization can be used to make autonomous driving decisions for the vehicle with enhanced accuracy, for example by combining representations of point cloud data characterizing a scene from multiple views of the scene.Type: ApplicationFiled: November 1, 2021Publication date: February 24, 2022Inventors: Yin Zhou, Pei Sun, Yu Zhang, Dragomir Anguelov, Jiyang Gao, Yu Ouyang, Zijian Guo, Jiquan Ngiam, Vijay Vasudevan
-
Publication number: 20220044068Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for processing a perspective view range image generated from sensor measurements of an environment. The perspective view range image includes a plurality of pixels arranged in a two-dimensional grid and including, for each pixel, (i) features of one or more sensor measurements at a location in the environment corresponding to the pixel and (ii) geometry information comprising range features characterizing a range of the location in the environment corresponding to the pixel relative to the one or more sensors. The system processes the perspective view range image using a first neural network to generate an output feature representation. The first neural network comprises a first perspective point-set aggregation layer comprising a geometry-dependent kernel.Type: ApplicationFiled: July 27, 2021Publication date: February 10, 2022Inventors: Yuning Chai, Pei Sun, Jiquan Ngiam, Weiyue Wang, Vijay Vasudevan, Benjamin James Caine, Xiao Zhang, Dragomir Anguelov
-
Patent number: 11232356Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training giant neural networks. One of the methods includes obtaining data specifying a partitioning of the neural network into N composite layers that form a sequence of composite layers, wherein each composite layer comprises a distinct plurality of layers from the multiple network layers of the neural network; obtaining data assigning each of the N composite layers to one or more computing devices from a set of N computing devices; partitioning a mini-batch of training examples into a plurality of micro-batches; and training the neural network, comprising: performing a forward pass through the neural network until output activations have been computed for each micro-batch for a final composite layer in the sequence, and performing a backward pass through the neural network until output gradients have been computed for each micro-batch for the first composite layer in the sequence.Type: GrantFiled: August 10, 2020Date of Patent: January 25, 2022Assignee: Google LLCInventors: Zhifeng Chen, Yanping Huang, Youlong Cheng, HyoukJoong Lee, Dehao Chen, Jiquan Ngiam
-
Patent number: 11164363Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using dynamic voxelization. When deployed within an on-board system of a vehicle, processing the point cloud data using dynamic voxelization can be used to make autonomous driving decisions for the vehicle with enhanced accuracy, for example by combining representations of point cloud data characterizing a scene from multiple views of the scene.Type: GrantFiled: July 8, 2020Date of Patent: November 2, 2021Assignee: Waymo LLCInventors: Yin Zhou, Pei Sun, Yu Zhang, Dragomir Anguelov, Jiyang Gao, Yu Ouyang, Zijian Guo, Jiquan Ngiam, Vijay Vasudevan
-
Publication number: 20210334651Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model to perform a machine learning task by processing input data to the model. For example, the input data can include image, video, or point cloud data, and the task can be a perception task such as classification or detection task. In one aspect, the method includes receiving training data including a plurality of training inputs; receiving a plurality of data augmentation policy parameters that define different transformation operations for transforming training inputs before the training inputs are used to train the machine learning model; maintaining a plurality of candidate machine learning models; for each of the plurality of candidate machine learning models: repeatedly determining an augmented batch of training data; training the candidate machine learning model using the augmented batch of the training data; and updating the maintained data.Type: ApplicationFiled: March 5, 2021Publication date: October 28, 2021Inventors: Zhaoqi Leng, Ekin Dogus Cubuk, Barret Zoph, Jiquan Ngiam, Congcong Li, Jonathon Shlens, Shuyang Cheng
-
Publication number: 20210279465Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing data generated by a sensing system that rotationally senses an environment. In one aspect, a method comprises partitioning a predetermined period of time into a plurality of sub-periods, wherein the predetermined period of time is a period of time for which data generated by the sensing system constitutes a complete rotational sensing of the environment; for each sub-period: receiving current data generated by the sensing system during the sub-period and characterizing a respective partial scene of the environment; processing the current data using an object detection neural network to generate a current object detection output that is specific to the respective partial scene of the environment.Type: ApplicationFiled: March 6, 2020Publication date: September 9, 2021Inventors: Jonathon Shlens, Vijay Vasudevan, Jiquan Ngiam, Wei Han, Zhifeng Chen, Brandon Chauloon Yang, Benjamin James Caine, Zhengdong Zhang, Christoph Sprunk, Ouais Alsharif, Junhua Mao, Chen Wu
-
Publication number: 20210042620Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training giant neural networks. One of the methods includes obtaining data specifying a partitioning of the neural network into N composite layers that form a sequence of composite layers, wherein each composite layer comprises a distinct plurality of layers from the multiple network layers of the neural network; obtaining data assigning each of the N composite layers to one or more computing devices from a set of N computing devices; partitioning a mini-batch of training examples into a plurality of micro-batches; and training the neural network, comprising: performing a forward pass through the neural network until output activations have been computed for each micro-batch for a final composite layer in the sequence, and performing a backward pass through the neural network until output gradients have been computed for each micro-batch for the first composite layer in the sequence.Type: ApplicationFiled: August 10, 2020Publication date: February 11, 2021Inventors: Zhifeng Chen, Yanping Huang, Youlong Cheng, HyoukJoong Lee, Dehao Chen, Jiquan Ngiam
-
Publication number: 20210012555Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using dynamic voxelization. When deployed within an on-board system of a vehicle, processing the point cloud data using dynamic voxelization can be used to make autonomous driving decisions for the vehicle with enhanced accuracy, for example by combining representations of point cloud data characterizing a scene from multiple views of the scene.Type: ApplicationFiled: July 8, 2020Publication date: January 14, 2021Inventors: Yin Zhou, Pei Sun, Yu Zhang, Dragomir Anguelov, Jiyang Gao, Yu Ouyang, Zijian Guo, Jiquan Ngiam, Vijay Vasudevan
-
Publication number: 20210012089Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data representing a sensor measurement of a scene captured by one or more sensors to generate an object detection output that identifies locations of one or more objects in the scene. When deployed within an on-board system of a vehicle, the object detection output that is generated can be used to make autonomous driving decisions for the vehicle with enhanced accuracy.Type: ApplicationFiled: July 8, 2020Publication date: January 14, 2021Inventors: Jonathon Shlens, Patrick An Phu Nguyen, Benjamin James Caine, Jiquan Ngiam, Wei Han, Brandon Chauloon Yang, Yuning Chai, Pei Sun, Yin Zhou, Xi Yi, Ouais Alsharif, Zhifeng Chen, Vijay Vasudevan
-
Patent number: 10796592Abstract: According to an implementation, a system includes an online education platform including a content manager configured to provide an authoring tool on a computing device associated with a learner of an online course. The authoring tool is configured to provide at least one user interface for creation of a learner-created question for an assessment for the online course. The content manager includes an education content converter configured to convert the learner-created question from a first format to a second format. The online education platform including an assessment bank configured to store the learner-created question, as well as other learner-created questions and instructor-created questions. The online education platform including a content selector configured to select a plurality of questions from the assessment bank for the assessment for the online course.Type: GrantFiled: December 20, 2017Date of Patent: October 6, 2020Assignee: Coursera, Inc.Inventors: Jacob K. Samuelson, Myra Liu, Jiquan Ngiam, Mustafa Furniturewala