Patents by Inventor Wan-Yen Lo
Wan-Yen Lo has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240036583Abstract: Disclosed herein are systems and methods for providing supplemental identification abilities to an autonomous vehicle system. The sensor unit of the vehicle may be configured to receive data indicating an environment of the vehicle, while the control system may be configured to operate the vehicle. The vehicle may also include a processing unit configured to analyze the data indicating the environment to determine at least one object having a detection confidence below a threshold. Based on the at least one object having a detection confidence below a threshold, the processor may communicate at least a subset of the data indicating the environment for further processing. The vehicle is also configured to receive an indication of an object confirmation of the subset of the data. Based on the object confirmation of the subset of the data, the processor may alter the control of the vehicle by the control system.Type: ApplicationFiled: October 10, 2023Publication date: February 1, 2024Inventors: David Ian Ferguson, Wan-Yen Lo, Nathaniel Fairfield
-
Patent number: 11815903Abstract: Disclosed herein are systems and methods for providing supplemental identification abilities to an autonomous vehicle system. The sensor unit of the vehicle may be configured to receive data indicating an environment of the vehicle, while the control system may be configured to operate the vehicle. The vehicle may also include a processing unit configured to analyze the data indicating the environment to determine at least one object having a detection confidence below a threshold. Based on the at least one object having a detection confidence below a threshold, the processor may communicate at least a subset of the data indicating the environment for further processing. The vehicle is also configured to receive an indication of an object confirmation of the subset of the data. Based on the object confirmation of the subset of the data, the processor may alter the control of the vehicle by the control system.Type: GrantFiled: March 4, 2021Date of Patent: November 14, 2023Assignee: Waymo LLCInventors: David Ian Ferguson, Wan-Yen Lo, Nathaniel Fairfield
-
Patent number: 11783180Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating object predictions using a neural network. One of the methods includes receiving respective projections of a plurality of channels of input sensor data, wherein each channel of input sensor data represents different respective characteristics of electromagnetic radiation reflected off of one or more objects. Each of the projections of the plurality of channels of input sensor data are provided to a neural network subsystem trained to receive projections of input sensor data as input and to provide an object prediction as an output. At the output of the neural network subsystem, an object prediction that predicts a region of space that is likely to be occupied by an object is received.Type: GrantFiled: August 3, 2020Date of Patent: October 10, 2023Assignee: Waymo LLCInventors: Abhijit Ogale, Alexander Krizhevsky, Wan-Yen Lo
-
Patent number: 11531894Abstract: A neural network system for identifying positions of objects in an input image can include an object detector neural network, a memory interface subsystem, and an external memory. The object detector neural network is configured to, at each time step of multiple successive time steps, (i) receive a first neural network input that represents the input image and a second neural network input that identifies a first set of positions of the input image that have each been classified as showing a respective object of the set of objects, and (ii) process the first and second inputs to generate a set of output scores that each represents a respective likelihood that an object that is not one of the objects shown at any of the positions in the first set of positions is shown at a respective position of the input image that corresponds to the output score.Type: GrantFiled: September 4, 2020Date of Patent: December 20, 2022Assignee: Waymo LLCInventors: Abhijit Ogale, Alexander Krizhevsky, Wan-Yen Lo
-
Patent number: 11281230Abstract: A computing device of a first vehicle may receive a first image and a second image of a second vehicle having flashing light signals. The computing device may determine, in the first image and the second image, an image region that bounds the second vehicle such that the image region substantially encompasses the second vehicle. The computing device may determine a polar grid that partitions the image region in the first image and the second image into polar bins, and identify portions of image data exhibiting a change in color and a change in brightness between the first image and the second image. The computing device may determine a type of the flashing light signals and a type of the second vehicle; and accordingly provide instructions to control the first vehicle.Type: GrantFiled: April 1, 2020Date of Patent: March 22, 2022Assignee: Waymo LLCInventors: Wan-Yen Lo, David Ian Franklin Ferguson, Abhijit Ogale
-
Patent number: 11281918Abstract: Disclosed herein are methods and systems for determining a location of an object within an environment. An example method may include determining a three-dimensional (3D) location of a plurality of reference points in an environment, receiving a two-dimensional (2D) image of a portion of the environment that contains an object, selecting certain reference points from the plurality of reference points that form a polygon when projected into the 2D image that contains at least a portion of the object, determining an intersection point of a ray directed toward the object and a 3D polygon formed by the selected reference points, and based on the intersection point of the ray directed toward the object and the 3D polygon formed by the selected reference points, determining a 3D location of the object in the environment.Type: GrantFiled: October 21, 2020Date of Patent: March 22, 2022Assignee: Waymo LLCInventors: Wan-Yen Lo, David I. Ferguson
-
Publication number: 20210364637Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a location of a particular object relative to a vehicle. In one aspect, a method includes obtaining sensor data captured by one or more sensors of a vehicle. The sensor data is processed by a convolutional neural network to generate a sensor feature representation of the sensor data. Data is obtained which defines a particular spatial region in the sensor data that has been classified as including sensor data that characterizes the particular object. An object feature representation of the particular object is generated from a portion of the sensor feature representation corresponding to the particular spatial region. The object feature representation of the particular object is processed using a localization neural network to generate the location of the particular object relative to the vehicle.Type: ApplicationFiled: August 5, 2021Publication date: November 25, 2021Inventors: Yu-Han Chen, Wan-Yen Lo, Alper Ayvaci, Abhijit Ogale, Hyman Murveit
-
Patent number: 11105924Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a location of a particular object relative to a vehicle. In one aspect, a method includes obtaining sensor data captured by one or more sensors of a vehicle. The sensor data is processed by a convolutional neural network to generate a sensor feature representation of the sensor data. Data is obtained which defines a particular spatial region in the sensor data that has been classified as including sensor data that characterizes the particular object. An object feature representation of the particular object is generated from a portion of the sensor feature representation corresponding to the particular spatial region. The object feature representation of the particular object is processed using a localization neural network to generate the location of the particular object relative to the vehicle.Type: GrantFiled: October 4, 2018Date of Patent: August 31, 2021Assignee: Waymo LLCInventors: Yu-Han Chen, Wan-Yen Lo, Alper Ayvaci, Abhijit Ogale, Hyman Murveit
-
Patent number: 11093799Abstract: In some implementations, an image classification system of an autonomous or semi-autonomous vehicle is capable of improving multi-object classification by reducing repeated incorrect classification of objects that are considered rarely occurring objects. The system can include a common instance classifier that is trained to identify and recognize general objects (e.g., commonly occurring objects and rarely occurring objects) as belonging to specified object categories, and a rare instance classifier that is trained to compute one or more rarity scores representing likelihoods that an input image is correctly classified by the common instance classifier. The output of the rare instance classifier can be used to adjust the classification output of the common instance classifier such that the likelihood of input images being incorrectly classified is reduced.Type: GrantFiled: June 20, 2019Date of Patent: August 17, 2021Assignee: Waymo LLCInventors: Wan-Yen Lo, Abhijit Ogale, Yang Gao
-
Patent number: 11055577Abstract: In some implementations, an image classification system of an autonomous or semi-autonomous vehicle is capable of improving multi-object classification by reducing repeated incorrect classification of objects that are considered rarely occurring objects. The system can include a common instance classifier that is trained to identify and recognize general objects (e.g., commonly occurring objects and rarely occurring objects) as belonging to specified object categories, and a rare instance classifier that is trained to compute one or more rarity scores representing likelihoods that an input image is correctly classified by the common instance classifier. The output of the rare instance classifier can be used to adjust the classification output of the common instance classifier such that the likelihood of input images being incorrectly classified is reduced.Type: GrantFiled: June 20, 2019Date of Patent: July 6, 2021Assignee: Waymo LLCInventors: Wan-Yen Lo, Abhijit Ogale, Yang Gao
-
Publication number: 20210191420Abstract: Disclosed herein are systems and methods for providing supplemental identification abilities to an autonomous vehicle system. The sensor unit of the vehicle may be configured to receive data indicating an environment of the vehicle, while the control system may be configured to operate the vehicle. The vehicle may also include a processing unit configured to analyze the data indicating the environment to determine at least one object having a detection confidence below a threshold. Based on the at least one object having a detection confidence below a threshold, the processor may communicate at least a subset of the data indicating the environment for further processing. The vehicle is also configured to receive an indication of an object confirmation of the subset of the data. Based on the object confirmation of the subset of the data, the processor may alter the control of the vehicle by the control system.Type: ApplicationFiled: March 4, 2021Publication date: June 24, 2021Inventors: David Ian Ferguson, Wan-Yen Lo, Nathaniel Fairfield
-
Patent number: 10962981Abstract: Disclosed herein are systems and methods for providing supplemental identification abilities to an autonomous vehicle system. The sensor unit of the vehicle may be configured to receive data indicating an environment of the vehicle, while the control system may be configured to operate the vehicle. The vehicle may also include a processing unit configured to analyze the data indicating the environment to determine at least one object having a detection confidence below a threshold. Based on the at least one object having a detection confidence below a threshold, the processor may communicate at least a subset of the data indicating the environment for further processing. The vehicle is also configured to receive an indication of an object confirmation of the subset of the data. Based on the object confirmation of the subset of the data, the processor may alter the control of the vehicle by the control system.Type: GrantFiled: October 6, 2019Date of Patent: March 30, 2021Assignee: Waymo LLCInventors: David Ian Ferguson, Wan-Yen Lo, Nathaniel Fairfield
-
Patent number: 10955846Abstract: Aspects of the disclosure relate to detecting and responding to stop signs. An object detected in a vehicle's environment having location coordinates may be identified as a stop sign and, it may be determined whether the location coordinates of the identified stop sign correspond to a location of a stop sign in detailed map information. Then, whether the identified stop sign applies to the vehicle may be determined based on the detailed map information or on a number of factors. Then, if the identified stop sign is determined to apply to the vehicle, responses of the vehicle to the stop sign may be determined, and, the vehicle may be controlled based on the determined responses.Type: GrantFiled: July 26, 2019Date of Patent: March 23, 2021Assignee: Waymo LLCInventors: David Harrison Silver, David Ian Franklin Ferguson, Abhijit S. Ogale, Wan-Yen Lo
-
Patent number: 10853668Abstract: Disclosed herein are methods and systems for determining a location of an object within an environment. An example method may include determining a three-dimensional (3D) location of a plurality of reference points in an environment, receiving a two-dimensional (2D) image of a portion of the environment that contains an object, selecting certain reference points from the plurality of reference points that form a polygon when projected into the 2D image that contains at least a portion of the object, determining an intersection point of a ray directed toward the object and a 3D polygon formed by the selected reference points, and based on the intersection point of the ray directed toward the object and the 3D polygon formed by the selected reference points, determining a 3D location of the object in the environment.Type: GrantFiled: December 20, 2018Date of Patent: December 1, 2020Assignee: Waymo LLCInventors: Wan-Yen Lo, David I. Ferguson
-
Patent number: 10769809Abstract: A neural network system for identifying positions of objects in an input image can include an object detector neural network, a memory interface subsystem, and an external memory. The object detector neural network is configured to, at each time step of multiple successive time steps, (i) receive a first neural network input that represents the input image and a second neural network input that identifies a first set of positions of the input image that have each been classified as showing a respective object of the set of objects, and (ii) process the first and second inputs to generate a set of output scores that each represents a respective likelihood that an object that is not one of the objects shown at any of the positions in the first set of positions is shown at a respective position of the input image that corresponds to the output score.Type: GrantFiled: June 29, 2018Date of Patent: September 8, 2020Assignee: Waymo LLCInventors: Abhijit Ogale, Alexander Krizhevsky, Wan-Yen Lo
-
Patent number: 10733506Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating object predictions using a neural network. One of the methods includes receiving respective projections of a plurality of channels of input sensor data, wherein each channel of input sensor data represents different respective characteristics of electromagnetic radiation reflected off of one or more objects. Each of the projections of the plurality of channels of input sensor data are provided to a neural network subsystem trained to receive projections of input sensor data as input and to provide an object prediction as an output. At the output of the neural network subsystem, an object prediction that predicts a region of space that is likely to be occupied by an object is received.Type: GrantFiled: December 14, 2016Date of Patent: August 4, 2020Assignee: Waymo LLCInventors: Abhijit Ogale, Alexander Krizhevsky, Wan-Yen Lo
-
Publication number: 20200233430Abstract: A computing device of a first vehicle may receive a first image and a second image of a second vehicle having flashing light signals. The computing device may determine, in the first image and the second image, an image region that bounds the second vehicle such that the image region substantially encompasses the second vehicle. The computing device may determine a polar grid that partitions the image region in the first image and the second image into polar bins, and identify portions of image data exhibiting a change in color and a change in brightness between the first image and the second image. The computing device may determine a type of the flashing light signals and a type of the second vehicle; and accordingly provide instructions to control the first vehicle.Type: ApplicationFiled: April 1, 2020Publication date: July 23, 2020Inventors: Wan-Yen Lo, David Ian Franklin Ferguson, Abhijit Ogale
-
Patent number: 10710579Abstract: In some implementations, an autonomous or semi-autonomous vehicle is capable of using a collision prediction system to determine a confidence that any objects detected within a vicinity of the vehicle are on a trajectory that will collide with the vehicle. Laser obstacle points derived from recent sensor readings of one or more sensors of a vehicle are initially obtained. The laser obstacle points are projected into a pose coordinate system to generate an occupancy grid of a vicinity of the vehicle. A confidence that any objects represented by the laser obstacle points are on a trajectory that will collide with the vehicle is determined by applying a particle filter to the occupancy grid.Type: GrantFiled: June 1, 2017Date of Patent: July 14, 2020Assignee: Waymo LLCInventors: Wan-Yen Lo, Abhijit Ogale, David Ferguson
-
Patent number: 10678258Abstract: A computing device of a first vehicle may receive a first image and a second image of a second vehicle having flashing light signals. The computing device may determine, in the first image and the second image, an image region that bounds the second vehicle such that the image region substantially encompasses the second vehicle. The computing device may determine a polar grid that partitions the image region in the first image and the second image into polar bins, and identify portions of image data exhibiting a change in color and a change in brightness between the first image and the second image. The computing device may determine a type of the flashing light signals and a type of the second vehicle; and accordingly provide instructions to control the first vehicle.Type: GrantFiled: November 15, 2018Date of Patent: June 9, 2020Assignee: Waymo LLCInventors: Wan-Yen Lo, David Ian Franklin Ferguson, Abhijit Ogale
-
Publication number: 20200110175Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a location of a particular object relative to a vehicle. In one aspect, a method includes obtaining sensor data captured by one or more sensors of a vehicle. The sensor data is processed by a convolutional neural network to generate a sensor feature representation of the sensor data. Data is obtained which defines a particular spatial region in the sensor data that has been classified as including sensor data that characterizes the particular object. An object feature representation of the particular object is generated from a portion of the sensor feature representation corresponding to the particular spatial region. The object feature representation of the particular object is processed using a localization neural network to generate the location of the particular object relative to the vehicle.Type: ApplicationFiled: October 4, 2018Publication date: April 9, 2020Inventors: Yu-Han Chen, Wan-Yen Lo, Alper Ayvaci, Abhijit Ogale, Hyman Murveit