Abstract: A method may include obtaining lidar data comprising a plurality of lidar returns from an environment of an autonomous vehicle. The lidar data may be processed with a machine learning model to generate, for the plurality of lidar returns, a plurality of first outputs that each identify a respective lidar return as belonging to an object or non-object and a plurality of second outputs that identify lidar returns belonging to objects as harmful or non-harmful to the autonomous vehicle. A subset of the lidar returns identified as belonging to objects that (i) do not correspond to any of a plurality of pre-classified objects and (ii) were identified as harmful to the autonomous vehicle may be determined. The autonomous vehicle may be controlled based at least in part on the subset of lidar returns.

Abstract: A method may include obtaining sensor data that include a plurality of sensor returns from an environment of an autonomous vehicle. A first set of features may be extracted from the sensor data. The first set of features may be processed with a machine learning model to generate, for at least a subset of the plurality of sensor returns, a first output that classifies a respective sensor return as corresponding to an object or non-object and a second output that indicates a property of the object. The sensor returns classified as corresponding to objects may be compared to a plurality of pre-classified objects to generate one or more generic object classifications. The autonomous vehicle may be controlled based at least in part on the one or more generic object classifications.