Abstract: A system for faster object attribute and/or intent classification may include an machine-learned (ML) architecture that processes temporal sensor data (e.g., multiple instances of sensor data received at different times) and includes a cache in an intermediate layer of the ML architecture. The ML architecture may be capable of classifying an object's intent to enter a roadway, idling near a roadway, or active crossing of a roadway. The ML architecture may additionally or alternatively classify indicator states, such as indications to turn, stop, or the like. Other attributes and/or intentions are discussed herein.

Abstract: Techniques for detecting, locating, and/or classifying objects based on multiple sensor data inputs received from different sensor modalities. The techniques may include receiving sensor data generated by different sensor modalities of a vehicle, the sensor data including at least first sensor data generated by a first sensor modality and second sensor data generated by a second sensor modality. In some examples, the sensor data may be input into a machine-learning pipeline. The machine-learning pipeline may be configured to determine locations of objects in an environment surrounding the vehicle based at least in part on a correlation, by the multi-attention component, of the first sensor data and the second sensor data. The techniques may also include receiving, from the machine-learning pipeline, an output indicating a location of an object in the environment.

Abstract: Techniques for adjusting vehicle models based on environmental conditions are discussed herein. The techniques may include receiving image data representing a portion of an environment in which a vehicle is operating and inputting the image data into a machine learned model. Additionally, data representing an environmental condition associated with the environment may be received from a sensor of the vehicle to detect changes in the environmental conditions such that one or more actions associated with the machine learned model or an output of the machine learned model may be performed. Some of the techniques may also include running multiple machine learned models or multiple configurations of a machine learned model in parallel and selecting different outputs of the machine learned model(s) to adjust for changes in the environmental conditions. For instance, individual outputs may be selected based on environmental conditions, confidence scores, thresholds, etc.

Abstract: Techniques for segmenting and classifying a representation of aggregated sensor data from a scene are discussed herein. Sensor data may be collected during multiple traversals of a same scene, and the sensor data may be filtered to remove portions of the sensor data not relevant to road network maps. In some examples, the filtered data may be aggregated and represented in voxels of a three-dimensional voxel space, from which an image representing a top-down view of the scene may be generated, though other views are also contemplated. Operations may include segmenting and/or classifying the image e.g., by a trained machine-learned model, to associate class labels indicative of map elements (e.g., driving lane, stop line, turn lane, and the like) with segments identified in the image. Additionally, techniques may create or update road network maps based on segmented and semantically labeled image(s) of various portions of an environment.

Abstract: There is provided methods, systems, and computer-readable media for determining intention of bicycles and other person-wide vehicles. A method comprises receiving, from a first sensor of an autonomous vehicle, first sensor data relating to an external environment of the autonomous vehicle; and receiving, from a second sensor of the autonomous vehicle, second sensor data the second sensor comprising a different sensor type to the first sensor. A person-wide vehicle in proximate to the autonomous vehicle is identified. First object data associated with the person-wide vehicle is determined. Based on the first object data, a future intention of the person-wide vehicle is received from a machine-learned model. The autonomous vehicle is controlled based at least in part on the future intention of the person-wide vehicle.

Abstract: Techniques for detecting, locating, and/or classifying objects based on multiple sensor data inputs received from different sensor modalities. The techniques may include receiving sensor data generated by different sensor modalities of a vehicle, the sensor data including at least first sensor data generated by a first sensor modality and second sensor data generated by a second sensor modality. In some examples, the sensor data may be input into a machine-learning pipeline. The machine-learning pipeline may be configured to determine locations of objects in an environment surrounding the vehicle based at least in part on a correlation, by the multi-attention component, of the first sensor data and the second sensor data. The techniques may also include receiving, from the machine-learning pipeline, an output indicating a location of an object in the environment.

Abstract: A system for faster object attribute and/or intent classification may include an machine-learned (ML) architecture that processes temporal sensor data (e.g., multiple instances of sensor data received at different times) and includes a cache in an intermediate layer of the ML architecture. The ML architecture may be capable of classifying an object's intent to enter a roadway, idling near a roadway, or active crossing of a roadway. The ML architecture may additionally or alternatively classify indicator states, such as indications to turn, stop, or the like. Other attributes and/or intentions are discussed herein.

Abstract: There is provided methods, systems, and computer-readable media for determining intention of bicycles and other person-wide vehicles. A method comprises: receiving, from one or more sensors of an autonomous vehicle, sensor data relating to an external environment of the autonomous vehicle; detecting, based at least in part on the sensor data, a person-wide vehicle in the external environment proximate to the autonomous vehicle; determining, based at least in part on the sensor data and the person-wide vehicle, image data including the person-wide vehicle; inputting the image data to a machine-learned model; receiving, from the machine-learned model based on the image data, a future intention of the person-wide vehicle; and controlling the autonomous vehicle based at least in part on the future intention of the person-wide vehicle.

Abstract: Techniques are discussed herein for controlling autonomous vehicles within a driving environment, including generating and using bounding contours associated with objects detected in the environment. Image data may be captured and analyzed to identify and/or classify objects within the environment. Image-based and/or lidar-based techniques may be used to determine depth data associated with the objects, and a bounding contour may be determined based on the object boundaries and associated depth data. An autonomous vehicle may use the bounding contours of objects within the environment to classify the objects, predict the positions, poses, and trajectories of the objects, and determine trajectories and perform other vehicle control actions while safely navigating the environment.

Abstract: A system for faster object attribute and/or intent classification may include an machine-learned (ML) architecture that processes temporal sensor data (e.g., multiple instances of sensor data received at different times) and includes a cache in an intermediate layer of the ML architecture. The ML architecture may be capable of classifying an object's intent to enter a roadway, idling near a roadway, or active crossing of a roadway. The ML architecture may additionally or alternatively classify indicator states, such as indications to turn, stop, or the like. Other attributes and/or intentions are discussed herein.

Abstract: Techniques are discussed herein for controlling autonomous vehicles within a driving environment, including generating and using bounding contours associated with objects detected in the environment. Image data may be captured and analyzed to identify and/or classify objects within the environment. Image-based and/or lidar-based techniques may be used to determine depth data associated with the objects, and a bounding contour may be determined based on the object boundaries and associated depth data. An autonomous vehicle may use the bounding contours of objects within the environment to classify the objects, predict the positions, poses, and trajectories of the objects, and determine trajectories and perform other vehicle control actions while safely navigating the environment.

Abstract: A system for faster object attribute and/or intent classification may include an machine-learned (ML) architecture that processes temporal sensor data (e.g., multiple instances of sensor data received at different times) and includes a cache in an intermediate layer of the ML architecture. The ML architecture may be capable of classifying an object's intent to enter a roadway, idling near a roadway, or active crossing of a roadway. The ML architecture may additionally or alternatively classify indicator states, such as indications to turn, stop, or the like. Other attributes and/or intentions are discussed herein.

Abstract: Techniques for segmenting and classifying a representation of aggregated sensor data from a scene are discussed herein. Sensor data may be collected during multiple traversals of a same scene, and the sensor data may be filtered to remove portions of the sensor data not relevant to road network maps. In some examples, the filtered data may be aggregated and represented in voxels of a three-dimensional voxel space, from which an image representing a top-down view of the scene may be generated, though other views are also contemplated. Operations may include segmenting and/or classifying the image e.g., by a trained machine-learned model, to associate class labels indicative of map elements (e.g., driving lane, stop line, turn lane, and the like) with segments identified in the image. Additionally, techniques may create or update road network maps based on segmented and semantically labeled image(s) of various portions of an environment.

Abstract: Techniques are described for detecting whether a lane of a roadway is open or closed. Detecting a lane as being closed may include detecting an object in or near the lane, which may comprise determining a size, location, and/or classification associated with the object, and dilating the size associated with the object. The lane may be indicated as being closed if a distance between a dilated object detection and another object detection, dilated object detection, or lane extent is less than a threshold distance. The techniques may additionally or alternatively comprise determining an alternative lane shape based at least in part on one or more object detections and/or determining that one or more lanes are closed and/or uploading a lane closure and/or alternative lane shape to a central database for retrieval by/dissemination to other computing devices.

Abstract: A system for faster object attribute and/or intent classification may include an machine-learned (ML) architecture that processes temporal sensor data (e.g., multiple instances of sensor data received at different times) and includes a cache in an intermediate layer of the ML architecture. The ML architecture may be capable of classifying an object's intent to enter a roadway, idling near a roadway, or active crossing of a roadway. The ML architecture may additionally or alternatively classify indicator states, such as indications to turn, stop, or the like. Other attributes and/or intentions are discussed herein.

Abstract: Techniques are disclosed for tracking objects in sensor data, such as multiple images or multiple LIDAR clouds. The techniques may include comparing segmentations of sensor data such as by, for example, determining a similarity of a first segmentation of first sensor data and a second segmentation of second sensor data. Comparing the similarity may comprise determining a first embedding associated with the first segmentation and a second embedding associated with the second segmentation and determining a distance between the first embedding and the second embedding. The techniques may improve the accuracy and/or safety of systems integrating the techniques discussed herein.

Abstract: Techniques are disclosed for tracking objects in sensor data, such as multiple images or multiple LIDAR clouds. The techniques may include comparing segmentations of sensor data such as by, for example, determining a similarity of a first segmentation of first sensor data and a second segmentation of second sensor data. Comparing the similarity may comprise determining a first embedding associated with the first segmentation and a second embedding associated with the second segmentation and determining a distance between the first embedding and the second embedding. The techniques may improve the accuracy and/or safety of systems integrating the techniques discussed herein.

Abstract: Techniques are described for detecting whether a lane of a roadway is open or closed. Detecting a lane as being closed may include detecting an object in or near the lane, which may comprise determining a size, location, and/or classification associated with the object, and dilating the size associated with the object. The lane may be indicated as being closed if a distance between a dilated object detection and another object detection, dilated object detection, or lane extent is less than a threshold distance. The techniques may additionally or alternatively comprise determining an alternative lane shape based at least in part on one or more object detections and/or determining that one or more lanes are closed and/or uploading a lane closure and/or alternative lane shape to a central database for retrieval by/dissemination to other computing devices.

Abstract: Techniques are described for detecting whether a lane of a roadway is open or closed. Detecting a lane as being closed may include detecting an object in or near the lane, which may comprise determining a size, location, and/or classification associated with the object, and dilating the size associated with the object. The lane may be indicated as being closed if a distance between a dilated object detection and another object detection, dilated object detection, or lane extent is less than a threshold distance. The techniques may additionally or alternatively comprise determining an alternative lane shape based at least in part on one or more object detections and/or determining that one or more lanes are closed and/or uploading a lane closure and/or alternative lane shape to a central database for retrieval by/dissemination to other computing devices.

Abstract: Techniques are described for detecting whether a lane of a roadway is open or closed. Detecting a lane as being closed may include detecting an object in or near the lane, which may comprise determining a size, location, and/or classification associated with the object, and dilating the size associated with the object. The lane may be indicated as being closed if a distance between a dilated object detection and another object detection, dilated object detection, or lane extent is less than a threshold distance. The techniques may additionally or alternatively comprise determining an alternative lane shape based at least in part on one or more object detections and/or determining that one or more lanes are closed and/or uploading a lane closure and/or alternative lane shape to a central database for retrieval by/dissemination to other computing devices.