Abstract: Autonomous vehicles (AVs) utilize perception and understanding of objects on the road to predict behaviors of the objects, and to plan a trajectory for the vehicle. In some situations, an object may be occluded and undetected by an AV. However, a different AV viewing the same scene may detect the object. With information from multiple views of the same scene, it is possible to determine occlusion attributes for the object, such as relational occlusion information and extent of occlusion. For the AV that is driving on the road, having knowledge of the occluded object and the occlusion attributes can improve the performance of perception, understanding, tracking, prediction, and/or planning algorithms. For the algorithms, occlusion attributes can be generated from the multi-view data and included as part of labeled data for machine learning training. The models in the algorithms can learn to better handle occluded objects.

Abstract: Autonomous vehicles utilize perception to identify objects in a vehicle's surrounding environment and plan a trajectory. In perceiving an AV's surroundings, an object detection model uses sensor data from AV sensors to detect objects in the AV surroundings. An object detection model can be trained to detect objects using training datasets having sensor data and ground truth objects. In training, the object detection model uses the sensor data to identify objects in the locations of the ground truth objects. However, sometimes there is little or no sensor data corresponding to the ground truth objects, and the model is not able to learn patterns in sensor data to correctly detect the objects. Systems and methods are provided herein for removing ground truth objects that do not have sufficient corresponding sensor data from the training dataset, thereby preventing these objects from dominating the loss and interfering with the model learning useful patterns.

Abstract: Systems and methods for automatic removal for selected training data from a dataset are provided. Systems and methods are provided for identifying portions of radar data that contain information that is not present in camera and/or lidar data. The identified portions of the radar data can be processed by a neural network to provide the additional information. The identified parts of the radar data can then be processed while the remaining parts of the radar data remain unprocessed. In various examples, features can be extracted from identified regions of the radar data using a neural network and fused with camera and/or lidar features. In some examples, the fused features can be used for object detection.

Abstract: Autonomous vehicles (AVs) utilize perception and understanding of objects on the road to predict behaviors of the objects, and to plan a trajectory for the vehicle. In some scenarios, an AV may benefit from having information about an object that is not within a perceivable area of the sensors of the AV. Another AV in the area that has the object within a perceivable area of the sensors of the other AV may share information with the AV. Rich information about the object determined by the other AV can be compressed using an encoder and transferred efficiently to the AV for inclusion in a temporal map that combines locally determined object information and transferred object information determined by other AV(s). The temporal map may be used in one or more parts of the software stack of the AV.

Abstract: A computer vision module facilitates multi-sensor vision of AVs based on privileged information learning. The module may input first sensor data captured by a first sensor into an already-trained privileged model, input second sensor data captured by a second sensor into a first backbone of a target model, and input third sensor data captured by a third sensor into a second backbone of the target model. The three sensors may be of different types. To train the target model, internal parameters of the target model may be modified to minimize a loss, which may include a privilege loss, which indicates a difference between a latent representation of the first sensor data and a latent representation of the second sensor data, and another privileged loss, which indicates a difference between the latent representation of the first sensor data and a latent representation of the third sensor data.

Abstract: A method includes dividing an image into a plurality of patches; generating for the patches image tokens each including a halting score associated with the image token; generating a contextual features token representative of at least one contextual feature in connection with the image; processing the image tokens and the contextual features token using a vision transformer block; performing adaptive halting on the image tokens output from the transformer block, the adaptive halting comprising updating the halting scores associated with the image tokens and discarding ones of the image tokens having halting scores greater than or equal to a predetermined threshold score, wherein the non-discarded image tokens comprise remaining image tokens; and forwarding the remaining image tokens to a next processing block.

Abstract: A method and apparatus for implementing a method are disclosed. The method includes providing point cloud data to a machine learning algorithm, the point cloud data detected in the vicinity of an autonomous vehicle. The method further includes differentiating, via the machine learning algorithm, in the point cloud, data directly representing a location of a first object and data indirectly representing a location of a second object. The method includes transforming the data indirectly representing the location of the second object into data directly representing the location of the second object and generating corrected point cloud data based on the data directly representing the location of the first object and the data directly representing the location of the second object. The method includes outputting the corrected point cloud data to the autonomous vehicle.

Abstract: A method of calibrating a radar sensor includes receiving radar returns from a plurality of objects based on a radar signal sent from the radar sensor, each of the radar returns having a magnitude, at least a subset of the objects are known static objects, identifying a location and orientation of the radar sensor when the signal was sent, identifying expected reflectance values for each of the plurality of known static objects, calculating a conversion function configured to adjust the magnitudes of each of the radar returns for the known static objects to an estimated reflectance value based on the expected reflectance values for each of the known static objects, and adjusting an output of the radar sensor based on the conversion function.

Abstract: The technologies described herein relate to a radar system that is configured to generate point clouds based upon radar tensors generated by the radar system. More specifically, the radar system is configured to generate heatmaps based upon radar tensors, wherein a neural network receives the radar tensors as input and constructs the heatmaps as output. Point clouds are generated based upon the heatmaps. A computing system detects objects in an environment of an autonomous vehicle (AV) based upon the point clouds, and the computing system further causes the AV to perform a driving maneuver based upon the detected objects.

Abstract: A system includes a first sensor system of a first modality and a second sensor system of a second modality. The system further includes a computing system that is configured to detect and identify objects represented in sensor signals output by the first and second sensor systems. The computing system employs a hierarchical arrangement of transformers to fuse features of first sensor data output by the first sensor system and second sensor data output by the second sensor system.

Abstract: The technologies described herein relate to a radar system that is configured to generate point clouds based upon radar tensors generated by the radar system. More specifically, the radar system is configured to identify bins in radar tensors that correspond to objects in an environment of the radar system, and to use energy values in other bins to construct a point cloud. A computing system detects objects in an environment of the radar system based upon the point clouds.

Abstract: Technologies described herein relate to learning parameter values of a preprocessing algorithm that is executed in a radar system. The preprocessing algorithm is configured to receive raw radar data as input and is further configured to generate three-dimensional point clouds as output, where the preprocessing algorithm generates the three-dimensional point clouds based upon the raw radar data and the parameter values that are assigned to the preprocessing algorithm. To learn the parameter values, the preprocessing algorithm is converted to auto-differentiated form and is added as an input network layer to a deep neural network (DNN) that is configured to identify objects represented in three-dimensional point clouds. The parameter values are learned jointly with weight matrices of the DNN.

Abstract: A switch, used as an electronic-memory element, comprising a conductive organic polymer layer sandwiched between, and in contact with, two metallic conductive elements. In an initial post-fabrication state, the organic polymer layer is relatively highly conductive, the post-fabrication constituting a first stable state of the memory element that can serve to represent a binary bit “1 or 0,” depending which of two possible encoding conventions is employed. A relatively high voltage pulse can be passed between the two metal conductive elements, resulting in a market decrease in the current-carrying capacity of the organic polymer layer sandwiched between the two conductive elements. This change in conductivity of the organic polymer layer is generally irreversible, and constitutes a second stable state of the memory element that may be used to encode a binary bit “0” or “1,” again depending on which of two possible encoding conventions are employed.

Abstract: A microlens array for a light emitting device is disclosed. The light emitting device includes a plurality of OLEDs, each OLED having a minimum planar dimension. The array includes a plurality of microlenses, each of which has a minimum planar dimension and a maximum planar dimension. The minimum planar dimensions of the microlenses are larger than the maximum wavelength of visible light emitted from the OLEDs. The maximum planar dimensions of the microlenses are smaller than the smallest minimum planar dimension of any of the OLEDs.

Abstract: A switch, used as an electronic-memory element, comprising a conductive organic polymer layer sandwiched between, and in contact with, two metallic conductive elements. In an initial post-fabrication state, the organic polymer layer is relatively highly conductive, the post-fabrication constituting a first stable state of the memory element that can serve to represent a binary bit “1 or 0,” depending which of two possible encoding conventions is employed. A relatively high voltage pulse can be passed between the two metal conductive elements, resulting in a market decrease in the current-caring capacity of the organic polymer layer sandwiched between the two conductive elements. This change in conductivity of the organic polymer layer is generally irreversible, and constitutes a second stable state of the memory element that may be used to encode a binary bit “0” or “1,” again depending on which of two possible encoding conventions are employed.