Abstract: In one embodiment, an exemplary computer-implemented method of storing point cloud data in an autonomous driving vehicle can include the operations of receiving raw point cloud data from a LiDAR sensor mounted on the autonomous driving vehicle, the raw point cloud data representing cloud data points acquired in response to laser beams emitted at a given angle; retrieving configuration information of the LiDAR sensor, the configuration information including at least a number of laser lines of the LiDAR sensor. The method further includes the operations of constructing, based on the configuration information, a data structure that includes a data entry for each of the cloud data points, the data entry including multiple fields for storing attributes of the cloud data point, each field having a bit width determined based on the configuration information using a predetermined algorithm; and writing the cloud data points to a storage medium using the data structure.

Abstract: Embodiments relate to systems and methods to optimize quantization of tensors of an AI model. According to one embodiment, a system receives an AI model having one or more layers. The system receives a number of input data for offline inferencing and applies offline inferencing to the AI model based on the input data to generate offline data distributions for the AI model. The system quantizes one or more tensors of the AI model based on the offline data distributions to generate a low-bit representation AI model, where each layer of the AI model includes the one or more tensors, where the one or more tensors include the one or more tensors. In one embodiment, the system applies online inferencing using the low-bit representation AI model to generate online data distributions for a feature map, and quantizes a feature map tensor based on the online data distributions.

Abstract: In one embodiment, a system for operating an autonomous driving vehicle (ADV) includes a number of modules. These modules include at least a perception module to perceive a driving environment surrounding the ADV and a planning module to plan a path to drive the ADV to navigate the driving environment. The system further includes a bus coupled to the modules and a sensor processing module communicatively coupled to the modules over the bus. The sensor processing module includes a bus interface coupled to the bus, a sensor interface to be coupled to a first set of one or more sensors mounted on the ADV, a message queue to store messages published by the sensors, and a message handler to manage the messages stored in the message queue. The messages may be subscribed by at least one of the modules to allow the modules to monitor operations of the sensors.

Abstract: An ADV includes a method to combine data from multiple sensors. The method compresses video data from a camera to generate compressed video data. The compressed video data are segmented. The method time synchronizes each segment of the compressed video data with data from other sensors. The method then combines each segment of the compressed video data with the corresponding time-synchronized sensor data for the other sensors. In one embodiment, each segment of the compressed video data is independently decodable. In another embodiment, each segment of the compressed video data includes a compressed video unit that is prepended with a buffered portion of the compressed video data that immediately precede the compressed video unit. The length of the compressed video unit is smaller than the length of the independently decodable segment to offer finer granularity in time synchronizing the compressed video data with the other sensor data with a tradeoff.

Abstract: A method, apparatus, and system for timing synchronization between multiple computing nodes in an autonomous vehicle host system is disclosed. Timing of a first computing node of an autonomous vehicle host system is calibrated based on an external time source. A first timing message is transmitted from the first computing node to a second computing node of the autonomous vehicle host system via a first communication channel between the first computing node and the second computing node. Timing of the second computing node is calibrated based on the first timing message, wherein immediately subsequent to the calibration of timing of the second computing node, timing of the first computing node and of the second computing node is synchronized.

Abstract: In one embodiment, a computer-implemented method of performing a secure boot operation in an autonomous driving vehicle includes reading a first marker from a storage device in which the storage device includes a plurality of partitions and at least the first marker. The plurality of partitions includes a first partition including stored software, the first marker associated with the first partition, and wherein the first marker includes a unique identifier and an authentication code. The method further includes determining if the read first marker associated with the first partition is valid during a boot-up operation and executing the stored software in the first partition if the read first marker is determined valid.

Abstract: A method to perform video compression for ADV is disclosed. The method receives multiple frames of image data from multiple cameras. Metadata are appended to each frame of the image data to generate one of multiple frames of uncompressed image data as the image data are received. The frames of uncompressed image data may be stored. To compress the image data later, the method retrieves the frames of uncompressed image data, extracts the metadata from each frame of the uncompressed image data to generate one of multiple frames of processed image data. The method compresses each frame of the processed image data with the metadata extracted to generate one of multiple frames of compressed image data. The method reattaches the metadata to a corresponding frame of the compressed image data to generate one of multiple compressed image frames. The metadata supports time synchronization and error handling of the image data.

Abstract: Embodiments relate to systems and methods to optimize quantization of tensors of an AI model. According to one embodiment, a system receives an AI model having one or more layers. The system receives a number of input data for offline inferencing and applies offline inferencing to the AI model based on the input data to generate offline data distributions for the AI model. The system quantizes one or more tensors of the AI model based on the offline data distributions to generate a low-bit representation AI model, where each layer of the AI model includes the one or more tensors, where the one or more tensors include the one or more tensors. In one embodiment, the system applies online inferencing using the low-bit representation AI model to generate online data distributions for a feature map, and quantizes a feature map tensor based on the online data distributions.

Abstract: An ADV includes a method to combine data from multiple sensors. The method compresses video data from a camera to generate compressed video data. The compressed video data are segmented. The method time synchronizes each segment of the compressed video data with data from other sensors. The method then combines each segment of the compressed video data with the corresponding time-synchronized sensor data for the other sensors. In one embodiment, each segment of the compressed video data is independently decodable. In another embodiment, each segment of the compressed video data includes a compressed video unit that is prepended with a buffered portion of the compressed video data that immediately precede the compressed video unit.

Abstract: A method to perform video compression for ADV is disclosed. The method receives multiple frames of image data from multiple cameras. Metadata are appended to each frame of the image data to generate one of multiple frames of uncompressed image data as the image data are received. The frames of uncompressed image data may be stored. To compress the image data later, the method retrieves the frames of uncompressed image data, extracts the metadata from each frame of the uncompressed image data to generate one of multiple frames of processed image data. The method compresses each frame of the processed image data with the metadata extracted to generate one of multiple frames of compressed image data. The method reattaches the metadata to a corresponding frame of the compressed image data to generate one of multiple compressed image frames. The metadata supports time synchronization and error handling of the image data.

Abstract: In one embodiment, a system for operating an autonomous driving vehicle (ADV) includes a number of modules. These modules include at least a perception module to perceive a driving environment surrounding the ADV and a planning module to plan a path to drive the ADV to navigate the driving environment. The system further includes a bus coupled to the modules and a sensor processing module communicatively coupled to the modules over the bus. The sensor processing module includes a bus interface coupled to the bus, a sensor interface to be coupled to a first set of one or more sensors mounted on the ADV, a message queue to store messages published by the sensors, and a message handler to manage the messages stored in the message queue. The messages may be subscribed by at least one of the modules to allow the modules to monitor operations of the sensors.

Abstract: A method, apparatus, and system for timing synchronization between multiple computing nodes in an autonomous vehicle host system is disclosed. Timing of a first computing node of an autonomous vehicle host system is calibrated based on an external time source. A first timing message is transmitted from the first computing node to a second computing node of the autonomous vehicle host system via a first communication channel between the first computing node and the second computing node. Timing of the second computing node is calibrated based on the first timing message, wherein immediately subsequent to the calibration of timing of the second computing node, timing of the first computing node and of the second computing node is synchronized.

Abstract: In one embodiment, a computer-implemented method of performing a secure boot operation in an autonomous driving vehicle includes reading a first marker from a storage device in which the storage device includes a plurality of partitions and at least the first marker. The plurality of partitions includes a first partition including stored software, the first marker associated with the first partition, and wherein the first marker includes a unique identifier and an authentication code. The method further includes determining if the read first marker associated with the first partition is valid during a boot-up operation and executing the stored software in the first partition if the read first marker is determined valid.