Abstract: In one embodiment, a system receives a number of times from a number of time sources including sensors and real-time clocks (RTCs), wherein the sensors are in communication with an autonomous driving vehicle (ADV) and the sensors include at least a GPS sensor. The system generates a difference histogram based on a time for each of the time sources for a difference between a time of the GPS sensor and a time for each of the other sensors and RTCs. The system ranks the sensors and RTCs based on the difference histogram. The system selects a time source from one of the sensors or RTCs with a least difference in time with respect to the GPS sensor. The system generates a timestamp based on the selected time source to timestamp sensor data for a sensor unit of the ADV.

Abstract: The disclosure describes various embodiments for online system-level validation of sensor synchronization. According to an embodiment, an exemplary method of analyzing sensor synchronization in an autonomous driving vehicle (ADV) include the operations of acquiring raw sensor data from a first sensor and a second sensor mounted on the ADV, the raw sensor data describing a target object in a surrounding environment of the ADV; and generating an accuracy map based on the raw sensor data in view of timestamps extracted from the raw sensor data. The method further includes the operations of generating a first bounding box and a second bounding box around the target object using the raw sensor data; and performing an analysis of the first and second bounding boxes and the accuracy map using a predetermined algorithm in view of one or more pre-configured sensor settings to determine whether the first sensor and the second sensor are synchronized with each other.

Abstract: The disclosure describes various embodiments of validating data synchronization between an active sensor and a passive sensor. According to an exemplary method of validating sensor synchronization between an active sensor and a passive sensor, a synchronization device receives a first signal from the active sensor, the first signal indicating that the active sensor has transmitted laser points to a measure board. In response to the first signal, the synchronization device transmits a second signal to the passive sensor to trigger the passive sensor to capture an image of the measure board. A synchronization validation application can perform an analysis of the image of the measure board in view of timing of the first signal and second signal to determine whether the passive sensor and the active sensor are synchronized with each other.

Abstract: Diagnosing a sensor processing unit of an autonomous driving vehicle is described. An example computer-implemented method can include transmitting an executable image of a sensor processing application from a host system to the sensor processing unit via at least one of a universal asynchronous receiver-transmitter (UART) or an Ethernet connection. The method also includes causing the sensor processing unit to execute and launch the executable image of the sensor processing application in the DRAM from the eMMC storage device. The method also includes transmitting a sequence of predetermined commands to the executed sensor processing application to perform a plurality of sensor data processing operations on sensor data obtained from a plurality of sensors or sensor simulators associated with an autonomous driving vehicle.

Abstract: In one embodiment, an exemplary computer-implemented method of storing point cloud data in an autonomous driving vehicle can include the operations of receiving raw point cloud data from a LiDAR sensor mounted on the autonomous driving vehicle, the raw point cloud data representing cloud data points acquired in response to laser beams emitted at a given angle; retrieving configuration information of the LiDAR sensor, the configuration information including at least a number of laser lines of the LiDAR sensor. The method further includes the operations of constructing, based on the configuration information, a data structure that includes a data entry for each of the cloud data points, the data entry including multiple fields for storing attributes of the cloud data point, each field having a bit width determined based on the configuration information using a predetermined algorithm; and writing the cloud data points to a storage medium using the data structure.

Abstract: A sensor unit includes a sensor interface coupled to a number of sensors and a host interface coupled to a host system utilized to autonomously drive the vehicle. The sensor unit further includes sensor control modules corresponding to the sensors. Each sensor control module includes delay time control logic, delay adjustment logic, and a trigger signal generator. The delay time control logic is to receive a pulse time adjustment (PTA) value from the host system. The delay adjustment logic is to receive a trigger time adjustment (TTA) value from the host system. The delay adjustment logic is to modify timing of at least a portion of the pulses of a pulse signal based on the PTA value and the TTA value. The trigger signal generator is to generate a trigger signal based on the modified pulse signal and to transmit the trigger signal to a corresponding sensor.

Abstract: Systems and methods are disclosed for performing manufacturing testing on an autonomous driving vehicle (ADV) sensor board. A sensor unit of the ADV includes a plurality of sensor I/O channels that provide information to the ADV perception and planning module, to navigate the ADV. An array of sensors is emulated on a sensor unit test board. The sensor unit includes a small software that manages the flow of testing the sensor unit. The sensor unit test board provides emulated sensor data for, e.g., GPS, LIDAR, RADAR, inertial measurement, one or more cameras, humidity, temperature, and pressure, and throttle, braking, and steering inputs. Each emulated sensor includes its own data validity checker to ensure that each sensor I/O channel of the sensor unit is tested.

Abstract: Flexible hardware designs for camera calibration and image pre-processing are disclosed for vehicles including autonomous driving (AD) vehicles. For one example, a sensor unit includes a sensor interface, host interface, and pre-processing hardware. The sensor interface is coupled to a plurality of cameras configured to capture images around an autonomous driving vehicle (ADV). The host interface is coupled to a perception and planning system. The pre-processing hardware is coupled to the sensor interface to receive images from the plurality of cameras and to perform one or more pre-processing functions on the images and to transmit pre-processed images to the perception and planning system via the host interface. The perception and planning system is configured to perceive a driving environment surrounding the ADV based on the pre-processed images and to plan a path to control the ADV to navigate through the driving environment.

Abstract: In one embodiment, a system receives, at a sensor unit, a global positioning system (GPS) pulse signal from a GPS sensor of the autonomous driving vehicle (ADV), where the GPS pulse signal is a RF signal transmitted by a satellite to the GPS sensor, where the sensor unit is coupled to a number of sensors mounted on the ADV to perceive a driving environment surrounding the ADV and to plan a path to autonomously drive the ADV. The system receives a first local oscillator signal from a local oscillator. The system synchronizes the first local oscillator signal to the GPS pulse signal in real-time, including modifying the first local oscillator signal based on the GPS pulse signal. The system generates a second oscillator signal based on the synchronized first local oscillator signal, where the second oscillator signal is used to provide a time to at least one of the sensors.

Abstract: A sensor unit used in an ADV includes a sensor interface that can be coupled to cameras mounted on the ADV. The sensor unit further includes a host interface that can be coupled to a host system. The host system is configured to perceive a driving environment surrounding the ADV based on at least image data obtained from the cameras and to plan a path to autonomously drive the ADV. The sensor unit further includes one or more data acquisition modules corresponding to the cameras. Each data acquisition module includes a pixel alignment module and a frame processing module. The pixel alignment module is configured to reformat pixels of image data from an original format associated with the corresponding camera to a predetermined format. The frame processing module is configured to generate an image frame based on the image data and to transmit the image frame to the host system.

Abstract: In one embodiment, a sensor unit receives a first GPS message from a GPS sensor, where the sensor unit is coupled between sensors and a perception and planning system of an autonomous driving vehicle (ADV). The sensor unit determines a type of the first GPS message by matching a predetermined field of the first GPS message with a list of predetermined data patterns. Each of the predetermined data patterns corresponds to one of the predetermined types of GPS messages and decodes a payload of the first GPS message using a decoding algorithm associated with the type of the first GPS message.

Abstract: A method, apparatus, and system for timing synchronization between multiple computing nodes in an autonomous vehicle host system is disclosed. Timing of a first computing node of an autonomous vehicle host system is calibrated based on an external time source. A first timing message is transmitted from the first computing node to a second computing node of the autonomous vehicle host system via a first communication channel between the first computing node and the second computing node. Timing of the second computing node is calibrated based on the first timing message, wherein immediately subsequent to the calibration of timing of the second computing node, timing of the first computing node and of the second computing node is synchronized.

Abstract: In one embodiment, a computer-implemented method of performing a secure boot operation in an autonomous driving vehicle includes reading a first marker from a storage device in which the storage device includes a plurality of partitions and at least the first marker. The plurality of partitions includes a first partition including stored software, the first marker associated with the first partition, and wherein the first marker includes a unique identifier and an authentication code. The method further includes determining if the read first marker associated with the first partition is valid during a boot-up operation and executing the stored software in the first partition if the read first marker is determined valid.

Abstract: In one embodiment, a sensor unit to be utilized in an autonomous driving vehicle (ADV) includes a sensor interface coupled to a number of sensors mounted on a number of locations of an autonomous driving vehicle (ADV). The sensor unit includes a host interface to be coupled to a host system, where the host system is configured to perceive a driving environment surrounding the ADV based on sensor data obtained from the sensors and to plan a path to autonomously drive the ADV. The sensor unit includes a time synchronization hub device coupled to the sensor interface. The time synchronization hub device includes one or more TX and/or RX timestamp generators coupled to a time source, where the TX/RX timestamp generators generate TX/RX timestamps based on a time obtained from the time source to provide the TX/RX timestamps to one or more of the sensors.

Abstract: A data processing system includes a host system and one or more expansion devices coupled to the host system over a bus. The host system may include one or more processors and a memory storing instructions, which when executed, cause the processors to perform autonomous driving operations to drive an autonomous driving vehicle (ADV). Each expansion device includes a switch device and one or more processing modules coupled to the switch device. Each processing module can be configured to perform at least one of the autonomous driving operations offloaded from the host system. At least one of the processing modules can be configured as a client node to perform an action in response to an instruction received from the host system. Alternatively, it can be configured as a host node to distribute a task to another client node within the expansion device. This host node in the expansion device can further cooperate with the host system via a host-to-host connection.

Abstract: A sensor unit includes a sensor interface and a host interface. The sensor interface can be coupled to a number of sensors mounted on various locations of the ADV. The host interface can be coupled to a host system that is configured to perceive a driving environment surrounding the ADV based on sensor data obtained from the sensors and to plan a path to autonomously drive the ADV. The sensor unit further includes a sensor processing module and a sensor control module coupled to the sensor interface, and a time module coupled to the sensor processing module and the sensor control module. The sensor processing module is configured to process sensor data received from the sensors via the sensor interface. The sensor control module is configured to control operations of the sensors. The time module is configured to generate time to synchronize timing of the operations of the sensors.

Abstract: In one embodiment, a system receives, at a sensor unit, a global positioning system (GPS) pulse signal from a GPS sensor of the autonomous driving vehicle (ADV), where the GPS pulse signal is a RF signal transmitted by a satellite to the GPS sensor, where the sensor unit is coupled to a number of sensors mounted on the ADV to perceive a driving environment surrounding the ADV and to plan a path to autonomously drive the ADV. The system receives a first local oscillator signal from a local oscillator. The system synchronizes the first local oscillator signal to the GPS pulse signal in real-time, including modifying the first local oscillator signal based on the GPS pulse signal. The system generates a second oscillator signal based on the synchronized first local oscillator signal, where the second oscillator signal is used to provide a time to at least one of the sensors.

Abstract: A sensor unit used in an ADV includes a sensor interface that can be coupled to cameras mounted on the ADV. The sensor unit further includes a host interface that can be coupled to a host system. The host system is configured to perceive a driving environment surrounding the ADV based on at least image data obtained from the cameras and to plan a path to autonomously drive the ADV. The sensor unit further includes one or more data acquisition modules corresponding to the cameras. Each data acquisition module includes a pixel alignment module and a frame processing module. The pixel alignment module is configured to reformat pixels of image data from an original format associated with the corresponding camera to a predetermined format. The frame processing module is configured to generate an image frame based on the image data and to transmit the image frame to the host system.

Abstract: In one embodiment, a sensor unit to be utilized in an autonomous driving vehicle (ADV) includes a sensor interface coupled to a number of sensors mounted on a number of locations of an autonomous driving vehicle (ADV). The sensor unit includes a host interface to be coupled to a host system, where the host system is configured to perceive a driving environment surrounding the ADV based on sensor data obtained from the sensors and to plan a path to autonomously drive the ADV. The sensor unit includes a time synchronization hub device coupled to the sensor interface. The time synchronization hub device includes one or more TX and/or RX timestamp generators coupled to a time source, where the TX/RX timestamp generators generate TX/RX timestamps based on a time obtained from the time source to provide the TX/RX timestamps to one or more of the sensors.

Abstract: In one embodiment, a system receives a number of times from a number of time sources including sensors and real-time clocks (RTCs), wherein the sensors are in communication with the ADV and the sensors include at least a GPS sensor, and where the RTCs include at least a central processing unit real-time clock (CPU-RTC). The system generating a difference histogram based on a time for each of the time sources for a difference between a time of the GPS sensor and a time for each of the other sensors and RTCs. The system ranks the sensors and RTCs based on the difference histogram. The system selects a time source from one of the sensors or RTCs with a least difference in time with respect to the GPS sensor. The system generates a timestamp based on the selected time source to timestamp sensor data for a sensor unit of the ADV.