Abstract: An ADV includes a method to combine data from multiple sensors. The method compresses video data from a camera to generate compressed video data. The compressed video data are segmented. The method time synchronizes each segment of the compressed video data with data from other sensors. The method then combines each segment of the compressed video data with the corresponding time-synchronized sensor data for the other sensors. In one embodiment, each segment of the compressed video data is independently decodable. In another embodiment, each segment of the compressed video data includes a compressed video unit that is prepended with a buffered portion of the compressed video data that immediately precede the compressed video unit. The length of the compressed video unit is smaller than the length of the independently decodable segment to offer finer granularity in time synchronizing the compressed video data with the other sensor data with a tradeoff.

Abstract: Based on sensor data obtained from a variety of sensors, a driving environment surrounding an autonomous driving vehicle (ADV) is perceived, including perceiving and identifying one or more obstacles. A trajectory is planned based on the perception data according to a set of rules to drive the ADV navigating through the driving environment. Trajectory data representing the trajectory is generated, where the trajectory data includes information indicating target or expected vehicle states at different points in time along the trajectory. The trajectory data is then transmitted in a sequence or a stream of one or more controller area network (CAN) messages to an electronic control unit (ECU) of the ADV over a CAN bus. The ECU is configured to generate and issue one or more control commands (e.g., throttle, brake, steering commands) based on the trajectory data to drive the ADV according to the trajectory.

Abstract: A calibration table usable in operating an autonomous driving vehicle (ADV) is updated. The operations comprise: determining a first torque value at a first time instant prior to executing a control command; determining a control command based on a speed of the ADV, a desired acceleration, and an associated entry in the calibration table; executing the control command; determining a second torque value at a second time instant subsequent to executing the control command; determining a torque error value as a difference between the first and second torque values; updating the associated entry in the calibration table based at least in part on the torque error value; and generating driving signals based at least in part on the updated calibration table to control operations of the ADV.

Abstract: According to various embodiments, systems and methods for smoothing a reference line for an autonomous driving vehicle are described. In an exemplary method, a raw reference line can be generated from a high definition map and routing results, and can be truncated based on a predetermined formula. The truncated raw reference line can include a number of points, each point representing a position of an autonomous driving vehicle in a global coordinate system. The number of points can be converted to points in a local coordinate system, where a polynomial curve that best fits the points are generated. The polynomial curve can subsequently be used to generate a new vertical coordinate for a horizontal coordinate of each of the number of points. The new vertical coordinates and their corresponding horizontal coordinates can be converted back to the global coordinate system. The polynomial curve can be used to derive a heading, a kappa, and a dkappa for each point in the global coordinate system.

Abstract: Embodiments of the present disclosure provide a method and apparatus for generating information, a device for human-computer interaction, and a computer readable medium. The method may include: acquiring gravity sensing data of a shelf carrying an item; and identifying, in response to determining that the item on the shelf is taken based on the gravity sensing data, the taken item based on the gravity sensing data and an acquired image of the taken item, and generating order information of the taken item.

Abstract: According to one embodiment, a system establishes a secure connection between a host system and a data processing (DP) accelerator over a bus, the secure connection including one or more data channels. The system transmits a first instruction from the host system to the DP accelerator over a command channel, the first instruction requesting the DP accelerator to perform a data preparation operation. The system receives a first request to read a first data from a first memory location of the host system from the DP accelerator over one data channel. In response to the request, the system transmits the first data to the DP accelerator over the data channel, where the first data is utilized for a computation or a configuration operation. The system transmits a second instruction from the host system to the DP accelerator over the command channel to perform the computation or the configuration operation.

Abstract: A method for processing autonomous driving simulation data. The method includes: determining a type of a message transmitted between a simulation system and an auto driving system (ADS); determining a data acquisition mode based on the type of the message; obtaining a data stream transmitted between the simulation system and the ADS based on the data acquisition mode; and determining performance of the ADS based on the data stream.

Abstract: In one embodiment, a system generates an occupancy grid map based on an initial frame of point clouds. The system receives one or more subsequent frames of the point clouds. For each of the subsequent frames, the system updates an occupancy grid map based on the subsequent frame. The system identifies one or more problematic voxels based on the update, the system determines whether the problematic voxels belong to a wall object, and in response to determining that the problematic voxels belong to a wall object, the system flags the problematic voxels as ghost effect voxels for the subsequent frame.

Abstract: An ADV perceives a driving environment surrounding the ADV based on sensor data obtained from a variety of sensors mounted on the ADV including, for example, perceiving and recognizing a corner the ADV may be about to turn. Based on the perception data of the driving environment, a set of features representing the characteristics of an entrance point of a corner that the ADV is about to turn. Based on the characteristics of the corner, an entrance point of the corner is determined. Based on the entrance point, a lookup operation is performed in a corner mapping table to locate a mapping entry matching the entrance point. A turning radius is then obtained from the mapping entry of the corner mapping table. The turning radius obtained from the corner mapping table is then utilized to plan a trajectory (e.g., steering angle) to drive the ADV to turn the corner.

Abstract: Network architecture search (NAS) received a lot of attention. The supernet-based differentiable approach is popular because it can effectively share the weights and lead to more efficient search. However, the mismatch between the architecture and weights caused by weight sharing still exists. Moreover, the coupling effects among different operators are also neglected. To alleviate these problems, embodiments of an effective NAS methodology by similarity-based operator ranking are presented herein. With the aim of approximating each layer's output in the supernet, a similarity-based operator ranking based on statistical random comparison is used. In one or more embodiments, then the operator that possibly causes the least change to feature distribution discrepancy is pruned. In one or more embodiments, a fair sampling process may be used to mitigate the operators' Matthew effect that happened frequently in previous supernet approaches.

Abstract: According to one embodiment, a DP accelerator includes one or more execution units (EUs) configured to perform data processing operations in response to an instruction received from a host system coupled over a bus. The DP accelerator includes a security unit (SU) configured to establish and maintain a secure channel with the host system to exchange commands and data associated with the data processing operations. The DP accelerator includes a time unit (TU) coupled to the security unit to provide timestamp services to the security unit, where the time unit includes a clock generator to generate clock signals locally without having to derive the clock signals from an external source. The TU includes a timestamp generator coupled to the clock generator to generate a timestamp based on the clock signals, and a power supply to provide power to the clock generator and the timestamp generator.

Abstract: Systems and methods of video inpainting for autonomous driving are disclosed. For example, the method stitches a multiplicity of depth frames into a 3D map, where one or more objects in the depth frames have previously been removed. The method further projects the 3D map onto a first image frame to generate a corresponding depth map, where the first image frame includes a target inpainting region. For each target pixel within the target inpainting region of the first image frame, based on the corresponding depth map, the method further maps the target pixel within the target inpainting region of the first image frame to a candidate pixel in a second image frame. The method further determines a candidate color to fill the target pixel. The method further performs Poisson image editing on the first image frame to achieve color consistency at a boundary and between inside and outside of the target inpainting region of the first image frame.

Abstract: Embodiments of the present disclosure disclose a method for determining a vehicle control parameter, an apparatus for the same, a vehicle on-board controller, and an autonomous vehicle. An embodiment of the method comprises: obtaining a lateral offset sequence of a vehicle and a control input sequence of a controller for controlling a lateral output of the vehicle, wherein a lateral offset in the lateral offset sequence is for characterizing an offset between an actual lateral output of the vehicle and a desired lateral output; executing a step of determining a vehicle control parameter; wherein the executing the step of determining the vehicle control parameter includes: with the lateral offset sequence as an input and the control input sequence as the desired output, training a pre-established vehicle dynamic model to obtain a trained vehicle dynamic model; and determining the vehicle control parameter from the trained vehicle dynamic model.

Abstract: In one embodiment, in response to a request to make a three-point turn for an autonomous driving vehicle (ADV), a forward turning path is generated using a first spiral function based on a maximum forward curvature change rate. A backward turning path is generated using a second spiral function based on a maximum backward curvature change rate. The forward and backward curvature change rates may be determined based on the maximum forward and backward turning angles associated with the ADV, which may be specified as a part of vehicle specification or vehicle design of the ADV. The backward turning path is initiated from an end point of the forward turning path. A three-point turn path is then generated based on the forward turning path and the backward turning path. The ADV is then driven according to the three-point turn path by issuing one or more proper control commands.

Abstract: A computer-implemented method, apparatus, and system for discretizing lane markings and for generating a lane reference line is disclosed. A polynomial defined over an (x,y) coordinate system is received, the polynomial being representative of at least a portion of a lane boundary line. A length of the polynomial is determined. The polynomial is discretized, comprising determining a plurality of discretization points on the polynomial to represent the polynomial, wherein a first discretization point is a first end of the polynomial, wherein subsequent discretization points are determined successively until the polynomial is completely discretized, and wherein each discretization point other than the first discretization point is determined based at least in part on a slope of the polynomial at a previous discretization point. Thereafter, a lane reference line comprising a plurality of points is generated based on the discretized polynomial.

Abstract: In one embodiment, instead of using map data, a relative coordinate system is utilized to assist perception of the driving environment surrounding an ADV for some driving situations. One of such driving situations is driving on a highway. Typically, a highway has fewer intersections and exits. The relative coordinate system is utilized based on the relative lane configuration and relative obstacle information to control the ADV to simply follow the lane and avoid potential collision with any obstacles discovered within the road, without having to use map data. Once the relative lane configuration and obstacle information have been determined, regular path and speed planning and optimization can be performed to generate a trajectory to drive the ADV. Such a perception system is referred to as a relative perception system based on a relative coordinate system.

Abstract: In one embodiment, a sensor unit to be utilized in an autonomous driving vehicle (ADV) includes a sensor interface that can be coupled to a number of sensors mounted on a number of different locations of the ADV. The sensor unit further includes a host interface that can be coupled to a host system such as a planning and control system utilized to autonomously drive the vehicle. The sensor unit further includes a number of data transfer modules corresponding to the sensors. Each of the data transfer modules can be configured to operate in one of the operating modes, dependent upon the type of the corresponding sensor. The operating modes include a low latency mode, a high bandwidth mode, and a memory mode.

Abstract: According to one embodiment, in response to a request for a three-point turn, a set of forward turning paths is generated based on a maximum forward turning angle associated with an ADV. A set of backward tuning paths is generated based on a maximum backward turning angle associated with the ADV. A set of three-point turn path candidates is generated based on the forward turning paths and the backward turning paths. For each of the three-point turn path candidates, a path cost is calculated using a predetermined cost function. One of the three-point turn path candidates with the lowest path cost is selected as the final three-point turn path to drive the ADV to make a three-point turn.

Abstract: In one embodiment, it is determined that a speed of an autonomous driving vehicle (ADV) is below a predetermined speed threshold during a current turn section of a three-point turn, where the three-point turn includes at least three turn sections. In response, detecting an obstacle within a predetermined proximity of the ADV, determining a type of obstacle. An amount of time during which the speed of the ADV remains below the predetermined speed threshold is determined. It is determined whether the amount of time is greater than a time threshold corresponding to the type of the obstacle. If the amount of time is greater than the time threshold, the current turn section is ended and a next turn section is started.

Abstract: A method to perform video compression for ADV is disclosed. The method receives multiple frames of image data from multiple cameras. Metadata are appended to each frame of the image data to generate one of multiple frames of uncompressed image data as the image data are received. The frames of uncompressed image data may be stored. To compress the image data later, the method retrieves the frames of uncompressed image data, extracts the metadata from each frame of the uncompressed image data to generate one of multiple frames of processed image data. The method compresses each frame of the processed image data with the metadata extracted to generate one of multiple frames of compressed image data. The method reattaches the metadata to a corresponding frame of the compressed image data to generate one of multiple compressed image frames. The metadata supports time synchronization and error handling of the image data.