Abstract: Systems and methods are disclosed for optimizing values of a set of tunable parameters of an autonomous driving vehicle (ADV). The controllers can be a linear quadratic regular, a “bicycle model,” a model-reference adaptive controller (MRAC) that reduces actuation latency in control subsystems such as steering, braking, and throttle, or other controller (“controllers”). An optimizer selects a set tunable parameters for the controllers. A task distribution system pairs each set of parameters with each of a plurality of simulated driving scenarios, and dispatches a task to the simulator to perform the simulation with the set of parameters. Each simulation is scored. A weighted score is generated from the simulation. The optimizer uses the weighted score as a target objective for a next iteration of the optimizer, for a fixed number of iterations. A physical real-world ADV is navigated using the optimized set of parameters for the controllers in the ADV.

Abstract: Systems and methods are disclosed for collecting driving data from simulated autonomous driving vehicle (ADV) driving sessions and real-world ADV driving sessions. The driving data is processed to exclude manual (human) driving data and to exclude data corresponding to the ADV being stationary (not driving). Data can further be filtered based on driving direction: forward or reverse driving. Driving data records are time stamped. The driving data can be aligned according to the timestamp, then a standardized set of metrics is generated from the collected, filtered, and time-aligned data. The standardized set of metrics are used to grade the performance the control system of the ADV, and to generate an updated ADV controller, based on the standardized set of metrics.

Abstract: Disclosed are performance metrics for evaluating the accuracy of a dynamic model in predicting the trajectory of ADV when simulating the behavior of the ADV under the control commands. The performance metrics may indicate the degree of similarity between the predicted trajectory of the dynamic model and the actual trajectory of the vehicle when applied with identical control commands. The performance metrics measure deviations of the predicted trajectory of the dynamic model from the actual trajectory based on the ground truths. The performance metrics may include cumulative or mean absolute trajectory error, end-pose difference (ED), two-sigma defect rate (?2?), the Hausdirff Distance (HAU), the longest common sub-sequence error (LCSS), or dynamic time warping (DTW). The two-sigma defect rate represents the ratio of the number of points with true location error falling out of the 2? range of the predicted location error over the total number of points in the trajectory.

Abstract: Systems and methods are disclosed for optimizing values of a set of tunable parameters of an autonomous driving vehicle (ADV). The controllers can be a linear quadratic regular, a “bicycle model,” a model-reference adaptive controller (MRAC) that reduces actuation latency in control subsystems such as steering, braking, and throttle, or other controller (“controllers”). An optimizer selects a set tunable parameters for the controllers. A task distribution system pairs each set of parameters with each of a plurality of simulated driving scenarios, and dispatches a task to the simulator to perform the simulation with the set of parameters. Each simulation is scored. A weighted score is generated from the simulation. The optimizer uses the weighted score as a target objective for a next iteration of the optimizer, for a fixed number of iterations. A physical real-world ADV is navigated using the optimized set of parameters for the controllers in the ADV.