Abstract: According to various embodiments, systems, methods, and mediums for operating an autonomous driving vehicles (ADV) are described. The embodiments use a number of machine learning models to extract features individually from audio data and visual data captured by sensors mounted on the ADV, and then to fuse these extracted features to create a concatenated feature vectors. The concatenated feature vector is provided to a multiplayer perceptron (MLP) as input to generate a detection result related to the presence of an emergency vehicle in the surrounding environment. The detection result can be used by the ADV to take appropriate actions to comply with the local traffic rules.

Abstract: In one embodiment, an emergency vehicle detection system can be provided in the ADV travelling on a road to detect the presence of an emergency vehicle in a surrounding environment of the ADV using both audio data and visual data. The emergency vehicle detection system can use a trained neutral network to independently generate a detection result from the audio data, and use another trained network to independently generate another detection result from the visual data. The emergency vehicle detection system can fuse the two detection results to determine the position and moving direction of the emergency vehicle. The ADV can take appropriate actions in response to the position and moving direction of the emergency vehicle.

Abstract: Systems and methods for sound source detection and localization utilizing an autonomous driving vehicle (ADV) are disclosed. The method includes receiving audio data from a number of audio sensors mounted on the ADV. The audio data comprises sounds captured by the audio sensors and emitted by one or more sound sources. Based on the received audio data, the method further includes determining a number of sound source information. Each sound source information comprises a confidence score associated with an existence of a specific sound. The method further includes generating a data representation to report whether there exists the specific sound within the driving environment of the ADV. The data representation comprises the determined sound source information. The received audio data and the generated data representation are utilized to subsequently train a machine learning algorithm to recognize the specific sound source during autonomous driving of the ADV in real-time.

Abstract: Generating control effort to control an autonomous driving vehicle (ADV) includes determining a direction (forward or reverse) in which the ADV is driving and selecting a driving model and a predictive model based upon the direction. In a forward direction, the driving model is a dynamic model, such as a “bicycle model,” and the predictive model is a look-ahead model. In a reverse direction, the driving model is a hybrid dynamic and kinematic model and the predictive model is a look-back model. Current and predicted lateral error and heading error are determined using the driving model and predictive model, respectively. A linear quadratic regulator (LQR) uses the current and predicted lateral error and heading errors to determine a first control effort An augmented control logic determines a second, additional, control effort, to determine a final control effort that is output to a control module to drive the ADV.

Abstract: According to one embodiment, an obstacle is predicted to move from a starting point to an end point based on perception data perceiving a driving environment surrounding an ADV that is driving within a lane. A longitudinal movement trajectory from the starting point to the end point is generated in view of a shape of the lane. A lateral movement trajectory from the starting point to the end point is generated, including optimizing a shape of the lateral movement trajectory using a first polynomial function. The longitudinal movement trajectory and the lateral movement trajectory are then combined to form a final predicted trajectory that predicts how the obstacle is to move. A path is generated to control the ADV to move in view of the predicted trajectory of the obstacle, for example, to avoid the collision with the obstacle.

Abstract: A moving obstacle such as a vehicle within a proximity of an intersection and one or more exits of the intersection are identified. An obstacle state evolution of a spatial position of the moving obstacle over a period of time is determined. For each of the exits, an intersection exit encoding of the exit is determined based on intersection exit features of the exit. An aggregated exit encoding based on aggregating all of the intersection exit encodings for the exits is determined. For each of the exits, an exit probability of the exit that the moving obstacle likely exits the intersection through the exit is determined based on the obstacle state evolution and the aggregated exit encoding. Thereafter, a trajectory of the ADV is planned to control the ADV to avoid a collision with the moving obstacle based on the exit probabilities of the exits.

Abstract: In one embodiment, in response to perception data perceiving a driving environment surrounding an ADV, a map image of a map covering a location associated with the driving environment is obtained. An image recognition is performed on the map image to recognize one or more objects from the map image. An object may represent a particular road, a building structure (e.g., a parking lot, an intersection, or a roundabout). One or more features are extracted from the recognized objects, where the features may indicate or describe the traffic condition of the driving environment. Behaviors of one or more traffic participants perceived from the perception data are predicted based on the extracted features. A trajectory for controlling the ADV to navigate through the driving environment is planned based on the predicted behaviors of the traffic participants. A traffic participant can be a vehicle, a cyclist, or a pedestrian.

Abstract: In one embodiment, an autonomous driving system of an autonomous driving vehicle perceives a driving environment surrounding the autonomous driving vehicle traveling along a path, including perceiving an obstacle in the driving environment. The system detects a vertical acceleration of the autonomous driving vehicle based on sensor data obtained from a sensor on the autonomous driving vehicle. The system further calibrates the perceived obstacle based on the vertical acceleration of the autonomous driving vehicle. The system then controls the autonomous driving vehicle to navigate through the driving environment in view of the calibrated perceived obstacle.

Abstract: In one embodiment, a computer-implemented method for calibrating autonomous driving vehicles at a cloud-based server includes receiving, at the cloud-based server, one or more vehicle calibration requests from at least one user, each vehicle calibration request including calibration data for one or more vehicles and processing in parallel, by the cloud-based server, the one or more vehicle calibration requests for the at least one user to generate a calibration result for each vehicle. The method further includes sending, by the cloud-based server, the calibration result for each vehicle to the at least one user.

Abstract: In response to perceiving a moving object, one or more possible object paths of the moving object are determined based on the prior movement predictions of the moving object, for example, using a machine-learning model, which may be created based on a large amount of driving statistics of different vehicles. For each of the possible object paths, a set of trajectory candidates is generated based on a set of predetermined accelerations. Each of the trajectory candidates corresponds to one of the predetermined accelerations. A trajectory cost is calculated for each of the trajectory candidates using a predetermined cost function. One of the trajectory candidates having the lowest trajectory cost amongst the trajectory candidates is selected. An ADV path is planned to navigate the ADV to avoid collision with the moving object based on the lowest costs of the possible object paths of the moving object.

Abstract: In one embodiment, a set of predetermined driving parameters is determined from a set of driving statistics data collected from a number of vehicles, which may be driven by human drivers. For each pair of the predetermined driving parameters, a distribution of the pair of driving parameters is plotted based on their relationship on a two-dimensional (2D) distribution space. The 2D distribution space is partitioned into a number of grid cells, each grid cell representing a particular pair of driving parameters. For each of the grid cells, a probability is calculated that the pair of driving parameter likely falls in the grid cell. A grid table is generated corresponding to the pair of driving parameters. The grid table can be utilized during the autonomous driving at real-time or during simulation to determine a ride stability of an autonomous driving vehicle (ADV) in view of the pair of driving parameters.

Abstract: Systems and methods are disclosed for optimizing values of a set of tunable parameters of an autonomous driving vehicle (ADV). The controllers can be a linear quadratic regular, a “bicycle model,” a model-reference adaptive controller (MRAC) that reduces actuation latency in control subsystems such as steering, braking, and throttle, or other controller (“controllers”). An optimizer selects a set tunable parameters for the controllers. A task distribution system pairs each set of parameters with each of a plurality of simulated driving scenarios, and dispatches a task to the simulator to perform the simulation with the set of parameters. Each simulation is scored. A weighted score is generated from the simulation. The optimizer uses the weighted score as a target objective for a next iteration of the optimizer, for a fixed number of iterations. A physical real-world ADV is navigated using the optimized set of parameters for the controllers in the ADV.

Abstract: In one embodiment, a system/method generates a driving trajectory for an autonomous driving vehicle (ADV). The system perceives an environment of an autonomous driving vehicle (ADV). The system determines one or more bounding conditions based on the perceived environment. The system generates a first trajectory using a neural network model, wherein the neural network model is trained to generate a driving trajectory. The system evaluates/determines if the first trajectory satisfies the one or more bounding conditions. If the first trajectory satisfies the one or more bounding conditions, the system controls the ADV autonomously according to the first trajectory. Otherwise, the system controls the ADV autonomously according to a second trajectory, where the second trajectory is generated based on an objective function, where the objective function is determined based on at least the one or more bounding conditions.

Abstract: In one embodiment, a set of parameters representing a first state of an autonomous driving vehicle (ADV) to be simulated and a set of control commands to be issued at a first point in time. In response, a localization predictive model is applied to the set of parameters to determine a first position (e.g., x, y) of the ADV. A localization correction model is applied to the set of parameters to determine a set of localization correction factors (e.g., ?x, ?y). The correction factors may represent the errors between the predicted position of the ADV by the localization predictive model and the ground truth measured by sensors of the vehicle. Based on the first position of the ADV and the correction factors, a second position of the ADV is determined as the simulated position of the ADV.

Abstract: In an embodiment, a learning-based dynamic modeling method is provided for use with an autonomous driving vehicle. A control module in the ADV can generate current states of the ADV and control commands for a first driving cycle, and send the current states and control commands to a dynamic model implemented using a trained neural network model. Based on the current states and the control commands, the dynamic model generates expected future states for a second driving cycle, during which the control module generates actual future states. The ADV compares the expected future states and the actual future states to generate a comparison result, for use in evaluating one or more of a decision module, a planning module and a control module in the ADV.

Abstract: According to one embodiment, in response to a request to park an ADV into a parking lot, a remote server is accessed over a network (e.g., a VX2 link) to obtain a list of parking spaces that appear to be available in the parking lot. Based on the list of available parking spaces and the map associated with the parking lot, a route is generated to navigate through at least the available parking spaces. The ADV is driven according to the route to locate at least one of the available parking spaces and to park the ADV into the located available parking space. The centralized server is configured to periodically receive signals from a number of parking lots indicating which of the parking spaces of the parking lots are apparently available.

Abstract: In one embodiment, an autonomous driving vehicle (ADV) operates in an on-lane mode, where the ADV follows a path along a vehicle lane. In response to determining that the ADV is approaching a dead-end, the ADV switches to an open-space mode. While in the open-space mode, the ADV conducts a three-point turn using a series of steering and throttle commands to generate forward and reverse movements until the ADV is within a) a threshold heading, and b) a threshold distance, relative to the vehicle lane. The ADV can then return to the on-lane mode and resume along the vehicle lane away from the dead-end.

Abstract: Systems and methods are disclosed for identifying time-latency and subsystem control actuation dynamic delay due to second order dynamics that are neglected in control systems of the prior art. Embodiments identify time-latency and subsystem control actuation delays by developing a discrete-time dynamic model having parameters and estimating the parameters using a least-squares method over selected crowd-driving data. After estimating the model parameters, the model can be used to identify dynamic actuation delay metrics such as time-latency, rise time, settling time, overshoot, bandwidth, and resonant peak of the control subsystem. Control subsystems can include steering, braking, and throttling.

Abstract: In one embodiment, a computer-implemented method for optimizing a controller of an autonomous driving vehicle (ADV) includes obtaining several samples, each sample having a set of parameters, iteratively performing, until a predetermined condition is satisfied: determining, for each sample, a score according to a configuration of the controller based on the set of parameters of the sample, applying a machine learning model to the samples and corresponding scores to determine a mean function and a variance function, producing a new sample as a minimum of a function of the mean function and the variance function with respect to an input space of the set of parameters, adding the new sample to the several samples, and outputting the new sample as an optimal sample, where parameters of the optimal sample are utilized to configure the controller to autonomously drive the ADV.

Abstract: Systems and methods are disclosed for reducing second order dynamics delays in a control subsystem (e.g. throttle, braking, or steering) in an autonomous driving vehicle (ADV) and increasing control system bandwidth by accounting for time-latency in a control subsystem actuation system. A control input is received from an ADV's autonomous driving system. The control input is translated into a control command of the control subsystem of the ADV. A reference actuation output and a predicted actuation output are generated corresponding to a by-wire (“real”) actuation action for the control subsystem. A control error is determined between the reference actuation action and the by-wire actuation action. A predicted control error is determined between the predicted actuation action and the between the by-wire actuation action. Adaptive gains are determined and applied to the by-wire actuation action to generate a second by-wire actuation action.