Abstract: According to various embodiments, systems, methods, and mediums for operating an autonomous driving vehicles (ADV) are described. The embodiments use a number of machine learning models to extract features individually from audio data and visual data captured by sensors mounted on the ADV, and then to fuse these extracted features to create a concatenated feature vectors. The concatenated feature vector is provided to a multiplayer perceptron (MLP) as input to generate a detection result related to the presence of an emergency vehicle in the surrounding environment. The detection result can be used by the ADV to take appropriate actions to comply with the local traffic rules.

Abstract: Systems and methods for automatic generation of labelled audio data are disclosed. The method is performed by an autonomous driving system (ADS) of an autonomous driving vehicle (ADV). The method includes recording a sound emitted by an object within a driving environment, and converting the recorded sound into audio data. The method further includes capturing at least one position of the object while the sound is being recorded. The method further includes automatically labelling the audio data using the captured at least one position of the object as an audio label, to generate labelled audio data, where the labelled audio data is utilized to subsequently train a machine learning algorithm to recognize a sound source during autonomous driving of the ADV.

Abstract: Sound signals that are not audible are generated by one or more speakers disposed in the vehicle. Reflected sound signals from a driver or a passenger of the vehicle that are not audible are received by one or more microphones disposed in the vehicle. A behavior-induced acoustic pattern is detected based on the reflected ultrasound signals. The behavior-induced acoustic pattern is analyzed to recognize a behavior of the driver or the passenger of the vehicle. A response or an alert is generated according to the recognized behavior of the driver or the passenger in the vehicle.

Abstract: Sound signals are received by one or more microphones disposed at an ADV. The sound signals are analyzed to extract a feature of a road on which the ADV is driving. A road condition of the road is determined based on the extracted feature. A path planning and speed planning is performed to generate a trajectory based on the road condition. The ADV is controlled to drive autonomously according to the generated trajectory.

Abstract: In one embodiment, an emergency vehicle detection system can be provided in the ADV travelling on a road to detect the presence of an emergency vehicle in a surrounding environment of the ADV using both audio data and visual data. The emergency vehicle detection system can use a trained neutral network to independently generate a detection result from the audio data, and use another trained network to independently generate another detection result from the visual data. The emergency vehicle detection system can fuse the two detection results to determine the position and moving direction of the emergency vehicle. The ADV can take appropriate actions in response to the position and moving direction of the emergency vehicle.

Abstract: Systems and methods for generating labelled audio data and onboard validation of the labelled audio data utilizing an autonomous driving vehicle (ADV) while the ADV is operating within a driving environment are disclosed. The method includes recording a sound emitted by an object within the driving environment of the ADV, and converting the recorded sound into audio samples. The method further includes labelling the audio samples, and refining the labelled audio samples to produce refined labelled audio data. The refined labelled audio data is utilized to subsequently train a machine learning algorithm to recognize a sound source during autonomous driving of the ADV. The method further includes generating a performance profile of the refined labelled audio data based at least on the audio samples, a position of the object, and a relative direction of the object. The position of the object and the relative direction of the object are determined by a perception system of the ADV.

Abstract: According to one embodiment, an obstacle is predicted to move from a starting point to an end point based on perception data perceiving a driving environment surrounding an ADV that is driving within a lane. A longitudinal movement trajectory from the starting point to the end point is generated in view of a shape of the lane. A lateral movement trajectory from the starting point to the end point is generated, including optimizing a shape of the lateral movement trajectory using a first polynomial function. The longitudinal movement trajectory and the lateral movement trajectory are then combined to form a final predicted trajectory that predicts how the obstacle is to move. A path is generated to control the ADV to move in view of the predicted trajectory of the obstacle, for example, to avoid the collision with the obstacle.

Abstract: A moving obstacle such as a vehicle within a proximity of an intersection and one or more exits of the intersection are identified. An obstacle state evolution of a spatial position of the moving obstacle over a period of time is determined. For each of the exits, an intersection exit encoding of the exit is determined based on intersection exit features of the exit. An aggregated exit encoding based on aggregating all of the intersection exit encodings for the exits is determined. For each of the exits, an exit probability of the exit that the moving obstacle likely exits the intersection through the exit is determined based on the obstacle state evolution and the aggregated exit encoding. Thereafter, a trajectory of the ADV is planned to control the ADV to avoid a collision with the moving obstacle based on the exit probabilities of the exits.

Abstract: In response to perceiving a moving object, one or more possible object paths of the moving object are determined based on the prior movement predictions of the moving object, for example, using a machine-learning model, which may be created based on a large amount of driving statistics of different vehicles. For each of the possible object paths, a set of trajectory candidates is generated based on a set of predetermined accelerations. Each of the trajectory candidates corresponds to one of the predetermined accelerations. A trajectory cost is calculated for each of the trajectory candidates using a predetermined cost function. One of the trajectory candidates having the lowest trajectory cost amongst the trajectory candidates is selected. An ADV path is planned to navigate the ADV to avoid collision with the moving object based on the lowest costs of the possible object paths of the moving object.

Abstract: In one embodiment, a set of dynamic vehicle parameters of an ADV associated with a road location are determined based on outputs of an IMU of the ADV measured when the ADV is traveling through the road location. Whether the set of dynamic vehicle parameters satisfy one of a first set of criteria and a second set of criteria is determined. In response to the dynamic vehicle parameters satisfying the first set of criteria for a first predetermined quantity of times or satisfying the second set of criteria for a second predetermined quantity of times, the speed limit associated with the road location is adjusted within a limited range spanning from a minimum speed limit to a maximum speed limit. Operations of the ADV when the ADV subsequently travels through the road location are controlled based at least in part on the adjusted speed limit.

Abstract: Described herein are various embodiments for identifying miscalibrations in LiDAR sensors installed on an ADV in real time while the ADV is in motion, and notifying a user that the LiDAR sensors needs to be recalibrated. An exemplary method includes calculating an initial number of overlapping cloud points between the cloud point data from a first LiDAR sensor and a second LiDAR sensor; and replacing a set of existing calibration parameters of the second LiDAR sensor with multiple sets of recalibration parameters to calculate multiple revised numbers of overlapping cloud points between the point cloud data. A potential miscalibration can be detected in the first LiDAR sensor or the second LiDAR sensor when the initial number of overlapping cloud points is smaller than at least one revised number of overlapping cloud points. The potential miscalibration can be verified by repeating the above operations multiple times with a time of time following each repetition.

Abstract: In an embodiment, a learning-based dynamic modeling method is provided for use with an autonomous driving vehicle. A control module in the ADV can generate current states of the ADV and control commands for a first driving cycle, and send the current states and control commands to a dynamic model implemented using a trained neural network model. Based on the current states and the control commands, the dynamic model generates expected future states for a second driving cycle, during which the control module generates actual future states. The ADV compares the expected future states and the actual future states to generate a comparison result, for use in evaluating one or more of a decision module, a planning module and a control module in the ADV.

Abstract: A battery pack (1) includes a housing (2) and an array (40) of electrochemical cells (80) disposed in the housing (2). The housing (2) includes a container (3) and a lid (30) that closes an open end of the container (3). The container (3) has a base (4), a sidewall (8) that surrounds the base (4), and a spring plate (110) disposed inside the side wall (8) between the cells (80) and the sidewall (8). The spring plate (110) is free standing within the container (3) and applies a spring force to the cell array (40) that restrains the cells (80) along an axis normal to the surface of the spring plates (110). The lid (30) includes inwardly-protruding pins (50, 60) that further restrain the cells (80) within the housing (2).

Abstract: In one embodiment, a computer-implemented method for optimizing a controller of an autonomous driving vehicle (ADV) includes obtaining several samples, each sample having a set of parameters, iteratively performing, until a predetermined condition is satisfied: determining, for each sample, a score according to a configuration of the controller based on the set of parameters of the sample, applying a machine learning model to the samples and corresponding scores to determine a mean function and a variance function, producing a new sample as a minimum of a function of the mean function and the variance function with respect to an input space of the set of parameters, adding the new sample to the several samples, and outputting the new sample as an optimal sample, where parameters of the optimal sample are utilized to configure the controller to autonomously drive the ADV.

Abstract: In one embodiment, method performed by an autonomous driving vehicle (ADV) that determines, within a driving space, a plurality of routes from a current location of the ADV to a desired location. The method determines, for each route of the plurality of routes, an objective function to control the ADV autonomously along the route and, for each of the objective functions, performs Differential Dynamic Programming (DDP) optimization in view of a set of constraints to produce a path trajectory. The method determines whether at least one of the path trajectories satisfies each constraint and, in response to a path trajectory satisfying each of the constraints, selects the path trajectory for navigating the ADV from the current location to the desired location.

Abstract: A moving object such as a vehicle is identified within an intersection having multiple exits. The moving object and the intersection and its exits may be identified based on sensor data obtained from various sensors mounted on an ADV. An exit coordinate map is generated based on the orientation of the moving object and a relative position of each of the exits of the intersection with respect to the current position of the moving object. For each of the exits, an exit probability of the exit that the moving object likely exits the intersection using the exit coordinate map. Thereafter, a trajectory of the ADV is planned to navigate through the intersection to avoid the collision with the moving object based on the exit probabilities of the exits of the intersection. The above process is iteratively performed for each of the moving objects detected within the proximity of the intersection.

Abstract: In one embodiment, a plurality of obstacles is sensed in an environment of an automated driving vehicle (ADV). One or more representations are formed to represent corresponding groupings of the plurality of obstacles. A vehicle route is determined in view of the one or more representations, rather than each and every one of the obstacles individually.

Abstract: In one embodiment, a control command is generated with an MPC controller, the MPC controller including a cost function with weights associated with cost terms of the cost function. The control command is applied to a dynamic model of an autonomous driving vehicle (ADV) to simulate behavior of the ADV. One or more of the weights are based on evaluation of the dynamic model in response to the control command, resulting in an adjusted cost function of the MPC controller. Another control command is generated with the MPC controller having the adjusted cost function. This second control command can be used to effect movement of the ADV.

Abstract: In one embodiment, an ADV is routed by executing a first driving scenario that is active. The first driving scenario is one of a plurality of driving scenario types, each driving scenario type being associated with one or more stages to be executed while a corresponding driving scenario type is active. Based on an environmental condition around the ADV, a second driving scenario is set as active. The ADV is routed by executing the second driving scenario. When the second driving scenario exits, execution of the first driving scenario resumes at the one or more stages of the first driving scenario that remains to be executed.

Abstract: In one embodiment, a method determines a route from a first location of an autonomous driving vehicle (ADV) to a second location within an open space, the first location being a current location of the ADV. The method determines an objective function based on the route, the objective function having a set of costs for maneuvering the ADV from the first location to the second location. The method determines environmental conditions of the open space and uses the environmental conditions to determine a set of weights, each weight to be applied to a corresponding cost of the objective function. The method optimizes the objective function in view of one or more constraints, such that an output of the objective function reaches minimum while the one or more constraints are satisfied and generates a path trajectory with the optimized objective function to control the ADV autonomously according to the path trajectory.