Abstract: In response to a request to pull over an ADV at a destination point at a side of a lane, a path including a first segment, a second segment and a transition point is planned. The transition point is determined based on at least one of a distance to the destination point or a predetermined distance to a boundary of the side of the lane. The first segment from a start point to the transition point is generated by using a quadratic programming (QP) operation. The second segment from the transition point to the destination is generated based on a shape of the boundary. The ADV is controlled to pull over to the destination point according to the planned path.

Abstract: In one embodiment, a method for solution inference using neural networks in LiDAR localization includes constructing a cost volume in a solution space for a predicted pose of an autonomous driving vehicle (ADV), the cost volume including a number of sub volumes, each sub volume representing a matching cost between a keypoint from an online point cloud and a corresponding keypoint on a pre-built point cloud map. The method further includes regularizing the cost volume using convention neural networks (CNNs) to refine the matching costs; and inferring, from the regularized cost volume, an optimal offset of the predicted pose. The optimal offset can be used to determine a location of the ADV.

Abstract: A method, apparatus, and system for determining a low-height obstacle based on outputs of a LIDAR device in an autonomous vehicle is disclosed. A point cloud comprising a plurality of points is generated based on outputs of a LIDAR device. For each point within a first number of lowest rings of points, a neighboring point in a same ring to a first direction is determined, and a first and a second coordinate values-related differences are determined. A first, a second, a third, and a fourth quantities are determined based on the first and second differences. In response to determining that the first, the second, the third, and the fourth quantities satisfy a predetermined condition, a low-height obstacle is determined based on the points within the first number of lowest rings of points. Operations of an autonomous vehicle are controlled based at least in part on the determined low-height obstacle.

Abstract: The present disclosure discloses an information processing method and device, and relates to the field of artificial intelligence, in particular, to a graph neural network in the field of deep learning. A specific implementation solution according to an embodiment includes determining initial representation of edges connected between a plurality of atoms in a molecule based on three-dimensional structure information of the molecule; determining first representation of a neighbor edge of each of the atoms based on the initial representation of the edges, the neighbor edge of each of the atoms indicating at least one edge connected with each of the atoms; determining first representation of each of the atoms based on the first representation of the neighbor edge of each of the atoms; determining feature representation for characterizing the molecule based on the first representation of each of the atoms.

Abstract: Systems and methods are disclosed for collecting driving data from simulated autonomous driving vehicle (ADV) driving sessions and real-world ADV driving sessions. The driving data is processed to exclude manual (human) driving data and to exclude data corresponding to the ADV being stationary (not driving). Data can further be filtered based on driving direction: forward or reverse driving. Driving data records are time stamped. The driving data can be aligned according to the timestamp, then a standardized set of metrics is generated from the collected, filtered, and time-aligned data. The standardized set of metrics are used to grade the performance the control system of the ADV, and to generate an updated ADV controller, based on the standardized set of metrics.

Abstract: Presented herein are novel embodiments for converting a given speech audio or text into a photo-realistic speaking video of a person with synchronized, realistic, and expressive body dynamics. In one or more embodiments, 3D skeleton movements are generated from the audio sequence using a recurrent neural network, and an output video is synthesized via a conditional generative adversarial network. To make movements realistic and expressive, the knowledge of an articulated 3D human skeleton and a learned dictionary of personal speech iconic gestures may be embedded into the generation process in both learning and testing pipelines. The former prevents the generation of unreasonable body distortion, while the later helps the model quickly learn meaningful body movement with a few videos. To produce photo-realistic and high-resolution video with motion details, a part-attention mechanism is inserted in the conditional GAN, where each detailed part is automatically zoomed in to have their own discriminators.

Abstract: In some implementations, a method of verifying operation of a sensor is provided. The method includes causing a sensor to obtain sensor data at a first time, wherein the sensor obtains the sensor data by emitting waves towards a detector. The method also includes determining that the detector has detected the waves at a second time. The method further includes receiving the sensor data from the sensor at a third time. The method further includes verifying operation of the sensor based on at least one of the first time, the second time, or the third time.

Abstract: In one embodiment, a system identifies a road to be navigated by an ADV, the road being captured by one or more point clouds from one or more LIDAR sensors. The system extracts road marking information of the identified road from the point clouds, the road marking information describing one or more road markings of the identified road. The system partitions the road into one or more road partitions based on the road markings. The system generates a point cloud map based on the road partitions, where the point cloud map is utilized to perceive a driving environment surrounding the ADV.

Abstract: In one embodiment, a system receives a stream of frames of point clouds from one or more LIDAR sensors of an ADV and corresponding poses in real-time. The system extracts segment information for each frame of the stream based on geometric or spatial attributes of points in the frame, where the segment information includes one or more segments of at least a first frame corresponding to a first pose. The system registers the stream of frames based on the segment information. The system generates a first point cloud map for the stream of frames based on the frame registration.

Abstract: According to one embodiment, in response to determining that an obstacle blocks at least a portion of a current lane in which an ADV is driving, an obstacle boundary of the obstacle is determined based on the size and shape of the obstacle. A lane configuration is determined based on map data of a map corresponding to a road associated with the lanes. A passing lane boundary that can be utilized by the ADV is determined based on the lane configuration of the road and the obstacle boundary of the obstacle. A passable area is calculated within the passing lane boundary based on a size of the ADV. The passable area is utilized by the ADV to pass the obstacle without collision. Thereafter, a trajectory is planned within the passable area boundary to control the ADV to pass the obstacle.

Abstract: In one embodiment, a method includes the operation of segmenting a raw reference line into a plurality of reference line segments, including a first reference line segment, a second reference segment, and a third reference line segment in a sequential order, in response to receiving the raw reference line representing a route from a first location to a second location associated with an autonomous driving vehicle (ADV). The method further includes the operations of smoothing the first reference line segment and the third reference line segment using a Quadratic programming (QP) spline smoother; and smoothing the second reference line segment using a spiral smoother. Smoothed reference line segments from the plurality of reference line segments are connected to generate a smoothed reference line, which is to be used as a reference line of the route to control the ADV.

Abstract: According to one embodiment, a system receives, at a host system a public attestation key (PK_ATT) or a signed PK_ATT from a data processing (DP) accelerator over a bus. The system verifies the PK_ATT using a public root key (PK_RK) associated with the DP accelerator. In response to successfully verifying the PK_ATT, the system transmits a kernel identifier (ID) to the DP accelerator to request attesting a kernel object stored in the DP accelerator. In response to the system receives a kernel digest or a signed kernel digest corresponding to the kernel object from the DP accelerator, verifying the kernel digest using the PK_ATT. The system sends the verification results to the DP accelerator for the DP accelerator to access the kernel object based on the verification results.

Abstract: Described herein are systems and embodiments for multispectral image demosaicking using deep panchromatic image guided residual interpolation. Embodiments of a ResNet-based deep learning model are disclosed to reconstruct the full-resolution panchromatic image from multispectral filter array (MSFA) mosaic image. In one or more embodiments, the reconstructed deep panchromatic image (DPI) is deployed as the guide to recover the full-resolution multispectral image using a two-pass guided residual interpolation methodology. Experiment results demonstrate that the disclosed method embodiments outperform some state-of-the-art conventional and deep learning demosaicking methods both qualitatively and quantitatively.

Abstract: In one embodiment, a method for generating a reference line for operating an autonomous driving vehicle includes determining a first ending reference point having a smallest curvature among a plurality of points within a first defined distance along a path, generating a first reference line based on a first initial reference point and the first ending reference point, determining a second ending reference point having a smallest curvature among a plurality of points within a second defined distance along the path, generating a second reference line based on the first and second ending reference points and an end section of the first reference line, connecting the first and second reference lines, and controlling the autonomous driving vehicle along the connected first reference line and the second reference line.

Abstract: According to one embodiment, a system receives, at a runtime library executed within a trusted execution environment (TEE) of a host system, a request from an application to invoke a predetermined function to perform a predefined operation. In response to the request, the system identifies a kernel object associated with the predetermined function. The system verifies an executable image of the kernel object using a public key corresponding to a private key that was used to sign the executable image of the kernel object. In response to successfully the system verifies the executable image of the kernel object, transmitting the verified executable image of the kernel object to a data processing (DP) accelerator over a bus to be executed by the DP accelerator to perform the predefined operation.

Abstract: In one embodiment, a lateral drifting error is determined based on at least a current location of an ADV. The lateral drifting error is segmented into a first drifting error and a second drifting error using a predetermined segmentation algorithm. A planning module plans a path or trajectory for a current driving cycle (e.g., planning cycle) to drive the ADV from the current location for a predetermined period of time. The planning module performs a first drifting error correction on the trajectory by modifying at least a starting point of the trajectory based on the first drifting error to generate a modified trajectory. A control module controls the ADV to drive according to the modified trajectory, including performing a second drifting error correction based on the second drifting error.

Abstract: A system is disclosed that receives, at a host system from a data processing (DP) accelerator, an accelerator identifier (ID) that uniquely identifies the DP accelerator, wherein the host system is coupled to the DP accelerator over a bus. The system transmits the accelerator ID to a predetermined trusted server over a network. The system receives a certificate from the predetermined trusted server over the network, the certificate certifying the DP accelerator. The system extracts a public root key (PK_RK) from the certificate for verification, the PK_RK corresponding to a private root key (SK_RK) associated with the DP accelerator. The system establishes a secure channel with the DP accelerator using the PK_RK based on the verification to exchange data securely between the host system and the DP accelerator.

Abstract: In one embodiment, a method for temporal smoothness in localization results for an autonomous driving vehicle includes: creating a probability offset volume that represents an overall matching cost between a first set of keypoints from the online point cloud and a second set of keypoints from a pre-built point cloud map for each of a series of sequential light detection and ranging (LiDAR) frames in an online point cloud. The method also includes compressing the probability offset volume into multiple probability vectors across a X dimension, a Y dimension and a yaw dimension; providing each probability vector of the probability offset volume to a number of recurrent neural networks (RNNs); and generating, by the RNNs, a trajectory of location results across the plurality of sequential LiDAR frames.

Abstract: Presented are systems and methods for improving speed and quality of real-time per-pixel depth estimation of scene layouts from a single image by using an end-to-end Convolutional Spatial Propagation Network (CSPN). An efficient linear propagation model performs propagation using a recurrent convolutional operation. The affinity among neighboring pixels may be learned through a deep convolutional neural network (CNN). The CSPN may be applied to two depth estimation tasks, given a single image: (1) to refine the depth output of existing methods, and (2) to convert sparse depth samples to a dense depth map, e.g., by embedding the depth samples within the propagation procedure. The conversion ensures that the sparse input depth values are preserved in the final depth map and runs in real-time and is, thus, well suited for robotics and autonomous driving applications, where sparse but accurate depth measurements, e.g., from LiDAR, can be fused with image data.

Abstract: In one embodiment, an autonomous driving system perceives a driving environment surrounding an ADV based on sensor data obtained from various sensors, including identifying one or more objects. For each of the objects, an arc curve is generated connecting the object and a current location of the ADV using a predetermined algorithm. A curvature of the arc curve is calculated. One of the arc curves is selected in which the curvature of the selected arc curve satisfies a predetermined condition, such as the lowest curvature amongst all objects. A reference line is then generated by connecting the selected object and the current location of the ADV. The connected line between the selected object and the ADV is utilized as a part of a reference line. The reference line is utilized to generate a trajectory to drive the ADV.