Abstract: A starting point, a set of one or more way points, and a destination point of a route along which the ADV is to be driven are determined. All lane segments near the starting point, the set of way points, and the destination point within a predetermined threshold distance are determined respectively. A set of route candidates are determined with an A-star (A*) searching algorithm based on a set of nodes representing all the lane segments near the starting point, the set of way points, and the destination point respectively. The route is selected from the set of route candidates based on respective costs of the set of route candidates. The ADV is being controlled to drive along the selected route autonomously.

Abstract: Described herein are embodiments of a reinforcement learning based large-scale multi-objective ranking system. Embodiments of the system may be used for optimizing short-video recommendation on a video sharing platform. Multiple competing ranking objective and implicit selection bias in user feedback are the main challenges in real-world platform. In order to address those challenges, multi-gate mixture of experts (MMoE) and soft actor critic (SAC) are integrated together into a MMoE_SAC system. Experiment results demonstrate that embodiments of the MMoE_SAC system may greatly reduce a loss function compared to systems only based on single strategies.

Abstract: The disclosure describes various embodiments for detecting an unexpected control state of an autonomous driving system. According to an embodiment, an exemplary method of detecting an unexpected control state of an autonomous driving system include the operations of generating environmental data of a vehicle; determining, by the autonomous driving system, a first control state based on the environmental data of the vehicle; determining, by a reference model, a second control state based on the environmental data, wherein the reference model defines at least one scenario each corresponding to a plurality of expected control states and a state switching condition, and in each of the expected control states corresponding to the scenario, an action of the vehicle in the scenario obeys a traffic rule; and determining the unexpected control state of the autonomous driving system by comparing the first control state with the second control state.

Abstract: A computer-implement method for operating an ADV is disclosed. A first trajectory for the ADV to drive along is planned. The ADV is to autonomously drive along the first trajectory. An obstacle in an affected region of the ADV is detected based on sensor data obtained from a plurality of sensors mounted on the ADV. An expected residence time of the obstacle in the affected region is determined. Whether to plan a second trajectory or to wait for the obstacle to leave the affected region is determined based on the expected residence time of the obstacle in the affected region. A second trajectory for the ADV to drive along is planned and the ADV is to autonomously drive along the second trajectory, or the ADV is to wait for the obstacle to leave the affected region and to autonomously drive along the first trajectory afterwards.

Abstract: Deep neural network (DNN) models have been widely used for user-relevance content prediction. Presented herein is a new user-relevance framework, embodiments of which may be referred as Gating-Enhanced Multi-task Neural Networks (GemNN). In one or more, neural network-based multi-task learning model embodiments herein predict user engagement with content in a coarse-to-fine manner, which gradually reduces content candidates and allows parameter sharing from upstream tasks to downstream tasks to improve the training efficiency. Also, in one or more embodiments, a gating mechanism was introduced between embedding layers and multi-layer perceptions to learn feature interactions and control the information flow fed to MLP layers. Tested embodiments demonstrated considerable improvements over prior approaches.

Abstract: A method, apparatus, and system for filtering a point cloud generated by a LIDAR device in an autonomous vehicle is disclosed. A point cloud comprising a plurality of points is generated based on outputs of the LIDAR device. The point cloud is filtered to remove a first set of points in the plurality of points that correspond to noise based on one or more of: point intensity measurements, distances between points, or a combination thereof. Perception data is generated based on the filtered point cloud. Operations of the autonomous vehicle are controlled based on the perception data.

Abstract: In some implementations, a method is provided. The method includes determining a first set of data acquisition characteristics of a first sensor of an autonomous driving vehicle. The method also includes determining a second set of data acquisition characteristics of a second sensor of the autonomous driving vehicle. The method further includes synchronizing a first data acquisition time of the first sensor and a second data acquisition time of the second sensor, based on the first set of data acquisition characteristics and the second set of data acquisition characteristics. The first sensor obtains first sensor data at the first data acquisition time. The second sensor obtains second sensor data at the second data acquisition time.

Abstract: According to one embodiment, a DP accelerator includes one or more execution units (EUs) configured to perform data processing operations in response to an instruction received from a host system coupled over a bus. The DP accelerator includes a time unit (TU) coupled to the security unit to provide timestamp services. The DP accelerator includes a security unit (SU) configured to establish and maintain a secure channel with the host system to exchange commands and data associated with the data processing operations, where the security unit includes a secure storage area to store a private root key associated with the DP accelerator, where the private root key is utilized for authentication. The SU includes a random number generator to generate a random number, and a cryptographic engine to perform cryptographic operations on data exchanged with the host system over the bus using a session key derived based on the random number.

Abstract: Described herein are systems and methods that improve the success rate of relocalization and eliminate the ambiguity of false relocalization by exploiting motions of the sensor system. In one or more embodiments, during a relocalization process, a snapshot is taken using one or more visual sensors and a single-shot relocalization in a visual map is implemented to establish candidate hypotheses. In one or more embodiments, the sensors move in the environment, with a movement trajectory tracked, to capture visual representations of the environment in one or more new poses. As the visual sensors move, the relocalization system tracks various estimated localization hypotheses and removes false ones until one winning hypothesis. Once the process is finished, the relocalization system outputs a localization result with respect to the visual map.

Abstract: A method for processing autonomous driving simulation data. The method includes: determining a type of a message transmitted between a simulation system and an auto driving system (ADS); determining a data acquisition mode based on the type of the message; obtaining a data stream transmitted between the simulation system and the ADS based on the data acquisition mode; and determining performance of the ADS based on the data stream.

Abstract: Embodiments of the present disclosure provide a method and apparatus for aligning a paragraph and a video. The method may include: acquiring a commentary and a candidate material resource set corresponding to the commentary, a candidate material resource being a video or an image; acquiring a matching degree between each paragraph in the commentary and each candidate material resource in the candidate material resource set; and determining a candidate material resource sequence corresponding to the each paragraph in the commentary based on the matching degrees between the paragraphs in the commentary and the candidate material resources, playing durations of the candidate material resources and text lengths of the paragraphs in the commentary, an image playing duration being a preset image playing duration.

Abstract: A simulation of an autonomous driving vehicle (ADV) includes capturing first data that includes a control command output by an autonomous vehicle controller of the ADV, and capturing second data that includes the control command being implemented at a control unit of the ADV. The control command, for example, a steering command, a braking command, or a throttle command, is implemented by the ADV to affect movement of the ADV. A latency model is determined based on comparing the first data with the second data, where the latency model defines time delay and/or amplitude difference between the first data and the second data. The latency model is applied in a virtual driving environment.

Abstract: Described herein are systems and methods for object detection to achieve hard real-time performance with low latency. Real-time object detection frameworks are disclosed. In one or more embodiments, a framework comprises a first CPU core, a second CPU core, and a plurality of shaves. In one or more embodiments, the first CPU core handles general CPU tasks, while the second CPU core handles the image frames from a camera sensor and computation task scheduling. In one or more embodiments, the scheduled computation tasks are implemented by the plurality of shaves using at least one object-detection model to detect an object in an image frame. In one or more embodiments, computation results from the object-detection model with a higher detection probability is used to form an output for object detection. In one or more embodiments, the object-detection models share some parameters for smaller size and higher implementing speed.

Abstract: A method, apparatus, and system for determining whether all extrinsic matrices are accurate is disclosed. A plurality of post-LIDAR fusion point clouds that are based on simultaneous outputs from a plurality of LIDAR devices installed at one ADV are obtained. The obtained plurality of point clouds are filtered to obtain a first set of points comprising all points in the plurality of point clouds that fall within a region of interest. Each point in the first set of points corresponds to one coordinate value on the axis in the up-down direction. A distribution of the first plurality of coordinate values is obtained. A quantity of peaks in the distribution of the first plurality of coordinate values is determined. Whether all extrinsic matrices associated with the plurality of LIDAR devices are accurate is determined based on the quantity of peaks in the distribution of the first plurality of coordinate values.

Abstract: Various embodiments of the invention enable an ADV to dynamically adjust its behaviors to emulate behaviors of a vehicle operated by a human driver when the ADV encounters an obstacle. A dynamic cost function can be used to collect real-time values of a set of parameters, and use the real-time values to constantly adjust a preferred safety distance where the ADV can be stopped ahead of the obstacle. An method includes determining a first distance to the obstacle in response to detecting an obstacle ahead of the ADV; and for each of a number of iterations, collecting a real-time value for each of a set of parameters, determining an offset to the first distance using the real-time value for each of the set of parameters, calculating a second distance based on the first distance and the offset, and controlling the ADV in view of the second distance using an expected value of each of the set of parameters, such that the ADV can stop at a point having the second distance to the obstacle.

Abstract: According to various embodiment, described herein are methods and systems for reliably detecting malfunctions in a variety of software or hardware components in an autonomous driving vehicle (ADV). In one embodiment, a redundant system can be provided on an independent computing device in an ADV to check for malfunctions in a number of software or hardware components. When no malfunction occurs in the ADV, an autonomous driving system (ADS) in the ADV operates to drive the ADV, while the redundant system can monitor the ADS in a standby mode. In the event of a malfunction, the redundant system can take over the control of the ADV, and take appropriate actions based on a severity level of the malfunction.

Abstract: A lateral control error of a steering system of an ADV is determined, which includes iteratively performing following operations for a predetermined time period. Whether the ADV is moving within a predetermined proximity of a current moving direction is determined. Next, whether a road condition of a road on which the ADV is driving satisfies a predetermined road condition is determined. Then, a first steering feedback of the ADV in response to a prior steering control command is measured. Thereafter, the lateral control error is determined based on at least a portion of the first steering feedback over the predetermined time period. Further, a steering command in view of the lateral control error of the steering system is generated. Finally, the steering command is applied to control the ADV to compensate the lateral control error of the steering system.

Abstract: One application of deep learning methods and labelled data is for industrial production or work applications. For such applications implemented with machine learning applications, massive amounts of data are required to train, validate, and/or tune models for better fitting the requirements. However, obtaining such data has typically be costly and difficult. Embodiments provide adaptable processes that provide data labelling methods for work settings. Embodiments take advantage of the work or production processes to label and collect data, which save time and money and improves accuracy. Embodiments prevent or reduce the need for worker training costs and human mistake-triggered data labelling problems. Embodiments also improve data labelling quality and speed-up of the development cycle.

Abstract: Embodiments presented herein facilitate improvement of a deployed neural network model's accuracy without significantly affecting its operation. In one or more embodiments, online training of the deployed model may be performed using a second neural network model that has higher accuracy than the deployed neural network model. In one or more embodiments, the second neural network model may also be improved online. Embodiments may be deployed in system, such as edge computing environments, in which neural networks deployed at the edge can be centrally monitored and updated.

Abstract: A method, apparatus, and system for estimating a moving speed of a detected pedestrian at an autonomous driving vehicle (ADV) is disclosed. A pedestrian is detected in a plurality of frames of point clouds generated by a LIDAR device installed at an autonomous driving vehicle (ADV). In each of at least two of the plurality of frames of point clouds, a minimum bounding box enclosing points corresponding to the pedestrian excluding points corresponding to limbs of the pedestrian is generated. A moving speed of the pedestrian is estimated based at least in part on the minimum bounding boxes across the at least two of the plurality of frames of point clouds. A trajectory for the ADV is planned based at least on the moving speed of the pedestrian. Thereafter, control signals are generated to drive the ADV based on the planned trajectory.