Abstract: A computer system obtains a plurality of road images captured by one or more cameras attached to one or more vehicles. The one or more vehicles execute a model that facilitates driving of the one or more vehicles. For each road image of the plurality of road images, the computer system determines, in the road image, a fraction of pixels having an ambiguous lane marker classification. Based on the fraction of pixels, the computer system determines whether the road image is an ambiguous image for lane marker classification. In accordance with a determination that the road image is an ambiguous image for lane marker classification, the computer system enables labeling of the image and adds the labeled image into a corpus of training images for retraining the model.

Abstract: Techniques are described for providing a hands-off steering wheel detection warning. An example method can include a vehicle computer determining a real-time level of fatigue of a driver of an autonomous vehicle. The vehicle computer can determine an operating parameter associated with an environment in which the autonomous vehicle is traveling. The vehicle computer can generate, using a machine learning model, a predicted driving pattern of a second vehicle traveling in the environment. The vehicle computer can determine a time interval for providing a hands-off steering wheel detection warning based at least in part on the real-time level of fatigue of the driver, the operating parameter, and the predicted driving pattern of the second vehicle. The vehicle computer can identify a final time interval for providing a hands-off steering wheel detection warning. The vehicle computer can output the hands-off steering wheel detection warning after the final time interval has elapsed.

Abstract: Embodiments provide a vehicle computer coupled to a vehicle. The vehicle computer may be configured to receive sensor data from a plurality of sensors of the autonomous vehicle, analyze the sensor data, predict a need for a steering torque overlay based on analyzing the sensor data, determine a steering torque overlay output based on at least the sensor data, provide the steering torque overlay output to a steering system of the vehicle, and control the steering system in view of the steering torque overlay output. The steering torque overlay output is provided to the steering system while the vehicle is transitioning from an autonomous mode or a manual mode to a hybrid mode where the steering system is controlled simultaneously by the vehicle computer and a driver when the steering torque overlay output is applied.

Abstract: Techniques are described for determining failover to a redundant braking system. An example method can include a system determining a fault occurring in a braking system of an autonomous vehicle (AV) operating in an autonomous mode, the braking system comprising a first braking system and a second braking system, the fault occurring in at least one of the first braking system and the second braking system, and the AV using the first braking system. The system can further determine whether to switch from using the first braking system based at least in part on determining the fault. The system can further transmit to an alert system of the AV, a request to generate an alert for manual control of the AV based at least in part on determining the fault. The system can further deactivate the autonomous mode of the AV based at least in part on determining the fault.

Abstract: A method includes an initial trailer health assessment and real-time trailer health monitoring. The initial trailer health assessment includes autonomous pre-trip maneuvers of the autonomous vehicle during a first time period, and detecting a pre-trip vehicle health condition. A vehicle health score is calculated based on the pre-trip vehicle health condition. If the vehicle health score is at least a threshold value, real-time trailer health monitoring is performed during a trip of the autonomous vehicle during a second time period, by actively monitoring vehicle dynamics data and/or image data associated with the autonomous vehicle, to determine a fault condition of the autonomous vehicle. If the fault condition meets a first criteria, a control parameter and/or a travel plan of the autonomous vehicle is adjusted. If the fault condition meets a second criteria different from the first criteria, a signal is sent to cause the autonomous vehicle to cease movement.

Abstract: An integrated sensor assembly is described. The integrated sensor assembly can include a housing comprising a compartment. The housing can be arranged on a side surface of a cabin of an autonomous vehicle and within a threshold distance of a cabin roof of the autonomous vehicle. The assembly can include a first sensor arranged in the compartment, the first sensor configured to receive a first signal from a vehicle computer and collect information for mapping an environment of the autonomous vehicle. The assembly can include a light source arranged in the compartment. The light source configured to receive a second signal from the vehicle computer and emit a light pattern for visually indicating one of a hazard, a turn, or an autonomous mode. The light source can be arranged in the compartment to emit light to traffic toward a rear of the autonomous vehicle.

Abstract: In an embodiment, a method comprises detecting, at a processor of an autonomous vehicle, a discrepancy between a map and a property sensed by at least one sensor onboard the autonomous vehicle, the property being associated with an external environment of the autonomous vehicle. In response to detecting the discrepancy, and based on the discrepancy, an annotation for the map is generated via the processor. A signal representing the annotation is caused to be transmit to a compute device that is remote from the autonomous vehicle. A signal representing a map update is received from the compute device that is remote form the autonomous vehicle. The map update is generated based on the annotation, the map update (1) including replacement information for a region of the map associated with the annotation, and (2) not including replacement information for a remainder of the map.

Abstract: In some embodiments, a method comprises receiving, at a processor of an autonomous vehicle and from at least one sensor, sensor data distributed within a time window. A first event being a first event type occurring at a first time in the time window is identified by the processor using a software model based on the sensor data. At least one first attribute associated with the first event is extracted by the processor. A second event being the first event type occurring at a second time in the time window is identified by the processor based on the at least one first attribute. In response to determining that the second event is not yet recognized as being the first event type, a first label for the second event is generated by the processor.

Abstract: The present teaching relates to method, system, medium, and implementation of a global model update center. At least one model is established at the model update center for detecting objects surrounding each of autonomous driving vehicles of a fleet. A plurality of labeled data items are received, from the fleet of autonomous driving vehicles, where each of the labeled data items is detected, based on the at least one model, from sensor data characterizing surroundings of the autonomous driving vehicles. The labeled data items are generated automatically on-the-fly by the autonomous driving vehicles. Based on the received labeled data items, at least some of the models are updated and model update information is accordingly generated. Such generated model update information is then distributed to the fleet of autonomous driving vehicles.

Abstract: The present teaching relates to method, system, medium, and implementation of lane planning in an autonomous vehicle. Sensor data are received that capture ground images of a road the autonomous vehicle is on. Based on the sensor data, a current lane of the road that autonomous vehicle is currently occupying is detected. Information indicating presence of a passenger in the vehicle is obtained and used to retrieve a personalized lane control model related to the passenger. Lane control for the autonomous vehicle is planned based on the detected current lane and self-aware capability parameters in accordance with the personalized lane control model. The self-aware capability parameters are used to predict operational capability of the autonomous vehicle with respect to a current location of the autonomous vehicle. The personalized lane control model is generated based on recorded human driving data.

Abstract: This application is directed to aerial view generation for vehicle control. A vehicle obtains a forward-facing view of a road captured by a front-facing camera of a vehicle and applies a machine learning model to process the forward-facing view to predict determine a trajectory of the vehicle and a road layout based on a Frenet-Serret coordinate system of the road for the vehicle. The trajectory of the vehicle is combined with the road layout to predict an aerial view of the road, and the aerial view of the road is used to at least partially autonomously drive the vehicle. In some embodiments, the machine learning model is applied to process the forward-facing view to determine a first location of each of an obstacle vehicle in the Frenet-Serret coordinate system. The first location of the obstacle vehicle is converted to a vehicle location on the aerial view of the road.

Abstract: The present teaching relates to method, system, medium, and implementation of lane planning in an autonomous vehicle. Sensor data are received that capture ground images of a road the autonomous vehicle is on. Based on the sensor data, a current lane of the road that autonomous vehicle is currently occupying is detected. Lane control for the autonomous vehicle is planned based on the detected current lane and self-aware capability parameters in accordance with a driving lane control model. The self-aware capability parameters are used to predict operational capability of the autonomous vehicle with respect to a current location of the autonomous vehicle. The driving lane control model is generated based on recorded human driving data to achieve human-like lane control behavior in different scenarios.

Abstract: Methods, systems, and non-transitory computer readable media are configured to perform operations comprising determining an occurrence of a sign in an environment of a vehicle; determining that at least a portion of the sign is unrecognized by a local machine learning model of the vehicle; and providing sensor data associated with the at least one portion of the sign to an operations center remote from the vehicle.

Abstract: The present teaching relates to method, system, medium, and implementation of in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data are acquired continuously via a plurality of types of sensors deployed on the vehicle, where the plurality of types of sensor data provide information about surrounding of the vehicle. One or more items surrounding the vehicle are tracked, based on some models, from a first of the plurality of types of sensor data from a first type of the plurality of types of sensors. A second of the plurality of types of sensor data are obtained from a second type of the plurality of sensors and are used to generate validation base data. Some of the one or more items are labeled, automatically, via validation base data to generate labeled at least some item, which is to be used to generate model updated information for updating the at least one model.

Abstract: This application is directed to predicting vehicle turn and brake actions for at least partially autonomous vehicle driving. A first vehicle obtains a plurality of images along a road. For each image of the plurality of images, the first vehicle detects, from the image, a plurality of image regions each corresponding to a respective vehicle light of a second vehicle positioned on the road near the first vehicle. The first vehicle determines, for each image region, a probability that a respective vehicle light of the second vehicle changed its state. The first vehicle predicts a vehicle action of the second vehicle based on the probability for each image region. The first vehicle at least partially autonomously driving the first vehicle based on the predicted vehicle action of the second vehicle.

Abstract: In some embodiments, an apparatus, comprises an unmanned vehicle configured to be disposed with a vehicle and a processor operatively coupled to the unmanned vehicle. The processor is configured to receive a first signal indicating a stop of the vehicle without a user request and a second signal indicating a location of the vehicle. The processor is configured to determine, based on the location of the vehicle, a target location in a pre-defined area surrounding the vehicle. The processor is configured to send a third signal to the unmanned vehicle to instruct the unmanned vehicle to move to the target location to alert other vehicles via a warning regarding occurrence of the stop.

Abstract: Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising determining map data and detection data for an area in an environment; determining a change in the area based on the map data and the detection data; and generating control data based on the change.

Abstract: Methods, systems, and non-transitory computer-readable media are configured to perform operations comprising determining lane detection data and object detection data associated with an environment; determining a lane template based on the lane detection data; and generating localization data that identifies a location of an object in the environment based on the lane template and the object detection data.

Abstract: In one or more embodiments, a method comprises receiving, at a processor, an input signal from an input device in response to a first actuation of the input device by a driver of an autonomous vehicle. The input device is a device disposed with the autonomous vehicle and has a second actuation of the input device associated with a standard operation of the input device. The second actuation has an actuation pattern different from an actuation pattern of the first actuation. In response to the input signal, a determination is made by the processor to determine whether the autonomous vehicle can perform a maneuver safely. In response to determining that the autonomous vehicle can perform the maneuver safely, a signal is sent by the processor to cause the autonomous vehicle to perform the maneuver.

Abstract: The present teaching relates to method, system, medium, and implementation of in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data are acquired continuously via a plurality of types of sensors deployed on the vehicle, where the plurality of types of sensor data provide information about surrounding of the vehicle. One or more items surrounding the vehicle are tracked, based on some models, from a first of the plurality of types of sensor data from a first type of the plurality of types of sensors. A second of the plurality of types of sensor data are obtained from a second type of the plurality of sensors and are used to generate validation base data. Some of the one or more items are labeled, automatically, via validation base data to generate labeled at least some item, which is to be used to generate model updated information for updating the at least one model.