Abstract: In some embodiments, a method can include executing a first machine learning model to detect at least one lane in each image from a first set of images. The method can further include determining an estimate location of a vehicle for each image, based on localization data captured using at least one localization sensor disposed at the vehicle. The method can further include selecting lane geometry data for each image, from a map and based on the estimate location of the vehicle. The method can further include executing a localization model to generate a set of offset values for the first set of images based on the lane geometry data and the at least one lane in each image. The method can further include selecting a second set of images from the first set of images based on the set of offset values and a previously-determined offset threshold.

Abstract: The present teaching relates to method, system, and medium, for generating an augmented alert in a hybrid vehicle. First information indicating an upcoming switch in an operating mode of the vehicle is received, which specifies a set of tasks, arranged in an order, to be completed by a driver in the vehicle to achieve the upcoming switch, and a task duration for each of the set of tasks by which the task is to be completed. A current state of the driver is obtained and used to determine a set of warnings to alert the driver to perform the set of tasks. Each warning corresponds to a task in the set of tasks and is created based on the current state of the driver. A warning schedule is generated based on the set of warnings in the order of the set of tasks and transmitted so that warnings in the warning schedule are delivered to the driver.

Abstract: Embodiments provide improved techniques for identifying object and vehicle locations on an HD map using camera images. Synthetic data may be generated by populating a high-definition (HD) map with an object. The populated HD map can be projected to a two-dimensional camera view image depicting the object. The HD map and object detection data generated from image (e.g., identifying lane/object locations within the image) can be used to train the model to identify HD map locations of an autonomous vehicle capturing images as it travels, as well as various objects detected from those captured images. Subsequently, a new image may be processed using object detection techniques to detect lane/object locations within the image. An HD map and the detected lane/object data may be provided to the model, which in turn identifies, on the HD map, the locations of the vehicle and the various objects.

Abstract: A system includes sensors onboard an autonomous vehicle, a processor, and a memory. The memory stores instructions for the processor to receive a first image pair from the sensors at a first time and a second image pair from the sensors at a second time. Each image pair includes at least one static feature in an environment of the autonomous vehicle. The memory also stores instructions to determine a distance travelled by the autonomous vehicle between the first and second times, and to determine a correction to a disparity map based on (1) a first disparity associated with the static feature(s) and the first image pair, (2) a second disparity associated with the static feature(s) and the second image pair, and (3) the distance travelled. The memory also stores instructions to cause the correction to be applied to the disparity map.

Abstract: Methods, systems, and non-transitory computer readable media configured to perform operations comprise determining, by a computing system, speeds and stations of a plurality of leading vehicles traveling ahead of an ego vehicle; calculating, by the computing system, a coasting viability factor based on the speeds and stations of the plurality of leading vehicles and a speed and station of the ego vehicle; comparing, by the computing system, the coasting viability factor to a coasting viability threshold; and determining, by the computing system, whether the ego vehicle should coast based on the comparing.

Abstract: The present teaching relates to a method, system, medium, and implementation of processing image data in an autonomous driving vehicle. Sensor data acquired by one or more types of sensors deployed on the vehicle are continuously received. The sensor data provide different information about surrounding of the vehicle. Based on a first data set acquired by a first sensor of a first type of the one or more types of sensors at a specific time, an object is detected, where the first data set provides a first type of information about the surrounding of the vehicle. Depth information of the object is then estimated via object centric stereo at object level based on the object detected as well as a second data set acquired by a second sensor of the first type of the one or more types of sensors at the specific time. The second data set provides the first type of information about the surrounding of the vehicle with a different perspective as compared with the first data set.

Abstract: A method includes detecting, via a processor of an autonomous vehicle, an upcoming downhill road segment of a route on which the autonomous vehicle is currently travelling. The detection is based on map data, camera data, and/or inertial measurement unit (IMU) data. In response to detecting the upcoming downhill road segment, a descent plan is generated for the autonomous vehicle. The descent plan includes a speed profile and a brake usage plan. The brake usage plan specifies a non-zero amount of retarder usage and an amount of foundation brake usage for a predefined time period. The method also includes autonomously controlling the autonomous vehicle, based on the descent plan, while the autonomous vehicle descends the downhill road segment.

Abstract: A representation of a first set of weights associated with a vehicle prior to a drive are received, the vehicle including a plurality of axles, a plurality of wheels attached to the plurality of axles, and a load. A warning is caused to be output in response to at least one weight from the first set of weights being outside a predetermined range. Longitudinal and lateral dynamics associated with the vehicle during the drive are determined. A second set of weights associated with at least one axle from the plurality of axles during the drive are determined. A determination is made if at least one remedial action should be performed based on the longitudinal dynamics, the lateral dynamics, and the second set of weights. In response to determining that the at least one remedial action should be performed, the at least one remedial action is caused to be performed.

Abstract: In an embodiment, a processor is configured to perform, while a vehicle is driving in an uncontrolled environment, a self-calibration routine for each sensor from the plurality of sensors to determine at least one calibration parameter value associated with that sensor. The processor is further configured to determine, while the vehicle is driving in the uncontrolled environment, and automatically in response to performing the self-calibration routine, that at least one sensor from the plurality of sensors has moved and/or is inoperative based on the at least one calibration parameter value associated with the at least one sensor being outside a predetermined acceptable range. The processor is further configured to perform, in response to determining that at least one sensor from the plurality of sensors has moved and/or is inoperative, at least one remedial action at the vehicle while the vehicle is driving in the uncontrolled environment.

Abstract: Sensor data indicating a substantially 360 degree surrounding of a vehicle is received via a processor included in the vehicle. The sensor data is collected using multiple sensors included in the vehicle. Additionally, an acceptable response time range for a driver of the vehicle to perform an action with the vehicle is obtained, via the processor, based on the sensor data. Additionally, an actual response time for the driver to perform the action is determined, via the processor, based on CAN data collected from a CAN bus included in the vehicle, and not based on the sensor data. Additionally, a determination is made, via the processor, that the actual response time is not within the acceptable response time range. Additionally, a remedial action is caused, via the processor, to be performed in response to the determining that the actual response time is not within the acceptable response time range.

Abstract: In some embodiments, a method includes receiving a first image and a second image from a stereo camera pair. The method includes selecting a first row of pixels from the rectified image and a set of rows of pixels from the second image and comparing the first row of pixels with each row of pixels from the set of rows of pixels to determine disparity values. The method includes determining a pair of rows of pixels having the first row of pixels and a second row of pixels from the set of rows of pixels. The pair of rows of pixels has an offset no greater than an offset between the first row of pixels and each row of pixels from remaining rows of pixels. The method includes adjusting, based on the offset, the relative rotational position between the first stereo camera and the second stereo camera.

Abstract: The present teaching relates to method, system, medium, and implementations for detecting a need for sensor adjustment. Information is received from an inertial measurement unit (IMU) attached to a sensor, including one or more measurements associated with the IMU, where the sensor is deployed on a vehicle for sensing surrounding information to facilitate autonomous driving. The one or more measurements are analyzed with respect to one or more corresponding known measurements of the IMU. The discrepancy between the one or more measurements and the one or more corresponding known measurements is used to determine whether an adjustment to the sensor is needed.

Abstract: The present teaching relates to method and system related to providing an anti-tampering sensor assembly on a vehicle. An inertial measurement unit (IMU) is attached to a structure hosting at least one sensor to form a rigid integral structure, which is deployed on a vehicle with an installation pose to enable the at least one sensor therein to sense surrounding information to facilitate autonomous driving. The installation pose of the IMU is indicative of an installation pose of the at least one sensor. The IMU is configured to obtain, in accordance with a schedule, one or more measurements associated with its state, which can be used to enable detecting tampering of the rigid integral structure.

Abstract: The present teaching relates to method, system, medium, and implementations for detecting sensor tampering. Information is received from an inertial measurement unit (IMU) attached to a structure hosting at least one sensor. The information includes one or more measurements associated with the IMU. The at least one sensor is deployed on a vehicle for sensing surrounding information to facilitate autonomous driving. The one or more measurements are analyzed with respect to pre-determined criteria to detect tampering of the at least one sensor. When the tampering is detected, a response is generated to the event.

Abstract: The present teaching relates to method and system for sensor protection. A sensor protection system includes an assembly having a first structure embedded in a second structure, wherein a sensor is firmly placed in the first structure for sensing information through the assembly and protected from negative impact of outside environment, and one or more devices mounted on the second structure for providing means to clean a slanted surface of the second structure through which the sensor senses the surrounding information, wherein the one or more devices can be individually or jointly activated as needed to clean the slanted surface in order to prevent degradation of information sensed by the sensor through the assembly.

Abstract: In some embodiments, a method comprises receiving, at a processor of an autonomous vehicle and from at least one sensor, sensor data distributed within a time window. A first event being a first event type occurring at a first time in the time window is identified by the processor using a software model based on the sensor data. At least one first attribute associated with the first event is extracted by the processor. A second event being the first event type occurring at a second time in the time window is identified by the processor based on the at least one first attribute. In response to determining that the second event is not yet recognized as being the first event type, a first label for the second event is generated by the processor.

Abstract: The present teaching relates to method, system, and implementations for glare control. A sensor housing assembly is provided on an autonomous vehicle to house at least one sensor therein and protect the at least one sensor from negative impact from an environment including glare from the environment that prevents the at least one sensor from capturing accurate information regarding a field of view with respect to the environment to facilitate autonomous driving of the autonomous vehicle. The negative impact on the at least one sensor is determined by a controller, which then activates a glare blocking mechanism in response to the determined negative impact. The glare blocking mechanism being mounted on the sensor housing assembly and configured to assist in blocking glare with respect to the at least one sensor.

Abstract: This application is directed to augmenting training images used for generating a model for monitoring vehicle drivers. A computer system obtains a first image of a first driver in an interior of a first vehicle and separates, from the first image, a first driver image from a first background image of the interior of the first vehicle. The computer system obtains a second background image and generates a second image by overlaying the first driver image on the second background image. The second image is added to a corpus of training images to be used by a machine learning system to generate a model for monitoring vehicle drivers. In some embodiments, at least one of the first driver image and the second background image is adjusted to match lighting conditions, average intensities, and sizes of the first driver image and the second background image.

Abstract: This application is directed to predicting vehicle actions according to a hierarchy of interconnected vehicle actions. The hierarchy of interconnected vehicle actions includes a plurality of predefined vehicle actions that are organized to define a plurality of vehicle action sequences. A first vehicle obtains one or more images of a road and a second vehicle, and predicts a sequence of vehicle actions of the second vehicle through the hierarchy of interconnected vehicle actions using the one or more images. The first vehicle is controlled to drive at least partially autonomously based on the predicted sequence of vehicle actions of the second vehicle. In some embodiments, the hierarchy of interconnected vehicle actions includes a first action level that is defined according to a stage of a trip and corresponds to three predefined vehicle actions of: “start a trip,” “move in a trip,” and “complete a trip.

Abstract: This application is directed to augmenting training data used for vehicle driving modelling. A computer system obtains a first image of a road and identifies a drivable area of the road within the first image. The computer system obtains an image of an object and generates a second image from the first image by overlaying the image of the object over the drivable area. The second image is added to a corpus of training images to be used by a machine learning system to generate a model for facilitating driving of a vehicle (e.g., at least partial autonomously). In some embodiments, the computer system applies machine learning to train a model using the corpus of training images and distributes the model to one or more vehicles. In use, the model processes road images captured by the one or more vehicles to facilitate vehicle driving.