Abstract: The present teaching relates to generating an alert in a vehicle. First information indicating an upcoming switch in an operating mode of the vehicle is received, which specifies a set of tasks, arranged in an order, to be completed by a driver in the vehicle to achieve the upcoming switch, and a task duration for each of the set of tasks by which the task is to be completed. A current state of the driver is obtained and used to determine a set of warnings to alert the driver to perform the set of tasks. Each warning corresponds to a task in the set of tasks and is created based on the current state of the driver. A warning schedule is generated based on the set of warnings in the order of the set of tasks and transmitted so that warnings in the warning schedule are delivered to the driver.

Abstract: The present teaching relates to method, system, and medium, for generating an augmented alert in a hybrid vehicle. First information indicating an upcoming switch in an operating mode of the vehicle is received, which specifies a set of tasks, arranged in an order, to be completed by a driver in the vehicle to achieve the upcoming switch, and a task duration for each of the set of tasks by which the task is to be completed. A current state of the driver is obtained and used to determine a set of warnings to alert the driver to perform the set of tasks. Each warning corresponds to a task in the set of tasks and is created based on the current state of the driver. A warning schedule is generated based on the set of warnings in the order of the set of tasks and transmitted so that warnings in the warning schedule are delivered to the driver.

Abstract: The present teaching relates to a method, system, medium, and implementation of processing image data in an autonomous driving vehicle. Sensor data acquired by one or more types of sensors deployed on the vehicle are continuously received. The sensor data provide different information about surrounding of the vehicle. Based on a first data set acquired by a first sensor of a first type of the one or more types of sensors at a specific time, an object is detected, where the first data set provides a first type of information about the surrounding of the vehicle. Depth information of the object is then estimated via object centric stereo at object level based on the object detected as well as a second data set acquired by a second sensor of the first type of the one or more types of sensors at the specific time. The second data set provides the first type of information about the surrounding of the vehicle with a different perspective as compared with the first data set.

Abstract: The present teaching relates to method, system, medium, and implementation of automatic motion planning for an autonomous driving vehicle. Information is obtained with respect to a current location of the autonomous driving vehicle operating in a current vehicle motion, wherein the information is to be used to estimate the operational capability of the autonomous driving vehicle with respect to the current location. Sensor data are obtained, via one or more sensors in one or more media types. A reaction of a passenger present in the vehicle with respect to a current vehicle motion is estimated based on the sensor data. Motion planning is performed based on the information with respect to the current location and the reaction of the passenger to the current vehicle motion.

Abstract: The present teaching relates to method, system, medium, and implementation of human-like vehicle control for an autonomous vehicle. Information related to a target motion to be achieved by the autonomous vehicle is received, wherein the information includes a current vehicle state of the autonomous vehicle. A first vehicle control signal is generated with respect to the target motion and the given vehicle state in accordance with a vehicle kinematic model. A second vehicle control signal is generated in accordance with a human-like vehicle control model, with respect to the target motion, the given vehicle state, and the first vehicle control signal, wherein the second vehicle control signal modifies the first vehicle control signal to achieve human-like vehicle control behavior.

Abstract: The present teaching relates to system, method, medium for in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data acquired continuously by a plurality of types of sensors deployed on the vehicle are first received, where the plurality of types of sensor data provide information about surrounding of the vehicle. Based on at least one model, one or more items are tracked from a first of the plurality of types of sensor data acquired by one or more of a first type of the plurality of types of sensors, wherein the one or more items appear in the surrounding of the vehicle. At least some of the one or more items are then automatically labeled on-the-fly via either cross modality validation or cross temporal validation of the one or more items and are used to locally adapt, on-the-fly, the at least one model in the vehicle.

Abstract: The present teaching relates to method, system, medium, and implementation of automatic motion planning for an autonomous driving vehicle. Information is obtained with respect to a current location of the autonomous driving vehicle operating in a current vehicle motion, wherein the information is to be used to estimate the operational capability of the autonomous driving vehicle with respect to the current location. Sensor data are obtained, via one or more sensors in one or more media types. A reaction of a passenger present in the vehicle with respect to a current vehicle motion is estimated based on the sensor data. Motion planning is performed based on the information with respect to the current location and the reaction of the passenger to the current vehicle motion.

Abstract: The present teaching relates to method, system, medium, and implementation of in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data are acquired continuously via a plurality of types of sensors deployed on the vehicle, where the plurality of types of sensor data provide information about surrounding of the vehicle. One or more items surrounding the vehicle are tracked, based on some models, from a first of the plurality of types of sensor data from a first type of the plurality of types of sensors. A second of the plurality of types of sensor data are obtained from a second type of the plurality of sensors and are used to generate validation base data. Some of the one or more items are labeled, automatically, via validation base data to generate labeled at least some item, which is to be used to generate model updated information for updating the at least one model.

Abstract: The present teaching relates to method, system, and medium, for switching a mode of a vehicle. Real-time data related to the vehicle are received, which include intrinsic/extrinsic capability parameters, based on which a set of tasks to switch from a current mode to a different mode is determined. A first duration of time required for the switch is determined based on a first risk evaluated with respect to the current mode and the real-time data. A task duration time needed by a driver to complete the task is estimated for each of the set of tasks. A second risk for the switching is estimated based on the required first duration of time and a total task duration times needed to complete the set of tasks. The switch is carried out when the second risk satisfying a criterion.

Abstract: The present teaching relates to method, system, and medium, for operating a vehicle. Real-time data related to the vehicle are received. A current mode of operation of the vehicle and a state of the driver present in the vehicle are determined. A first risk associated with the current mode of operation of the vehicle is evaluated based on the real-time data and the state of the driver in accordance with a risk model. In response to the first risk satisfying a first criterion, a second risk associated with switching the current mode to a different mode of operation of the vehicle is determined based on the state of the driver. The vehicle is switched from the current mode to the different mode when the second risk satisfies a second criterion.

Abstract: The present teaching relates to method, system, and medium, for generating an augmented alert in a hybrid vehicle. First information indicating an upcoming switch in an operating mode of the vehicle is received, which specifies a set of tasks, arranged in an order, to be completed by a driver in the vehicle to achieve the upcoming switch, and a task duration for each of the set of tasks by which the task is to be completed. A current state of the driver is obtained and used to determine a set of warnings to alert the driver to perform the set of tasks. Each warning corresponds to a task in the set of tasks and is created based on the current state of the driver. A warning schedule is generated based on the set of warnings in the order of the set of tasks and transmitted so that warnings in the warning schedule are delivered to the driver.

Abstract: The present teaching relates to method, system, medium, and implementation of determining depth information in autonomous driving. Stereo images are first obtained from multiple stereo pairs selected from at least two stereo pairs. The at least two stereo pairs have stereo cameras installed with the same baseline and in the same vertical plane. Left images from the multiple stereo pairs are fused to generate a fused left image and right images from the multiple stereo pairs are fused to generate a fused right image. Disparity is then estimated based on the fused left and right images and depth information can be computed based on the stereo images and the disparity.

Abstract: The present teaching relates to method, system, and medium, for switching a mode of a vehicle. Real-time data related to the vehicle are received, which include intrinsic/extrinsic capability parameters, based on which a set of tasks to switch from a current mode to a different mode is determined. A first duration of time required for the switch is determined based on a first risk evaluated with respect to the current mode and the real-time data. A task duration time needed by a driver to complete the task is estimated for each of the set of tasks. A second risk for the switching is estimated based on the required first duration of time and a total task duration times needed to complete the set of tasks. The switch is carried out when the second risk satisfying a criterion.

Abstract: The present teaching relates to system, method, medium for in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data acquired continuously by a plurality of types of sensors deployed on the vehicle are first received, where the plurality of types of sensor data provide information about surrounding of the vehicle. Based on at least one model, one or more items are tracked from a first of the plurality of types of sensor data acquired by one or more of a first type of the plurality of types of sensors, wherein the one or more items appear in the surrounding of the vehicle. At least some of the one or more items are then automatically labeled on-the-fly via either cross modality validation or cross temporal validation of the one or more items and are used to locally adapt, on-the-fly, the at least one model in the vehicle.

Abstract: The present teaching relates to system, method, medium for in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data are received, which are acquired by a plurality of types of sensors deployed on the vehicle to provide information about surrounding of the vehicle. Based on at least one model, one or more surrounding items are tracked from a first of the plurality of types of sensor data acquired by a first type sensors. At least some of the tracked items are automatically labeled via cross validation and are used to locally adapt, on-the-fly, the at least one model. Model update information is received which from a model update center, which is derived based on the labeled at least some items. The at least one model is updated using the model update information.

Abstract: The present teaching relates to method, system, medium, and implementation of in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data are acquired continuously via a plurality of types of sensors deployed on the vehicle, where the plurality of types of sensor data provide information about surrounding of the vehicle. One or more items surrounding the vehicle are tracked, based on some models, from a first of the plurality of types of sensor data from a first type of the plurality of types of sensors. A second of the plurality of types of sensor data are obtained from a second type of the plurality of sensors and are used to generate validation base data. Some of the one or more items are labeled, automatically, via validation base data to generate labeled at least some item, which is to be used to generate model updated information for updating the at least one model.

Abstract: The present teaching relates to method, system, and medium, for operating a vehicle. The method includes the steps of receiving Real-time data related to the vehicle are received. A current mode of operation of the vehicle is determined. A first risk associated with the current mode of operation of the vehicle is evaluated based on the real-time data in accordance with a risk model. If the first risk satisfies a first criterion, a second risk associated with switching the current mode to a different mode of operation of the vehicle is determined. The vehicle is switched from the current mode to the different mode if the second risk satisfies a second criterion.

Abstract: The present teaching relates to a method, system, medium, and implementation of processing image data in an autonomous driving vehicle. Sensor data acquired by one or more types of sensors deployed on the vehicle are continuously received. The sensor data provide different information about surrounding of the vehicle. Based on a first data set acquired by a first sensor of a first type of the one or more types of sensors at a specific time, an object is detected, where the first data set provides a first type of information about the surrounding of the vehicle. Depth information of the object is then estimated via object centric stereo at object level based on the object detected as well as a second data set acquired by a second sensor of the first type of the one or more types of sensors at the specific time. The second data set provides the first type of information about the surrounding of the vehicle with a different perspective as compared with the first data set.

Abstract: The present teaching relates to a method, system, medium, and implementation of processing image data in an autonomous driving vehicle. Sensor data acquired by one or more types of sensors deployed on the vehicle are continuously received to provide different types of information about surrounding of the vehicle. Based on a first data set acquired by a first sensor of a first type of sensors at a specific time, an object is detected, where the first data set provides a first type of information with a first perspective. Depth information of the object is estimated via object centric stereo based on the object and a second data set, acquired at the specific time by a second sensor of the first type of sensors with a second perspective. The estimated depth information is further enhanced based on a third data set acquired by a third sensor of a second type of sensors at the specific time, providing a second type of information about the surrounding of the vehicle.

Abstract: The present teaching relates to a method, system, medium, and implementation of processing image data in an autonomous driving vehicle. Sensor data acquired by one or more types of sensors deployed on the vehicle are continuously received. The sensor data provide different information about surrounding of the vehicle. Based on a first data set acquired by a first sensor of a first type of the one or more types of sensors at a specific time, an object is detected, where the first data set provides a first type of information about the surrounding of the vehicle. Depth information of the object is then estimated via object centric stereo at object level based on the object detected as well as a second data set acquired by a second sensor of the first type of the one or more types of sensors at the specific time. The second data set provides the first type of information about the surrounding of the vehicle with a different perspective as compared with the first data set.