Abstract: A method for generating training data for machine learning models includes generating a driving scenario, displaying the driving scenario to a group of annotators through one or more interfaces, and providing a feedback interface to collect multimodal feedback from one or more annotators of the group of annotators. The method also includes receiving multimodal feedback from a second annotator of the group of annotators in response to one or more prompts by at least a first annotator of the group of annotators. The one or more prompts corresponding to the driving scenario. The method further includes receiving, from a third annotator, a first evaluation score of the first multimodal feedback. The method also includes integrating the multimodal feedback into a training dataset for training the machine learning model in accordance with the first evaluation score being greater than a score threshold.

Abstract: Systems and methods for enhancing instructions provided by a vehicular virtual assistant are disclosed herein. One embodiment of a virtual assistant enhancement system includes a plurality of markers installed in a vehicle in a corresponding plurality of different locations. The system activates, via a generative artificial intelligence (AI)-based virtual assistant of the vehicle, one or more of the plurality of electronic markers. The system also refers to the one or more activated electronic markers in instructions communicated to a user by the generative AI-based virtual assistant to assist the user in performing a task pertaining to the vehicle.

Abstract: A method for a planner-based driving teacher is described. The method includes predicting a future student driver trajectory in response to a past driving sequence of a student driver and a surrounding area map. The method also includes fusing, by an encoder, a plurality of predicted vehicle trajectories, including the predicted future student driver trajectory, into a compact feature space. The method further includes observing, by a teacher action model, subsequent driving maneuvers of the student driver. The method also includes decoding, by a feature space decoding model, the compact feature space to generate cues for coaching the student driver during subsequent driving maneuvers.

Abstract: Systems and methods for assisting a driver using a foundation model in a shared-autonomy driving mode of a vehicle are disclosed herein. One embodiment of a shared-autonomy assistance subsystem processes, in a vehicle operating in a shared-autonomy driving mode, inputs including vehicle state information, external-road-agent state information, vehicle environmental sensor data, and map data using one or more encoder neural networks that have been trained to extract features for a large language model (LLM). The subsystem inputs the extracted features to the LLM. The subsystem predicts, using the LLM, an objective of a driver of the vehicle. The subsystem then executes, based on an output from the LLM, one or more actions to assist the driver in meeting the predicted objective. The one or more actions include controlling, at least in part, operation of the vehicle.

Abstract: A method for multi-modal annotation is described. The method includes generating, using a training data generation model, a pre-annotated training data for human annotations. The method also includes iteratively verifying the human annotations of the pre-annotated training data using a different annotator from a human annotator or the human annotations. The method further includes adjusting the human annotations of the pre-annotated training data based on interactively verifying to finalize an annotated training data. The method also includes training a machine learning model using the annotated training data.

Abstract: A method for scenario-based event triggers is described. The method includes generating, by a first machine-learning (ML) model, feature vectors encoding driving scenarios surrounding an ego vehicle. The method also includes detecting, by a second machine-learning (ML) model, a unique driving scenario outside of pre-programmed event triggers corresponding to one of the feature vectors encoding driving scenarios surrounding the ego vehicle. The method further includes triggering uploading of the unique driving scenario outside of pre-programmed event triggers to a central scenario-based event control server.

Abstract: Systems, methods, and other embodiments described herein relate to estimating a policy for object motion by training a multi-modal model using diffusion through inferred goals and noise. In one embodiment, a method includes training a multi-modal model to generate a policy using semi-labeled data derived from wild data, and the multi-modal model predicts operator intent and a parameter associated with the policy for an agent in motion. The method also includes expanding outputs from the multi-modal model using noise within a diffusion model, and the noise augmenting the semi-labeled data. The method also includes feeding the outputs including the noise and the wild data to the multi-modal model until satisfying a training parameter associated with the policy.

Abstract: Systems, methods, and other embodiments described herein relate to integrating human decision-making into a model-based system. In one embodiment, a method includes acquiring sensor data, including driver data about a driver of a vehicle and driving data about the vehicle and a surrounding environment of the vehicle. The method includes encoding, using a world encoder, the sensor data into a latent representation. The method includes determining human decision-making characteristics according to the latent representation. The method includes generating a control signal for providing shared control of the vehicle according to the human decision-making characteristics and the latent representation.

Abstract: Systems, methods, and other embodiments described herein relate to predicting future trajectories of ado vehicles and an ego vehicle based on the awareness of the driver of the ego vehicle towards the ado vehicles. In one embodiment, a method includes determining an awareness of a driver of an ego vehicle to ado vehicles in the vicinity of the ego vehicle. The method also includes altering track data of ado vehicles based on a lack of awareness of the driver towards the ado vehicles. The method also includes transmitting altered track data of the ado vehicles to a prediction module. The prediction module predicts future trajectories of the ado vehicles and the ego vehicle based on the altered track data and an ego vehicle track data.

Abstract: Systems and methods are provided for an advanced driver-assistance system (ADAS) that obtains data from a plurality of sensors. In some embodiments, the system can retrieve data regarding a user's past interactions and analyze the data with the sensor data to determine the user's behavior. In some embodiments, the ADAS can determine whether a user is unaware of an ADAS feature based on this behavior and a prompt that recommends the ADAS feature. The user's response to this prompt may be incorporated into the user's behavior for future recommendations.

Abstract: Systems, methods, and other embodiments described herein relate to building the trust of an occupant in an automated vehicle function. In one embodiment, a method for buildling the trust includes acquiring trust data regarding an automated function of a vehicle, an external environment of the vehicle, and an occupant of the vehicle. The method also includes processing the trust data to determine a baseline trust of the occupant in the automated function executing an action for the vehicle. The method also includes identifying a trust level of the occupant. The method further includes determining, based, at least in part, on the trust data and in response to identifying that the trust level satisfies a threshold, a trust message and a content type and a delivery type of the trust message. The method further includes delivering the trust message to the occupant.

Abstract: Systems and methods for identifying artificial intelligence (AI) personas to optimally influence driving behavior are provided. For example, a methodology of the presently disclosed technology may comprise: (1) determining a target driving behavior for a driver of a vehicle based on driving situation; (2) identifying a persona for an AI assistant with a highest predicted probability of influencing the driver to engage in the target driving behavior; and (3) using the AI assistant with the identified persona to present information to the driver to influence the driver to engage in the target driving behavior. In certain embodiments, identifying the persona for the AI assistant with the highest predicted probability of influencing the driver to engage in the target driving behavior may comprise determining the identified persona most reduces, among a plurality of personas, an objective function.

Abstract: Systems and methods for assisting a driver using a foundation model in a shared-autonomy driving mode of a vehicle are disclosed herein. One embodiment of a shared-autonomy assistance subsystem processes, in a vehicle operating in a shared-autonomy driving mode, inputs including vehicle state information, external-road-agent state information, vehicle environmental sensor data, and map data using one or more encoder neural networks that have been trained to extract features for a large language model (LLM). The subsystem inputs the extracted features to the LLM. The subsystem predicts, using the LLM, an objective of a driver of the vehicle. The subsystem then executes, based on an output from the LLM, one or more actions to assist the driver in meeting the predicted objective. The one or more actions include controlling, at least in part, operation of the vehicle.

Abstract: Systems, methods, and other embodiments described herein relate to generating instructions using multiple models for maneuvering on a road according to an operator profile and adapting the instructions using multi-modal data. In one embodiment, a method includes generating an operator profile by a learning model using road history and an operator goal on a road during a driving scenario for a vehicle. The method also includes estimating a driving command using an automated driving system (ADS) and directions using a language model for the driving scenario. The method also includes communicating maneuvers for the road to an operator using the driving command, the directions, the operator profile, and a track profile.

Abstract: A method for a vehicle-based driving simulator is described. The method includes reading a current configuration/setting/driving mode of a vehicle. The method also includes generating a dynamic model of the vehicle based on the current configuration/setting/driving mode of the vehicle. The method further includes selecting a virtual driving scenario for the vehicle according to the current configuration/setting/driving mode of the vehicle. The method includes actuating hardware of the vehicle to simulate performance of the selected virtual driving scenario in the vehicle.

Abstract: Systems, methods, and other embodiments described herein relate to building the trust of an occupant in an automated vehicle function. In one embodiment, a method for building the trust includes acquiring trust data regarding an automated function of a vehicle, an external environment of the vehicle, and an occupant of the vehicle. The method also includes processing the trust data to determine a baseline trust of the occupant in the automated function executing an action for the vehicle. The method also includes identifying a trust level of the occupant. The method further includes determining, based, at least in part, on the trust data and in response to identifying that the trust level satisfies a threshold, a trust message and a content type and a delivery type of the trust message. The method further includes delivering the trust message to the occupant.

Abstract: Systems, methods, and other embodiments described herein relate to detecting dissonance by an operator using a learning model during shared control involving a driving scenario from an operator preference and cue for adapting a driving model. In one embodiment, a method includes detecting characteristics about a driving scenario and an operator from acquired sensor data and an operator factor. The method also includes predicting dissonance for an automated takeover using a learning model with the characteristics, a driving command, and a cue about the operator. The method also includes adapting a shared-driving model (SDM) associated with a vehicle during a maneuver using the dissonance.

Abstract: Systems, methods, and other embodiments described herein relate to training a learning model using labeled data generated through a suggestion from an assisting operator to another operator for executing a task. In one embodiment, a method includes acquiring a driving suggestion from an assisting operator associated with a driving scenario involving a vehicle. The method also includes receiving a driving command and vocal data from the vehicle about following the driving suggestion during the driving scenario. The method also includes training a shared-driving model using the driving suggestion, the driving command, and the vocal data.

Abstract: A system includes sensors that determine one or more attributes associated with associated with an occupant within a vehicle. The system includes one or more datastores, one or more processors, and a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations. The operations include displaying one or more stimuli on a screen within an interior of the vehicle; prompting the occupant to perform one or more visual or motor actions in response to the one or more stimuli; obtaining the one of more attributes; assessing the one or more attributes to determine a level of fitness of the occupant; and based on the level of fitness, activating or deactivating one or more functions within the vehicle.

Abstract: are disclosed herein. One embodiment of a virtual assistant enhancement system includes a plurality of markers installed in a vehicle in a corresponding plurality of different locations. The system activates, via a generative artificial intelligence (AI)-based virtual assistant of the vehicle, one or more of the plurality of electronic markers. The system also refers to the one or more activated electronic markers in instructions communicated to a user by the generative AI-based virtual assistant to assist the user in performing a task pertaining to the vehicle.