Abstract: Systems, methods, and other embodiments described herein relate to building the trust of an occupant in an automated vehicle function. In one embodiment, a method for buildling the trust includes acquiring trust data regarding an automated function of a vehicle, an external environment of the vehicle, and an occupant of the vehicle. The method also includes processing the trust data to determine a baseline trust of the occupant in the automated function executing an action for the vehicle. The method also includes identifying a trust level of the occupant. The method further includes determining, based, at least in part, on the trust data and in response to identifying that the trust level satisfies a threshold, a trust message and a content type and a delivery type of the trust message. The method further includes delivering the trust message to the occupant.

Abstract: Systems and methods for identifying artificial intelligence (AI) personas to optimally influence driving behavior are provided. For example, a methodology of the presently disclosed technology may comprise: (1) determining a target driving behavior for a driver of a vehicle based on driving situation; (2) identifying a persona for an AI assistant with a highest predicted probability of influencing the driver to engage in the target driving behavior; and (3) using the AI assistant with the identified persona to present information to the driver to influence the driver to engage in the target driving behavior. In certain embodiments, identifying the persona for the AI assistant with the highest predicted probability of influencing the driver to engage in the target driving behavior may comprise determining the identified persona most reduces, among a plurality of personas, an objective function.

Abstract: Systems and methods for assisting a driver using a foundation model in a shared-autonomy driving mode of a vehicle are disclosed herein. One embodiment of a shared-autonomy assistance subsystem processes, in a vehicle operating in a shared-autonomy driving mode, inputs including vehicle state information, external-road-agent state information, vehicle environmental sensor data, and map data using one or more encoder neural networks that have been trained to extract features for a large language model (LLM). The subsystem inputs the extracted features to the LLM. The subsystem predicts, using the LLM, an objective of a driver of the vehicle. The subsystem then executes, based on an output from the LLM, one or more actions to assist the driver in meeting the predicted objective. The one or more actions include controlling, at least in part, operation of the vehicle.

Abstract: A method for a vehicle-based driving simulator is described. The method includes reading a current configuration/setting/driving mode of a vehicle. The method also includes generating a dynamic model of the vehicle based on the current configuration/setting/driving mode of the vehicle. The method further includes selecting a virtual driving scenario for the vehicle according to the current configuration/setting/driving mode of the vehicle. The method includes actuating hardware of the vehicle to simulate performance of the selected virtual driving scenario in the vehicle.

Abstract: Systems, methods, and other embodiments described herein relate to generating instructions using multiple models for maneuvering on a road according to an operator profile and adapting the instructions using multi-modal data. In one embodiment, a method includes generating an operator profile by a learning model using road history and an operator goal on a road during a driving scenario for a vehicle. The method also includes estimating a driving command using an automated driving system (ADS) and directions using a language model for the driving scenario. The method also includes communicating maneuvers for the road to an operator using the driving command, the directions, the operator profile, and a track profile.

Abstract: Systems, methods, and other embodiments described herein relate to building the trust of an occupant in an automated vehicle function. In one embodiment, a method for building the trust includes acquiring trust data regarding an automated function of a vehicle, an external environment of the vehicle, and an occupant of the vehicle. The method also includes processing the trust data to determine a baseline trust of the occupant in the automated function executing an action for the vehicle. The method also includes identifying a trust level of the occupant. The method further includes determining, based, at least in part, on the trust data and in response to identifying that the trust level satisfies a threshold, a trust message and a content type and a delivery type of the trust message. The method further includes delivering the trust message to the occupant.

Abstract: Systems, methods, and other embodiments described herein relate to detecting dissonance by an operator using a learning model during shared control involving a driving scenario from an operator preference and cue for adapting a driving model. In one embodiment, a method includes detecting characteristics about a driving scenario and an operator from acquired sensor data and an operator factor. The method also includes predicting dissonance for an automated takeover using a learning model with the characteristics, a driving command, and a cue about the operator. The method also includes adapting a shared-driving model (SDM) associated with a vehicle during a maneuver using the dissonance.

Abstract: Systems, methods, and other embodiments described herein relate to training a learning model using labeled data generated through a suggestion from an assisting operator to another operator for executing a task. In one embodiment, a method includes acquiring a driving suggestion from an assisting operator associated with a driving scenario involving a vehicle. The method also includes receiving a driving command and vocal data from the vehicle about following the driving suggestion during the driving scenario. The method also includes training a shared-driving model using the driving suggestion, the driving command, and the vocal data.

Abstract: A system includes sensors that determine one or more attributes associated with associated with an occupant within a vehicle. The system includes one or more datastores, one or more processors, and a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations. The operations include displaying one or more stimuli on a screen within an interior of the vehicle; prompting the occupant to perform one or more visual or motor actions in response to the one or more stimuli; obtaining the one of more attributes; assessing the one or more attributes to determine a level of fitness of the occupant; and based on the level of fitness, activating or deactivating one or more functions within the vehicle.

Abstract: are disclosed herein. One embodiment of a virtual assistant enhancement system includes a plurality of markers installed in a vehicle in a corresponding plurality of different locations. The system activates, via a generative artificial intelligence (AI)-based virtual assistant of the vehicle, one or more of the plurality of electronic markers. The system also refers to the one or more activated electronic markers in instructions communicated to a user by the generative AI-based virtual assistant to assist the user in performing a task pertaining to the vehicle.

Abstract: A method for triggering capture of diverse driving data from captions is described. The method includes training a discriminator network to identify similarities between a received text description and a received scene description. The method also includes feeding a trained discriminator network with real scene information along with text/sentence descriptions to verify whether the real scene information matches the text/sentence description. The method further includes generating a dataset of diverse driving scenarios retrieved from a dataset of vehicle driving log data in response to a text/sentence query.

Abstract: A method for a driver prediction system is described. The method includes training a neural network to a learn a set of polynomial basis functions. The method also includes selecting, using a trained neural network, a learned polynomial basis function to perform a prediction of an action of an autonomous dynamic object (ADO) agent. The method further includes computing projection coefficients of the learned polynomial basis function to weigh the learned polynomial basis function. The method also includes using the projection coefficients of the learned polynomial basis function to weigh the learned polynomial basis function to provide a distribution regarding a likelihood of the prediction of the action of the ADO agent.

Abstract: A method for controlling an ego agent includes periodically receiving policy information comprising a spatial environment observation and a current state of the ego agent. The method also includes selecting, for each received policy information, a low-level policy from a number of low-level policies. The low-level policy may be selected based on a high-level policy. The method further includes controlling an action of the ego agent based on the selected low-level policy.

Abstract: A method for task-informed planning by a behavior planning system of a vehicle includes observing a previous trajectory of an agent within a distance from the vehicle. The method also includes predicting, by the behavior planning system, a set of potential trajectories for the agent and/or the vehicle based on observing the previous trajectory. The method further includes selecting, by the behavior planning system, a potential action from a set of potential actions associated with a task to be performed by the vehicle, each potential action being associated with a utility value based on the respective potential action and the set of potential trajectories, the selected potential action being associated with a highest utility value of respective utility values associated with the set of potential actions. The method still further includes controlling the vehicle to perform an action associated with the potential action selected by the behavior planning system.

Abstract: Systems and methods described herein relate to using multimodal foundation models. In one embodiment, a method includes receiving images and a foundation multi-model, selecting a mask set, modifying the foundation multi-model to include query, key, and value matrices, and applying the mask set to the foundation multi-model to obtain patch-aligned features.

Abstract: Systems and methods are provided for generating a statistical parameter representing a state of a surgical procedure from sensor data. Sensor data representing a time period. is received from a sensor. Numerical features representing the time period are generated from the sensor data. Each of a plurality of long short term memory units are updated according to the plurality of numerical features via a message passing process. The long short term memory units are connected to form a graph, with a first set of the long short term memory units representing a plurality of nodes of the graph and a second set of the long short term memory units representing a plurality of hyperedges of the graph. A statistical parameter representing a state of the surgical procedure for the time period is derived from an output of one of the long short term memory units and provided to a user.

Abstract: A method for indicating occlusion information at an ego agent includes observing a spatial area from a first viewpoint of one or more first sensors associated with the ego agent. The method also includes identifying the spatial area as an occluded area in accordance with observing the spatial area from a second viewpoint of the one or more first sensors after observing the spatial area from the first viewpoint. The method further includes receiving, from a target agent, a message indicating the spatial area is occluded from one or more second sensors associated with the target agent. The method still further includes transmitting, to the target agent in accordance with receiving the message, the occlusion information indicating information associated the spatial area based on identifying the spatial area as the occluded area.

Abstract: Systems and methods of model or prediction algorithm selection are provided. An autonomous control system may include a perception component that, based on environmental inputs regarding an object(s), a vehicle's operating characteristics, etc., outputs a current state of the vehicle's surrounding environment. This in turn, is used as input to a prediction component comprising a plurality of prediction algorithms. The prediction component outputs a set of predictions regarding the trajectory of the object(s). Accordingly, for each object, a set of trajectories at specific timesteps may be generated by the different prediction algorithms which are input to a planner component. These trajectories may then be analyzed, compared, or otherwise processed to determine which trajectory regarding the object is most accurate. The prediction algorithm or model that produced the most accurate predicted trajectory may then be used for subsequent predictions/timesteps.

Abstract: Systems, methods, and other embodiments described herein relate to integrating human decision-making into a model-based system. In one embodiment, a method includes acquiring sensor data, including driver data about a driver of a vehicle and driving data about the vehicle and a surrounding environment of the vehicle. The method includes encoding, using a world encoder, the sensor data into a latent representation. The method includes determining human decision- making characteristics according to the latent representation. The method includes generating a control signal for providing shared control of the vehicle according to the human decision-making characteristics and the latent representation.

Abstract: In one embodiment, a computer-implemented method for driver training using zone of proximal learning (ZPL) includes receiving, by one or more processors, driving data with respect to a driver operating a vehicle, estimating, using a personal behavior model, a driver profile based on the driving data, estimating one or more zone of proximal development (ZPD) states based at least in part on the driver profile, and performing one or more vehicle actions to place the driver into the one or more ZPD states.