Abstract: A sensor system includes a ranged sensor that generates time-series data indicating positions of objects in an environment, and at least one processor that receives the time-series data generated by the ranged sensor, encodes the time-series data into edge embeddings with an encoder, and computes edge features and edge logits of the objects in the environment, represented in a latent space, based on the edge embeddings. The at least one processor also disentangles the edge features in the latent space, and generates a representation of time-invariant latent characteristics of interactions between the edge features in the latent space.

Abstract: A vehicle includes a ranged sensor that generates time-series data indicating positions of objects in an environment surrounding the vehicle, a user interface configured to warn the driver of a predicted collision between the vehicle and one of the objects in the environment, and at least one processor including an ECU operatively connected to the ranged sensor and the user interface. The processor records control inputs by the driver driving the vehicle, and develops a driver behavior model associated with the driver driving the vehicle based on the control inputs. The processor also predicts trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicts a collision between the vehicle and one of the objects based on the predicted trajectories. The processor also generates a warning indicating the predicted collision to the driver.

Abstract: A system and method for improving the accuracy of human-object interaction tracking includes a unified tracking system. The tracking system uses an autoregressive architecture to process incoming image data and motion data in real-time and generates mesh states and a pose distribution. Post sampling leverages motion data to select optimal samples from the pose distribution.

Abstract: According to one aspect, enhancing reasoning capabilities in a vision language model (VLM) with generative flow networks (GFlowNets) may include generating a chain of thought (CoT) reasoning and an action for a first time-step based on a vision language model (VLM), an input observation image, and an input text prompt, generating an action space for a second time-step and a sequence of transitions based on a simulation environment, the CoT, and the action for the first time-step, and fine-tuning the VLM based on updating a forward policy of a generative flow network (Gflownet) based on buffering the sequence of transitions and one or more losses.

Abstract: A system and method for providing a situational awareness based adaptive driver vehicle interface that include receiving data associated with a driving scene of an ego vehicle and eye gaze data and analyzing the driving scene and the eye gaze data and performing real time fixation detection pertaining to the driver's eye gaze behavior to determine a level of situational awareness with respect to objects that are located within the driving scene. The system and method also include determining at least one level of importance associated with each of the objects and communicating control signals to control at least one component based on at least one of: the at least one level of importance associated with each of the objects that are located within the driving scene and the level of situational awareness with respect to each of the objects that are located within the driving scene.

Abstract: According to one aspect, ego-body pose estimation may include imputing a trajectory and a pose for a hand of a human based on an ego-centric input video and an input head tracking signal, generating a whole-body pose for the human based on the trajectory and the pose for the hand and by denoising the trajectory and the pose of the hand, and implementing an action based on the whole-body pose determined for the human via an actuator.

Abstract: According to one aspect, causal trajectory prediction may include generating a sparsified causal graph including two or more nodes and two or more edges. A node of the two or more nodes may represent an agent of one or more agents within an environment. An edge of the two or more edges between a first node and a second node may represent a causal relationship between the first node and the second node. The computer-implemented method for causal trajectory prediction may include generating one or more agent future features based on the sparsified causal graph and an encoder and generating a trajectory prediction for a target agent based on the one or more agent future features and a decoder.

Abstract: According to one aspect, spatiotemporal stimuli-aware video affective reasoning may include identifying one or more event-driven frames from a set of one or more frames of a training video based on an optical flow associated with one or more of the frames of the training video and training a projector based on the event-driven frames and an associated emotional response. The projector may receive an encoding of the event-driven frames and generate a visual token indicative of the event-driven frames based on the encoding of the event-driven frames.

Abstract: Systems and methods for training a neural network for generating a reasoning statement are provided. In one embodiment, a method includes receiving sensor data from a perspective of an ego agent. The method includes identifying a plurality of captured objects in the at least one roadway environment. The method includes receiving a set of ranking classifications for a captured object of the plurality of captured objects. The annotator reasoning statement is a natural language explanation for the applied attribute. The method includes generating a training dataset for the object type including the annotator reasoning statements of the set of ranking classifications that include the applied attribute from the plurality of importance attributes in the importance category. The method includes training the neural network to generate a generated reasoning statement based on the training dataset in response to a training agent detecting a detected object of the object type.

Abstract: A sensor system includes a ranged sensor that generates time-series data indicating positions of objects in an environment, and at least one processor that receives the time-series data generated by the ranged sensor, encodes the time-series data into edge embeddings with an encoder, and computes edge features and edge logits of the objects in the environment, represented in a latent space, based on the edge embeddings. The at least one processor also disentangles the edge features in the latent space, and generates a representation of time-invariant latent characteristics of interactions between the edge features in the latent space.

Abstract: A system for generating a control action for an ego-vehicle based on an action prediction for each participant within an operating environment is achieved through the generation of a time-varying dynamic causal graph. The system comprises a memory storing one or more instructions and a processor executing one or more stored instructions. The processor is configured to generate the time-varying dynamic causal graph of one or more participants within the operating environment, including the ego-vehicle, one or more agents, and one or more potential obstacles. Additionally, the processor generates the action prediction for each participant within the operating environment based on the dynamic causal graph. Furthermore, the processor generates the control action for the ego-vehicle based on the action prediction for each participant within the operating environment.

Abstract: A vehicle includes a ranged sensor that generates time-series data indicating positions of objects in an environment surrounding the vehicle, a user interface configured to warn the driver of a predicted collision between the vehicle and one of the objects in the environment, and at least one processor including an ECU operatively connected to the ranged sensor and the user interface. The processor records control inputs by the driver driving the vehicle, and develops a driver behavior model associated with the driver driving the vehicle based on the control inputs. The processor also predicts trajectories of the objects and the vehicle based on the time-series data and the driver behavior model, and predicts a collision between the vehicle and one of the objects based on the predicted trajectories. The processor also generates a warning indicating the predicted collision to the driver.

Abstract: Systems and methods for training a neural network for generating a reasoning statement are provided. In one embodiment, a method includes receiving sensor data from a perspective of an ego agent. The method includes identifying a plurality of captured objects in the at least one roadway environment. The method includes receiving a set of ranking classifications for a captured object of the plurality of captured objects. The annotator reasoning statement is a natural language explanation for the applied attribute. The method includes generating a training dataset for the object type including the annotator reasoning statements of the set of ranking classifications that include the applied attribute from the plurality of importance attributes in the importance category. The method includes training the neural network to generate a generated reasoning statement based on the training dataset in response to a training agent detecting a detected object of the object type.

Abstract: According to one aspect, causal graph chain reasoning predictions may be implemented by generating a causal graph of one or more participants within an operating environment including an ego-vehicle, one or more agents, and one or more potential obstacles, generating a prediction for each participant within the operating environment based on the causal graph, and generating an action for the ego-vehicle based on the prediction for each participant within the operating environment. Nodes of the causal graph may represent the ego-vehicle or one or more of the agents. Edges of the causal graph may represent a causal relationship between two nodes of the causal graph. The causal relationship may be a leader-follower relationship, a trajectory-dependency relationship, or a collision relationship.

Abstract: According to one aspect, discovering interpretable dynamically evolving relations (DIDER) may including using a DIDER model for multi-agent interactions represented by an execution set of edge embeddings indicative of trajectory interactions between two or more agents for one or more time steps. The DIDER model may be trained by feeding a training set of edge embeddings to a long short-term memory network (LSTM) forward to generate an LSTM forward output, feeding the training set of edge embeddings to a LSTM reverse to generate an LSTM reverse output, feeding the LSTM forward output to a duration encoder to generate an edge duration output, and training the DIDER model based on a probability distribution for one or more different edge types obtained by feeding the LSTM forward output or the LSTM reverse output to an edge prior and an edge encoder.

Abstract: A system and method for providing a situational awareness based adaptive driver vehicle interface that include receiving data associated with a driving scene of an ego vehicle and eye gaze data and analyzing the driving scene and the eye gaze data and performing real time fixation detection pertaining to the driver's eye gaze behavior to determine a level of situational awareness with respect to objects that are located within the driving scene. The system and method also include determining at least one level of importance associated with each of the objects and communicating control signals to control at least one component based on at least one of: the at least one level of importance associated with each of the objects that are located within the driving scene and the level of situational awareness with respect to each of the objects that are located within the driving scene.

Abstract: A telescopic differential screw mechanism based 3-DOF Parallel Manipulator system to enable differential length and omnidirectional bending is provided. The telescopic differential screw mechanism based 3-DOF Parallel Manipulator system includes a first circular rotating plate 102, a second circular rotating plate 104, three or more telescopic screw assemblies 106A-C and three or more actuators 108A-C. Each telescopic screw assembly 106 includes a pair of master screws 110, a pair of successive screws 112 and a universal joint 116. The three or more telescopic screw assemblies 106A-C are actuated differentially using the three or more actuators 108A-C to achieve omnidirectional bending with high angular rotations in a range of 0 to 75 degree.

Abstract: A multidirectional locomotive module with omnidirectional bending that is compliant along multiple axis is provided. The locomotive module includes, (A) a first part that includes (i) one or more circular rigid components which are coupled using a two degree of freedom joint, (ii) bending actuator that actuates the two degree of freedom joint enabling bending of the multidirectional locomotive module to an angle ranging from 0 to 90 degrees about a Z-axis in a direction to achieve surface compliance with an external surface, and (B) a second part that is elongated in shape with circular cross-section along a surface length and hemispherical in shape an end portion with a surface that is formed by a power transmission sprocket chain and an arrangement of curved components enabling sideways rolling of the multidirectional locomotive module, also enabling wheeled and legged locomotion in vertical position.

Abstract: A multidirectional locomotive module with omnidirectional bending that is compliant along multiple axis is provided.

Abstract: A telescopic differential screw mechanism based 3-DOF Parallel Manipulator system to enable differential length and omnidirectional bending is provided. The telescopic differential screw mechanism based 3-DOF Parallel Manipulator system includes a first circular rotating plate 102, a second circular rotating plate 104, three or more telescopic screw assemblies 106A-C and three or more actuators 108A-C. Each telescopic screw assembly 106 includes a pair of master screws 110, a pair of successive screws 112 and a universal joint 116. The three or more telescopic screw assemblies 106A-C are actuated differentially using the three or more actuators 108A-C to achieve omnidirectional bending with high angular rotations in a range of 0 to 75 degree.