Abstract: Iterative prediction systems and methods for the task of action detection process an inputted sequence of video frames to generate an output of both action tubes and respective action labels, wherein the action tubes comprise a sequence of bounding boxes on each video frame. An iterative predictor processes large offsets between the bounding boxes and the ground-truth.

Abstract: Iterative prediction systems and methods for the task of action detection process an inputted sequence of video frames to generate an output of both action tubes and respective action labels, wherein the action tubes comprise a sequence of bounding boxes on each video frame. An iterative predictor processes large offsets between the bounding boxes and the ground-truth.

Abstract: Multi-modal data representing driving events and corresponding actions related to the driving events can be obtained and used to train a neural network at least in part by using a triplet loss computed for the driving events as a regression loss to determine an embedding of driving event data. In some cases, using the trained neural network, a retrieval request for an input driving event and corresponding action can be processed by determining, from the neural network, one or more similar driving events or corresponding actions in the multi-modal data.

Abstract: There are numerous features in video that can be detected using computer-based systems, such as objects and/or motion. The detection of these features, and in particular the detection of motion, has many useful applications, such as action recognition, activity detection, object tracking, etc. The present disclosure provides a neural network that learns motion from unlabeled video frames. In particular, the neural network uses the unlabeled video frames to perform self-supervised hierarchical motion learning. The present disclosure also describes how the learned motion can be used in video action recognition.

Abstract: Multi-modal data representing driving events and corresponding actions related to the driving events can be obtained and used to train a neural network at least in part by using a triplet loss computed for the driving events as a regression loss to determine an embedding of driving event data. In some cases, using the trained neural network, a retrieval request for an input driving event and corresponding action can be processed by determining, from the neural network, one or more similar driving events or corresponding actions in the multi-modal data.

Abstract: Iterative prediction systems and methods for the task of action detection process an inputted sequence of video frames to generate an output of both action tubes and respective action labels, wherein the action tubes comprise a sequence of bounding boxes on each video frame. An iterative predictor processes large offsets between the bounding boxes and the ground-truth.

Abstract: A multimodal sensing system includes various devices that work together to automatically classify an action. A video camera captures a sequence of digital images. At least one other sensor device captures other sensed data (e.g., motion data). The system will extract video features from the digital images so that each extracted image feature is associated with a time period. It will extract other features from the other sensed data so that each extracted other feature is associated with a time period. The system will fuse a group of the extracted video features and a group of the extracted other features to create a fused feature representation for a time period. It will then analyze the fused feature representation to identify a class, access a data store of classes and actions to identify an action that is associated with the class, and save the identified action to a memory device.

Abstract: A multimodal sensing system includes various devices that work together to automatically classify an action. A video camera captures a sequence of digital images. At least one other sensor device captures other sensed data (e.g., motion data). The system will extract video features from the digital images so that each extracted image feature is associated with a time period. It will extract other features from the other sensed data so that each extracted other feature is associated with a time period. The system will fuse a group of the extracted video features and a group of the extracted other features to create a fused feature representation for a time period. It will then analyze the fused feature representation to identify a class, access a data store of classes and actions to identify an action that is associated with the class, and save the identified action to a memory device.