Abstract: This invention relates to a self-supervised privacy preservation action recognition system leveraging a learnable transformation anonymization function. The system is designed to process videos by removing spatial cues to protect privacy while retaining critical information for action recognition. The anonymization function, based on an encoder-decoder model, undergoes iterative training on a dataset to optimize the balance between obscuring privacy-sensitive information and preserving capability of recognizing actions. This training involves freezing and adjusting the weights of an action recognition branch and a self-supervised privacy removal branch to refine the model's effectiveness. The outcome is an anonymized video with minimized privacy information leakage, suitable for action analysis without privacy labels.

Abstract: This invention relates to a self-supervised privacy preservation action recognition system leveraging a learnable transformation anonymization function. The system is designed to process videos by removing spatial cues to protect privacy while retaining critical information for action recognition. The anonymization function, based on an encoder-decoder model, undergoes iterative training on a dataset to optimize the balance between obscuring privacy-sensitive information and preserving capability of recognizing actions. This training involves freezing and adjusting the weights of an action recognition branch and a self-supervised privacy removal branch to refine the model's effectiveness. The outcome is an anonymized video with minimized privacy information leakage, suitable for action analysis without privacy labels.

Abstract: CNN-based methods for cross-view image geo-localization rely on polar transform and fail to model global correlation. A pure transformer-based approach (TransGeo) is described to address these limitations from a different perspective. TransGeo takes full advantage of the strengths of the transformer related to global information modeling and explicit position information encoding. The claimed invention further leverages transformer input's flexibility and discloses an attention-guided non-uniform cropping method so that uninformative image patches are removed with a negligible drop in performance to reduce computation cost. The saved computation can be reallocated to increase resolution only for informative patches, resulting in performance improvement with no additional computation cost. This “attend and zoom-in” strategy is highly similar to human behavior when observing images.

Abstract: Methods of detecting and categorizing an action in an untrimmed video segment regardless of the scale of the action and the close proximity of other actions. The methods improve upon the prior art which either require trimmed video segments including only a single activity depicted therein, or untrimmed video segments including relatively few actions, persons, or objects of interest, thereby directing the classification. Instead, the methods utilize a plurality of tubelets used to represent discreet actions, persons, and objects of interest within the comprehensive untrimmed video segment. The tubelets are localized to correct for pixel-level foreground-background biases, which are then turned into short spatio-temporal action tubelets that are passed to a classification network to obtain multi-label predictions. After classification, the tubelets are be linked together to obtain the final detections with varying lengths, and the method merges the short action tubelets into final action detections.

Abstract: Methods of detecting and categorizing an action in an untrimmed video segment regardless of the scale of the action and the close proximity of other actions. The methods improve upon the prior art which either require trimmed video segments including only a single activity depicted therein, or untrimmed video segments including relatively few actions, persons, or objects of interest, thereby directing the classification. Instead, the methods utilize a plurality of tubelets used to represent discreet actions, persons, and objects of interest within the comprehensive untrimmed video segment. The tubelets are localized to correct for pixel-level foreground-background biases, which are then turned into short spatio-temporal action tubelets that are passed to a classification network to obtain multi-label predictions. After classification, the tubelets are be linked together to obtain the final detections with varying lengths, and the method merges the short action tubelets into final action detections.

Abstract: A system and method for estimating a sequence of human poses in an unconstrained video. In the present invention, a unified two stage, tree-based, optimization problem is solved for which an efficient and exact solution exists. While the proposed method finds an exact solution, it does not sacrifice the ability to model the spatial and temporal constraints between body parts in the video frames on the unconstrained video.

Abstract: A method for counting individuals in an image containing a dense, uniform or non-uniform crowd. The current invention leverages multiple sources of information to compute an estimate of the number of individuals present in a dense crowd visible in a single image. This approach relies on multiple sources, such as low confidence head detections, repetition of texture elements (using SIFT), and frequency-domain analysis to estimate counts, along with confidence associated with observing individuals in an image region. Additionally, a global consistency constraint can be employed on counts using Markov Random Field. This caters for disparity in counts in local neighborhoods and across scales. The methodology was tested on a new dataset of fifty (50) crowd images containing over 64,000 annotated humans, with the head counts ranging from 94 to 4,543. Efficient and accurate results were attained.

Abstract: A method for counting individuals in an image containing a dense, uniform or non-uniform crowd. The current invention leverages multiple sources of information to compute an estimate of the number of individuals present in a dense crowd visible in a single image. This approach relies on multiple sources, such as low confidence head detections, repetition of texture elements (using SIFT), and frequency-domain analysis to estimate counts, along with confidence associated with observing individuals in an image region. Additionally, a global consistency constraint can be employed on counts using Markov Random Field. This caters for disparity in counts in local neighborhoods and across scales. The methodology was tested on a new dataset of fifty (50) crowd images containing over 64,000 annotated humans, with the head counts ranging from 94 to 4,543. Efficient and accurate results were attained.

Abstract: A system and method for estimating a sequence of human poses in an unconstrained video. In the present invention, a unified two stage, tree-based, optimization problem is solved for which an efficient and exact solution exists. While the proposed method finds an exact solution, it does not sacrifice the ability to model the spatial and temporal constraints between body parts in the video frames on the unconstrained video.

Abstract: The present invention is a complete end-to-end video face recognition system. The invention performs a track-by-track labeling of all tracks within a video. A novel algorithm Mean Sequence SRC (MSSRC) is applied to a face track to perform video face recognition using a joint optimization to leverage all of the available video data and the knowledge that the face track frames belong to the same individual. Additionally the system constructs a probabilistic affinity graph combining appearance and co-occurrence similarities to model the relationship between face tracks in a video. Finally, using this relationship graph, random walk analysis is employed to propagate strong class predictions among similar face tracks, while dampening weak predictions.

Abstract: Systems and methods for processing image data are provided. A computer implemented method for processing image data, comprises gathering 4-D image data from a subject, extracting time series data, and spatial and degree data of each voxel of the subject, deriving at least one feature from the time series data, deriving at least one feature from the spatial and degree data, combining the at least one feature from the time series data, and the at least one feature from the spatial and degree data to generate combined data, and inputting the combined data to a classifier, wherein the classifier outputs a classification based on the combined data.

Abstract: In one embodiment, a system and method for modeling a three-dimensional object includes capturing two-dimensional images of the object from multiple different viewpoints to obtain multiple views of the object, estimating slices of the object that lie in parallel planes that cut through the object, and computing a surface of the object from the estimated slices.

Abstract: In one embodiment, a system and method for three-dimensionally modeling a moving object pertain to capturing sequential images of the moving object from multiple different viewpoints to obtain multiple views of the moving object, identifying silhouettes of the moving object in each view, determining the location in each view of a temporal occupancy point for each silhouette boundary pixel, each temporal occupancy point being the estimated localization of a three-dimensional scene point that gave rise to its associated silhouette boundary pixel, generating blurred occupancy images that comprise silhouettes of the moving object composed of the temporal occupancy points, deblurring the blurred occupancy images to generate deblurred occupancy maps of the moving object, and reconstructing the moving object by performing visual hull intersection using the blurred occupancy maps to generate a three-dimensional model of the moving object.

Abstract: In one embodiment, a system and method for modeling a three-dimensional object includes capturing two-dimensional images of the object from multiple different viewpoints to obtain multiple views of the object, estimating slices of the object that lie in parallel planes that cut through the object, and computing a surface of the object from the estimated slices.

Abstract: Tracking and surveillance methods and systems for monitoring objects passing in front of non-overlapping cameras. Invention finds corresponding tracks from different cameras and works out which object passing in front of the camera(s) made the tracks, in order to track the object from camera to camera. The invention uses an algorithm to learn inter-camera spatial temporal probability using Parzen windows, learns inter-camera appearance probabilities using distribution of Bhattacharyya distances between appearance models, establishes correspondences based on Maximum A Posteriori (MAP) framework combining both spatial temporal and appearance probabilities, and updates learned probabilities throughout the lifetime of the system.

Abstract: Visual methods and systems are described for detecting alertness and vigilance of persons under conditions of fatigue, lack of sleep, and exposure to mind altering substances such as alcohol and drugs. In particular, the intention can have particular applications for truck drivers, bus drivers, train operators, pilots and watercraft controllers and stationary heavy equipment operators, and students and employees during either daytime or nighttime conditions. The invention robustly tracks a person's head and facial features with a single on-board camera with a fully automatic system, that can initialize automatically, and can reinitialize when it need's to and provide outputs in realtime. The system can classify rotation in all viewing direction, detects' eye/mouth occlusion, detects' eye blinking, and recovers the 3D(three dimensional) gaze of the eyes. In addition, the system is able to track both through occlusion like eye blinking and also through occlusion like rotation.