Abstract: Systems and methods are described to analyze visual data and trajectory data from sports and games, to determine the content of the input data and detect actions and events, quantitatively measure their impact on the future activities and events and make predictions. In real-time, at every moment of the game, the probability of the next action/event in the game is estimated; the impact of the current state of the game on the future outcomes are predicted; and the current player action/match event is labelled. More specifically, given a predefined objective, a quantitative value is assigned to the input data based on the current observation characterizing the probability of reaching an objective, or incremental changes to that probability based on the observed state of the game from the video or trajectory data.

Abstract: A system and method are described, for automatically analyzing and understanding individual and group activities and interactions. The method includes receiving at least one image from a video of a scene showing one or more individual objects or humans at a given time; applying at least one machine learning or artificial intelligence technique to automatically learn a spatial, temporal or a spatio-temporal informative representation of the image and video content for activity recognition; and identifying and analyzing individual and group activities in the scene.

Abstract: A method is described, for processing images of persons or objects to generate an identity preservative feature descriptor learnt for each person or object. The method includes obtaining an image of a person or object, extracting at least one spatial attribute of the person or object from the obtained image, and extracting at least one appearance feature of the person or object from the image by using a mapping function to translate image pixels into appearance attributes represented by at least one numerical feature. The method also includes combining the at least one spatial attribute and the at least one appearance feature to generate the unique feature descriptor representing the person or the object, to assign the unique feature descriptor to the image to enable feature descriptors representing the same person or object to be compared to feature descriptors representing different people or objects given a predefined mathematical pseudo-distance metric according to a least a distance from each other.

Abstract: A system and method are provided for registering input images to reference objects to generate output parameters. The method includes obtaining an input image and a reference object, wherein the reference object comprises a reference image or a template corresponding to a model of content in the reference image. The method also includes performing a registration of the input image by estimating a geometric transform and/or a set of non-linear transform parameters, without using predefined image features. The method also includes measuring a misalignment error between the input images and the reference object and generating a set of output parameters. The output parameters can include a set of one or more camera parameters or a set of geometric and non-linear transformation parameters.

Abstract: There are provided methods and systems to generate a summary of a video by decomposing the video into segments automatically, where each segment has a quantitative score. The assigned scores to those segments can be generated using models that are quantitatively describing and/or evaluating the individual or group activities of the objects in the scene. The segments can be grouped based on their scores to generate a video summary. In an implementation, such a system can generate video summaries of a game based on the quantitative game models. Using a game model that assigns values to different game events and actions in a game, a set of most interesting, least interesting and neutral plays can be identified in the video and a highlight or lowlight reel generated. With adjusting the valuation of player actions and game events based on their impact on the game's result, the monotony in highlight reels can be avoided.

Abstract: A system and method for calibrating and re-calibrating an imaging device. The image device includes at least one varying intrinsic or extrinsic parameter value based on a view of 3D shapes and a 2D template therefor. The method and system are operable to receive a sequence of images from the imaging device, wherein a planar surface with known geometry is observable from the sequence of images; perform an initial estimation of a set of camera parameters for a given image frame; perform a parameter adjustment operation to minimize a dissimilarity between the template and the sequence of images; perform a parameter adjustment operation to maximize similarities in either the camera's coordinate system or the template's coordinate system; and update an adaptive world template to incorporate an additional stable point from the observed images to the template.

Abstract: A system and method are provided for creating and evaluating computational models for games (e.g., individual or team sports games, etc.), team performance, and individual player performance evaluation. The method comprises obtaining data associated with the team game, the information comprising at least one individual player activity, at least one team activity, at least one game event, and a location in space and time for each of the events and activities; generating quantitative values for the data associated with the team game; and evaluating either or both an individual player and a team using the quantitative values.

Abstract: A system and method are provided for registering input images to reference objects to generate output parameters. The method includes obtaining an input image and a reference object, wherein the reference object comprises a reference image or a template corresponding to a model of content in the reference image. The method also includes performing a registration of the input image by estimating a geometric transform and/or a set of non-linear transform parameters, without using predefined image features. The method also includes measuring a misalignment error between the input images and the reference object and generating a set of output parameters. The output parameters can include a set of one or more camera parameters or a set of geometric and non-linear transformation parameters.

Abstract: A system and method are provided for analyzing a video. The method comprises: sampling the video to generate a plurality of spatio-temporal video volumes; clustering similar ones of the plurality of spatio-temporal video volumes to generate a low-level codebook of video volumes; analyzing the low-level codebook of video volumes to generate a plurality of ensembles of volumes surrounding pixels in the video; and clustering the plurality of ensembles of volumes by determining similarities between the ensembles of volumes, to generate at least one high-level codebook. Multiple high-level codebooks can be generated by repeating steps of the method. The method can further include performing visual event retrieval by using the at least one high-level codebook to make an inference from the video, for example comparing the video to a dataset and retrieving at least one similar video, activity and event labeling, and performing abnormal and normal event detection.

Abstract: A system and method are provided for creating and evaluating computational models for games (e.g., individual or team sports games, etc.), team performance, and individual player performance evaluation. The method comprises obtaining data associated with the team game, the information comprising at least one individual player activity, at least one team activity, at least one game event, and a location in space and time for each of the events and activities; generating quantitative values for the data associated with the team game; and evaluating either or both an individual player and a team using the quantitative values.

Abstract: A system and method are provided for calibrating and re-calibrating an imaging device. The image device comprises at least one varying intrinsic or extrinsic parameter value based on a view of 3D shapes and a 2D template therefor. The method and system are operable to receive a sequence of images from the imaging device, wherein a planar surface with known geometry is observable from the sequence of images; perform an initial estimation of a set of camera parameters for a given image frame; perform a parameter adjustment operation to minimize a dissimilarity between the template and the sequence of images; perform a parameter adjustment operation to maximize similarities in either the camera's coordinate system or the template's coordinate system; and update an adaptive world template to incorporate an additional stable point from the observed images to the template.

Abstract: A system and method are provided for tracking objects in a scene from a sequence of images captured by an imaging device. The method includes processing the sequence of images to generate sequential images at a plurality of hierarchical levels to generate a set of regions of interest; and, at each of the hierarchical levels: examining pairs of sequential images to link pixels into short tracklets; and grouping short tracklets that indicate similar motion patterns to generate representative tracklets. The representative tracklets are grouped to generate a tracking result for at least one object.

Abstract: A system and method are provided for analyzing a video. The method comprises: sampling the video to generate a plurality of spatio-temporal video volumes; clustering similar ones of the plurality of spatio-temporal video volumes to generate a low-level codebook of video volumes; analyzing the low-level codebook of video volumes to generate a plurality of ensembles of volumes surrounding pixels in the video; and clustering the plurality of ensembles of volumes by determining similarities between the ensembles of volumes, to generate at least one high-level codebook. Multiple high-level codebooks can be generated by repeating steps of the method. The method can further include performing visual event retrieval by using the at least one high-level codebook to make an inference from the video, for example comparing the video to a dataset and retrieving at least one similar video, activity and event labeling, and performing abnormal and normal event detection.

Abstract: A system and method are provided for tracking objects in a scene from a sequence of images captured by an imaging device. The method includes processing the sequence of images to generate sequential images at a plurality of hierarchical levels to generate a set of regions of interest; and, at each of the hierarchical levels: examining pairs of sequential images to link pixels into short tracklets; and grouping short tracklets that indicate similar motion patterns to generate representative tracklets. The representative tracklets are grouped to generate a tracking result for at least one object.