Abstract: A method detects short term, unusual events in a video. First, features are extracted features from the audio and the video portions of the video. Segments of the video are labeled according to the features. A global sliding window is applied to the labeled segments to determine global characteristics over time, while a local sliding window is applied only to the labeled segments of the global sliding window to determine local characteristic over time. The local window is substantially shorter in time than the global window. A distance between the global and local characteristic is measured to determine occurrences of the unusual short time events.

Abstract: A method segments a video into objects, without user assistance. An MPEG compressed video is converted to a structure called a pseudo spatial/temporal data using DCT coefficients and motion vectors. The compressed video is first parsed and the pseudo spatial/temporal data are formed. Seeds macro-blocks are identified using, e.g., the DCT coefficients and changes in the motion vector of macro-blocks.

Abstract: A method detects an unusual event in a video. Motion vectors are extracted from each frame in a video acquired by a camera of a scene. Zero run-length parameters are determined for each frame from the motion vectors. The zero run-length parameters are summed over predetermined time intervals of the video, and a distance is determined between the sum of the zero run-lengths of a current time interval and the sum of the zero run-lengths of a previous time interval. Then, the unusual event is detected if the distance is greater than a predetermined threshold.

Abstract: A system and method analyzes a compressed video including a sequence of frames. The amount of a dominant feature in each frame of the compressed video is measured. A label is associated with each frame according the measured amount of the dominant feature. Views in the video are identified according to the labels, and the video is segmented into actions according to the views. The video can then be analyzed according to the action to determine significant events in the video.

Abstract: A method extracts highlights from an audio signal of a sporting event. The audio signal can be part of a sports videos. First, sets of features are extracted from the audio signal. The sets of features are classified according to the following classes: applause, cheering, ball hit, music, speech and speech with music. Adjacent sets of identically classified features are grouped. Portions of the audio signal corresponding to groups of features classified as applause or cheering and with a duration greater than a predetermined threshold are selected as highlights.

Abstract: A compressed bit-stream represents a corresponding sequence having intra-coded frames and inter-coded frames. The compressed bit-stream includes bits associated with each of the inter-coded frames representing a displacement from the associated inter-coded frame to a closest matching of the intra-coded frames. A magnitude of the displacement of a first of the inter-coded frames is determined based on the bits in the compressed bit-stream associated with that inter-coded frame. The inter-coded frame is then identified based on the determined displacement magnitude. The inter-coded frame includes macro-blocks. Each macro-block is associated with a respective portion of the inter-coded frame bits which represent the displacement from that macro-block to the closest matching intra-coded frame. The displacement magnitude is an average of the displacement magnitudes of all the macro-blocks associated with the inter-coded frame.

Abstract: A method segments and summarizes a news video using both audio and visual features extracted from the video. The summaries can be used to quickly browse the video to locate topics of interest. A generalized sound recognition hidden Markov model (HMM) framework for joint segmentation and classification of the audio signal of the news video is used. The HMM not only provides a classification label for audio segment, but also compact state duration histogram descriptors.

Abstract: A method extracts high-level features from a video including a sequence of frames. Low-level features are extracted from each frame of the video. Each frame of the video is labeled according to the extracted low-level features to generate sequences of labels. Each sequence of labels is associated with one of the extracted low-level feature. The sequences of labels are analyzed using learning machine learning techniques to extract high-level features of the video.

Abstract: A method discovers patterns in unknown content of a video. The video is partitioned into sets of disjoint segments. Each set includes all frames of the video, and each set is partitioned according to a selected low-level feature of the video. The disjoint segments are grouped into corresponding sets of clusters, each cluster including similar segments. The clusters are then labeled, and association rules are identified among the labels to discover high-level patterns in the unknown content of the video.

Abstract: A method mines unknown content of a video by first selecting one or more low-level features of the video. For each selected feature, or combination of features, time series data is generated. The time series data is then self-correlated to identify similar segments of the video according to the low-level features. The similar segments are grouped into clusters to discover high-level patterns in the unknown content of video.

Abstract: A method summarizes unknown content of a video. First, low-level features of the video are selected. The video is then partitioned into segments according to the low-level features. The segments are grouped into disjoint clusters where each cluster contains similar segments. The clusters are labeled according to the low-level features, and parameters characterizing the clusters are assigned. High-level patterns among the labels are found, and the these patterns are used to extract frames from the video according to form a content-adaptive summary of the unknown content of the video.

Abstract: A method extracts an intensity of motion activity from shots in a compressed video. The method then uses the intensity of motion activity to segment the video into easy and difficult segments to summarize. Easy to summarize segments are represented by any frames selected from the easy to summarize segments, while a color based summarization process extracts generates sequences of frames from each difficult to summarize segment. The selected and generated frames of each segment in each shot are combined to form the summary of the compressed video.

Abstract: A method segments a compressed video by extracting audio and visual features from the compressed video. The audio features are clustered according to K-means clustering in a set of classes, and the compressed video is then partitioned into first segments according to the set of classes. The visual features are then used to partitioning each first segment into second segments using motion analysis. Summaries of the second segments can be provided to assist in the browsing of the compressed video.

Abstract: A surveillance and control system includes a feature extraction unit to dynamically extract low-level features from a compressed digital video signal, a description encoder, coupled to the feature extraction unit, to encode the low-level features as content descriptors. An event detector is coupled to the description encoder to detect security events from the content descriptors, and a control signal processor, coupled to the event detector, to generate control signals in response to detecting the security events.

Abstract: This invention relates to methods of feature extraction from MPEG-2 and MPEG-4 compressed video sequences. The spatio-temporal compression complexity of video sequences is evaluated for feature extraction by inspecting the compressed bitstream and the complexity is used as a descriptor of the spatio-temporal characteristics of the video sequence. The spatio-temporal compression complexity measure is used as a matching criterion and can also be used for absolute indexing. Feature extraction can be accomplished in conjunction with scene change detection techniques and the combination has reasonable accuracy and the advantage of high simplicity since it is based on entropy decoding of signals in compressed form and does not require computationally expensive inverse Discrete Cosine Transformation (DCT).

Abstract: A method describes motion activity in a video sequence. A motion activity matrix is determined for the video sequence. A threshold for the motion activity matrix is determined. Connected regions of motion vectors at least equal to the threshold are identified and measured for size. A histogram of the distribution of the sizes of the connected areas is constructed for the entire video sequence. The histogram is normalized to characterize the spatial distribution of the video sequence in a motion activity descriptor.

Abstract: A method for summarizing a video first detects audio peaks in a sub-sampled audio signal of the video. Then, motion activity in the video is extracted and filtered. The filtered motion activity is quantized to a continuous stream of digital pulses, one pulse for each frame. If the motion activity is greater than a predetermined threshold the pulse is one, otherwise the pulse is zero. Each quantized pulse is tested with respect to the timing of rising and falling edges. If the pulse meets the condition of the test, then the pulse is selected as a candidate pulse related to an interesting event in the video, otherwise the pulse is discarded. The candidate pulses are correlated, time-wise to the audio peaks, and patterns between the pulses and peaks are examined. The correlation patterns segment the video into uninteresting and interesting portions, which can then be summarized.

Abstract: A method identifies a talking head or principal cast in a compressed video by first segmenting the video into shots. Motion activity descriptors are extracted from each of the shots, and combined into a shot motion activity descriptor. A distance between the shot motion activity descriptor and a template motion activity descriptor is measured. The template motion activity descriptor is obtained from a training video. If the measured distance is less than a predetermined threshold, then the shot is identified as including a talking head.

Abstract: A method analyzes a high-level syntax and structure of a continuous compressed video according to a plurality of states. First, a set of hidden Markov models for each of the states is trained with a training video segmented into known states. Then, a set of domain specific features are extracted from a fixed-length sliding window of the continuous compressed video, and a set of maximum likelihoods is determined for each set of domain specific features using the sets of trained hidden Markov models. Finally, dynamic programming is applied to each set of maximum likelihoods to determine a specific state for each fixed-length sliding window of frames of the compressed video.

Abstract: A method for transcoding a compressed video partitions the compressed video into hierarchical levels, and extracts features from each of the hierarchical levels. One of a number of conversion modes of a transcoder is selected dependent on the features extracted from the hierarchical levels. The compressed video is then transcoded according to the selected conversion mode.