Abstract: A system and method summarizes multimedia stored in a compressed multimedia file partitioned into a sequence of segments, where the content of the multimedia is, for example, video signals, audio signals, text, and binary data. An associated metadata file includes index information and an importance level for each segment. The importance information is continuous over as closed interval. An importance level threshold is selected in the closed interval, and only segments of the multimedia having a particular importance level greater than the importance level threshold are reproduced. The importance level can also be determined for fixed-length windows of multiple segments, or a sliding window. Furthermore, the importance level can be weighted by a factor, such as the audio volume.

Abstract: A system and method summarizes multimedia stored in a compressed multimedia file partitioned into a sequence of segments, where the content of the multimedia is, for example, video signals, audio signals, text, and binary data. An associated metadata file includes index information and an importance level for each segment. The importance information is continuous over as closed interval. An importance level threshold is selected in the closed interval, and only segments of the multimedia having a particular importance level greater than the importance level threshold are reproduced.

Abstract: A method learns a structure of a video, in an unsupervised setting, to detect events in the video consistent with the structure. Sets of features are selected from the video. Based on the selected features, a hierarchical statistical model is updated, and an information gain of the hierarchical statistical model is evaluated. Redundant features are then filtered, and the hierarchical statistical model is updated, based on the filtered features. A Bayesian information criteria is applied to each model and feature set pair, which can then be rank ordered according to the criteria to detect the events in the video.

Abstract: A method uses probabilistic fusion to detect highlights in videos using both audio and visual information. Specifically, the method uses coupled hidden Markov models (CHMMs). Audio labels are generated using audio classification via Gaussian mixture models (GMMs), and visual labels are generated by quantizing average motion vector magnitudes. Highlights are modeled using discrete-observation CHMMs trained with labeled videos. The CHMMs have better performance than conventional hidden Markov models (HMMs) trained only on audio signals, or only on video frames.

Abstract: A method analyzes a high-level syntax and structure of a continuous compressed video according to a plurality of states. First, a set of hidden Markov models for each of the states is trained with a training video segmented into known states. Then, a set of domain specific features are extracted from a fixed-length sliding window of the continuous compressed video, and a set of maximum likelihoods is determined for each set of domain specific features using the sets of trained hidden Markov models. Finally, dynamic programming is applied to each set of maximum likelihoods to determine a specific state for each fixed-length sliding window of frames of the compressed video.

Abstract: A method plays frames of a video adaptively according to a visual complexity of the video. First a spatial frequency of pixel within frames of the video is measured, as well as a temporal velocity of corresponding pixels between frames of the video. The spatial frequency is multiplied by the temporal velocity to obtain a measure of the visual complexity of the frames of the video. The frames of the video are then played at a speed that corresponds to the visual complexity.

Abstract: A method identifies a talking head or principal cast in a compressed video by first segmenting the video into shots. Motion activity descriptors are extracted from each of the shots, and combined into a shot motion activity descriptor. A distance between the shot motion activity descriptor and a template motion activity descriptor is measured. The template motion activity descriptor is obtained from a training video. If the measured distance is less than a predetermined threshold, then the shot is identified as including a talking head.

Abstract: A method detects short term, unusual events in a video. First, features are extracted features from the audio and the video portions of the video. Segments of the video are labeled according to the features. A global sliding window is applied to the labeled segments to determine global characteristics over time, while a local sliding window is applied only to the labeled segments of the global sliding window to determine local characteristic over time. The local window is substantially shorter in time than the global window. A distance between the global and local characteristic is measured to determine occurrences of the unusual short time events.

Abstract: A method segments a video into objects, without user assistance. An MPEG compressed video is converted to a structure called a pseudo spatial/temporal data using DCT coefficients and motion vectors. The compressed video is first parsed and the pseudo spatial/temporal data are formed. Seeds macro-blocks are identified using, e.g., the DCT coefficients and changes in the motion vector of macro-blocks.

Abstract: A method detects an unusual event in a video. Motion vectors are extracted from each frame in a video acquired by a camera of a scene. Zero run-length parameters are determined for each frame from the motion vectors. The zero run-length parameters are summed over predetermined time intervals of the video, and a distance is determined between the sum of the zero run-lengths of a current time interval and the sum of the zero run-lengths of a previous time interval. Then, the unusual event is detected if the distance is greater than a predetermined threshold.

Abstract: A system and method analyzes a compressed video including a sequence of frames. The amount of a dominant feature in each frame of the compressed video is measured. A label is associated with each frame according the measured amount of the dominant feature. Views in the video are identified according to the labels, and the video is segmented into actions according to the views. The video can then be analyzed according to the action to determine significant events in the video.

Abstract: A method extracts highlights from an audio signal of a sporting event. The audio signal can be part of a sports videos. First, sets of features are extracted from the audio signal. The sets of features are classified according to the following classes: applause, cheering, ball hit, music, speech and speech with music. Adjacent sets of identically classified features are grouped. Portions of the audio signal corresponding to groups of features classified as applause or cheering and with a duration greater than a predetermined threshold are selected as highlights.

Abstract: A compressed bit-stream represents a corresponding sequence having intra-coded frames and inter-coded frames. The compressed bit-stream includes bits associated with each of the inter-coded frames representing a displacement from the associated inter-coded frame to a closest matching of the intra-coded frames. A magnitude of the displacement of a first of the inter-coded frames is determined based on the bits in the compressed bit-stream associated with that inter-coded frame. The inter-coded frame is then identified based on the determined displacement magnitude. The inter-coded frame includes macro-blocks. Each macro-block is associated with a respective portion of the inter-coded frame bits which represent the displacement from that macro-block to the closest matching intra-coded frame. The displacement magnitude is an average of the displacement magnitudes of all the macro-blocks associated with the inter-coded frame.

Abstract: A method segments and summarizes a news video using both audio and visual features extracted from the video. The summaries can be used to quickly browse the video to locate topics of interest. A generalized sound recognition hidden Markov model (HMM) framework for joint segmentation and classification of the audio signal of the news video is used. The HMM not only provides a classification label for audio segment, but also compact state duration histogram descriptors.

Abstract: A method extracts high-level features from a video including a sequence of frames. Low-level features are extracted from each frame of the video. Each frame of the video is labeled according to the extracted low-level features to generate sequences of labels. Each sequence of labels is associated with one of the extracted low-level feature. The sequences of labels are analyzed using learning machine learning techniques to extract high-level features of the video.

Abstract: A method discovers patterns in unknown content of a video. The video is partitioned into sets of disjoint segments. Each set includes all frames of the video, and each set is partitioned according to a selected low-level feature of the video. The disjoint segments are grouped into corresponding sets of clusters, each cluster including similar segments. The clusters are then labeled, and association rules are identified among the labels to discover high-level patterns in the unknown content of the video.

Abstract: A method summarizes unknown content of a video. First, low-level features of the video are selected. The video is then partitioned into segments according to the low-level features. The segments are grouped into disjoint clusters where each cluster contains similar segments. The clusters are labeled according to the low-level features, and parameters characterizing the clusters are assigned. High-level patterns among the labels are found, and the these patterns are used to extract frames from the video according to form a content-adaptive summary of the unknown content of the video.

Abstract: A method mines unknown content of a video by first selecting one or more low-level features of the video. For each selected feature, or combination of features, time series data is generated. The time series data is then self-correlated to identify similar segments of the video according to the low-level features. The similar segments are grouped into clusters to discover high-level patterns in the unknown content of video.

Abstract: A method extracts an intensity of motion activity from shots in a compressed video. The method then uses the intensity of motion activity to segment the video into easy and difficult segments to summarize. Easy to summarize segments are represented by any frames selected from the easy to summarize segments, while a color based summarization process extracts generates sequences of frames from each difficult to summarize segment. The selected and generated frames of each segment in each shot are combined to form the summary of the compressed video.

Abstract: A method segments a compressed video by extracting audio and visual features from the compressed video. The audio features are clustered according to K-means clustering in a set of classes, and the compressed video is then partitioned into first segments according to the set of classes. The visual features are then used to partitioning each first segment into second segments using motion analysis. Summaries of the second segments can be provided to assist in the browsing of the compressed video.