Abstract: A video signal is processed to generate one or more signatures associated with a broadcast program to be recorded by a recording device. The signatures are then processed to determine an actual start time and end time of the desired broadcast program, such that the program can be properly recorded despite delays or other changes in a pre-scheduled broadcast time of the program. One or more of the extracted signatures may be based at least in part on, e.g., a keyframe similarity measure, a histogram, one or more detected commercials, a transcript, a program logo or other detected object, detected text, and a sign-on or sign-off of the desired program. Other types of signatures can also be used.

Abstract: A video indexing method and device for selecting keyframes from each detected scene in the video. The method and device detects fast motion scenes by counting the number of consecutive scene changes detected.

Abstract: A content-based classification system is provided that detects the presence of object images within a frame and determines the path, or trajectory, of each object image through multiple frames of a video segment. In a preferred embodiment, face objects and text objects are used for identifying distinguishing object trajectories. A combination of face, text, and other trajectory information is used in a preferred embodiment of this invention to classify each segment of a video sequence. In one embodiment, a hierarchical information structure is utilized to enhance the classification process. At the upper, video, information layer, the parameters used for the classification process include, for example, the number of object trajectories of each type within the segment, an average duration for each object type trajectory, and so on. At the lowest, model, information layer, the parameters include, for example, the type, color, and size of the object image corresponding to each object trajectory.

Abstract: A method and system which enable a user to query a multimedia archive in one media modality and automatically retrieve correlating data in another media modality without the need for manually associating the data items through a data structure. The correlation method finds the maximum correlation between the data items without being affected by the distribution of the data in the respective subspace of each modality. Once the direction of correlation is disclosed, extracted features can be transferred from one subspace to another.

Abstract: The present invention provides a method of detecting segment boundaries for a series of successive frames in a video sequence. The method includes the steps of acquiring color information from each frame, determining a color histogram for each frame, and applying a boundary detection technique utilizing the color histograms. Finally, the method includes segmenting the frames of the video sequence into uniform color segments. In addition, a system is provided for detecting segment boundaries for a series of successive frames in a video sequence. The system includes means for acquiring color information from each frame, means for determining a color histogram for each frame, and means for applying a boundary detection technique utilizing the color histograms. Finally, the system includes means for segmenting the frames of the video sequence into uniform color segments.

Abstract: A method and apparatus for editing a source video that has already been taken to stabilize images in the video. To eliminate jerky motion from a video, changes in shots are first detected. Then, any jerkiness within the video of that shot is classified and the video is segmented further into smaller segments based on this classification. The jerkiness within the selected segments is removed. The corrected shot, comprising a plurality of frames, is then added to the preceding shot until all shots of the video have been appropriately corrected for jerkiness. To help the user identify the shots being edited, keyframes or snapshots of the shots are displayed, thereby allowing the user to decide whether processing of the shot is desired and which shots should be incorporated into the final video.

Abstract: An image processing device and method for classifying symbols, such as text, in a video stream employs a back propagation neural network (BPNN) whose feature space is derived from size, translation, and rotation invariant shape-dependent features. Various example feature spaces are discussed such as regular and invariant moments and an angle histogram derived from a Delaunay triangulation of a thinned, thresholded, symbol. Such feature spaces provide a good match to BPNN as a classifier because of the poor resolution of characters in video streams.

Abstract: A system that utilizes a determined time between characteristics from content for subsequently identifying the content and/or the corresponding content review position in the time domain. The system utilizes a database of previously stored time between characteristics from known content for matching to the determined time between characteristics from the content source. The characteristics may correspond to video tape indexing data, keyframe data, audio characteristics, text occurrences, and/or other known characteristics from the content. When a match is found between the determined time between characteristics and the stored times between characteristics, the system identifies the content. If corresponding time domain data is stored in the database, the system identifies the current content review position in the time domain.

Abstract: The process of compressing video requires the calculation of a variety data that are used in the process of compression. The invention exploits some or all of these data for purposes of content detection. For example, these data may be leveraged for purposes of commercial detection. The luminance, motion vector field, residual values, quantizer, bit rate, etc. may all be used either directly or in combination, as signatures of content. A process for content detection may employ one or more features as indicators of the start and/or end of a sequence containing a particular type of content and other features as verifiers of the type of content bounded by these start/end indicators. The features may be combined and/or refined to produce higher-level feature data with good computational economy and content-classification utility.

Abstract: In a television receiver having Picture-In-Picture (PIP), a controller analyzes the content of a video signal forming a main picture, and automatically adjusts the size and position of a PIP image to correspond to regions of the main picture exhibiting the least amount of motion, texture, and/or a repeating texture. The controller also prevents the PIP image from being positioned over text or faces or other important objects in the main picture.

Abstract: In a television receiver having Picture-In-Picture (PIP), a controller analyzes the content of an auxiliary video signal forming a PIP image, and automatically adjusts the shape and transparency of the PIP image in accordance with the content of the auxiliary video signal.

Abstract: The present invention provides a method, system and program product for generating a content-based table of contents for a program. Specifically, under the present invention the genre of a program having sequences is determined. Once the genre has been determined, each sequence is assigned a classification. The classifications are assigned based on video content, audio content and textual content within the sequences. Based on the genre and the classifications, keyframe(s) are selected from the sequences for use in a content-based table of contents.

Abstract: A method for operating a neural network, and a program and apparatus that operate in accordance with the method. The method comprises the steps of applying data indicative of predetermined content, derived from an electronic signal including a representation of the predetermined content, to an input of at least one neural network, to cause the at least one network to generate at least one output indicative of either a detection or a non-detection of the predetermined content. Each neural network has an architecture specified by at least one corresponding parameter. The method also comprises a step of evolving the at least one parameter to modify the architecture of the at least one neural network, based on the at least one output, to increase an accuracy at which that at least one neural network detects the predetermined content indicated by the data.

Abstract: A content maintenance system uses a time-dependent precipitation function for iteratively augmenting or removing content over time, after an initial demonstration of user interest. A plurality of parallel precipitation processes can be launched simultaneously in response to different facets of a user expression of interest. Precipitation is dependent on highlighting or extracting segment descriptors from content of interest to the user. Then segments are filtered, rated, annotated and/or prioritized from that content. The remaining segments are matched against stored search structures. When the segments match, they are precipitated out for storage and can generate new search structures.

Abstract: A memory storing computer readable instructions for causing a processor associated with a mega speaker identification (ID) system to instantiate functions including an audio segmentation and classification function receiving general audio data (GAD) and generating segments, a feature extraction function receiving the segments and extracting features based on mel-frequency cepstral coefficients (MFCC) therefrom, a learning and clustering function receiving the extracted features and reclassifying segments, when required, based on the extracted features, a matching and labeling function assigning a speaker ID to speech signals within the GAD, and a database function for correlating the assigned speaker ID to the respective speech signals within the GAD. The audio segmentation and classification function can assign each segment to one of N audio signal classes including silence, single speaker speech, music, environmental noise, multiple speaker's speech, simultaneous speech and music, and speech and noise.

Abstract: A method, process and system for performing content augmentation of personal profiles includes (a) building a user history of a plurality of augmented content information of relevant TV programs; (b) analyzing user queries and determining a degree to which the user queried for additional content information; (c) inferring values about the user from user queries for additional content information so as to augment the additional content information; (d) updating the augmented content information to at least one of the user history, Internet and specialized databases; (e) linking individual ones of the plurality of augmented content information to each other; and (f) determining inferences about the user's interests and preferences based on the linkage of the plurality of augmented content information. The updating of the augmented content information includes segmenting and indexing of multimedia content.

Abstract: A method and apparatus for providing anticipatory content augmentation includes: (a) storing user personal preference requests as items in a Content Preference File (CPF); (b) updating the CPF in accordance with subsequent personal requests that are incremental of the personal preference requests from step (a); (c) categorizing the personal preference requests stored and updated in the CPF into lists, and weight-basing specific personal preference requests on the lists; (d) dynamically linking items categorized in step (c); (e) determining whether steps (a)-(d) have recorded predetermined threshold criteria; and repeating steps (a)-(d) if the threshold has not been reached; (f) identifying the CPF as a master CPF when it has been determined in step (e) that the predetermined threshold has been reached; (g) matching content of a program being viewed to the master CPF; (h) initiating searches in a database/network for augmented information for items identified in the content of the program that are also found i

Abstract: To prevent undesired images from appearing in moving video images, short-duration portions of video information contained within longer video works are detected and reported by, for example, displaying the short-duration portions on a monitor. These short-duration portions are detected, for example, by locating scene cuts, region cuts, or object cuts that occur in rapid succession.

Abstract: A method and apparatus for creating a personalized channel capable of providing a selected commercial to display on a display device includes a viewer reference profile defined with respect to at least one attribute descriptive of a commercial is acquired and used to select a commercial from a plurality of commercials stored in a database. A virtual channel is allocated from available channels in a channel list having a plurality of program channels. The selected commercial is then associated with the virtual channel so as to allow display of the selected channel on the display device when the virtual channel is accessed by a user.

Abstract: An audio recorder-player includes M tuners that generate N audio signals transmitted by N audio sources, an analyzer that extracts R×N audio signal characteristics from the N audio signals, a memory that stores the R×N audio signal characteristics, and output circuitry that reproduces an audio signal corresponding to one of the N audio signals responsive to selection of at least one of the R×N audio signal characteristics, where R is a positive integer and M and N are positive integers greater than 1. If desired, the audio recorder-player advantageously can be included in one of a radio, a computer, or a set-top box. Methods for operating the audio recorder-player are also described.