Abstract: An image processing device and method for classifying symbols relies on a connected-component technique for isolating symbol regions. The device and method form connected components from an image derived by the application of an edge detection filter. The formation of connected components from this filtered image defines the edges of character shapes. As a result, the number of pixels that must be connected to define each connected component is substantially reduced and the speed of processing thereby increased. The application of the method is discussed primarily with respect to text in a video stream.

Abstract: There is disclosed, for use in video text analysis system, a video processing device for searching video streams for one or more user-selected image text attributes. The video processing device comprises an image processor capable detecting and extracting image text from video frames, determining attributes of the extracted image text, comparing the extracted image text attributes and the user-selected image text attributes, and, if a match occurs, modifying, transferring, and/or labeling at least a portion of the video stream in accordance with user commands.

Abstract: A method and system are disclosed for determining who is the speaking person in video data. This may be used to add in person identification in video content analysis and retrieval applications. A correlation is used to improve the person recognition rate relying on both face recognition and speaker identification. Latent Semantic Association (LSA) process may also be used to improve the association of a speaker's face with his voice. Other sources of data (e.g., text) may be integrated for a broader domain of video content understanding applications.

Abstract: Stories are detected in multimedia data composed of concurrent streams for different modes such as audio, video and text and linked to related stories. First, time periods of uniformity in attributes of the streams serve as “building blocks” that are consolidated according to rules that are characteristic of the story to be detected. The attributes are then ranked by their respective reliabilities for detecting the story to be detected. An inter-attribute union of the time periods is cumulated attribute-to-attribute in an order that is based on the ranking. A buffered portion of the multimedia data that is delimited by the starting and ending times is retained in mass storage. The starting and ending times are indexed by characteristics of content of the portion to form a story segment which is maintained in a data structure with links to related story segments.

Abstract: A method for optimizing the performance of an algorithm for detecting predetermined content in a media information stream, and a program and apparatus that operate in accordance with the method. The algorithm is a function of a set of parameters. The method comprises the steps of performing the algorithm at least once to detect the predetermined content in the media information stream, while employing a respective set of parameters in the algorithm for each performance thereof, and automatically evolving at least one respective set of parameters employed in the algorithm to maximize the degree of accuracy at which the algorithm detects the predetermined content in the media information stream.

Abstract: A processing system is provided that detects an object of interest. The system receives an input image of the object. At least one feature is extracted from the input image. The extracted feature is then used to determine a set of candidate models by filtering out image models that do not contain the extracted feature. A sample image template is then formed based on a candidate sample image. The object of interest is then detected by comparing the input image to the sample image template. In a preferred embodiment, the formation of the template further includes calculating a parameter, such as direction, expression, articulation or lighting, of the object.

Abstract: A method for providing name-face/voice-role association includes determining whether a closed captioned text accompanies a video sequence, providing one of text recognition and speech to text conversion to the video sequence to generate a role-name versus actor-name list from the video sequence, extracting face boxes from the video sequence and generating face models, searching a predetermined portion of text for an entry on the role-name versus actor-name list, searching video frames for face models/voice models that correspond to the text searched by using a time code so that the video frames correspond to portions of the text where role-names are detected, assigning an equal level of certainty for each of the face models found, using lip reading to eliminate face models found that pronounce a role-name corresponding to said entry on the role-name versus actor-name list, scanning a remaining portion of text provided and updating a level of certainty for said each of the face models previously found.

Abstract: An interactive imaging system and method thereof are provided. The invention provides a real-time interactive video system. The input signals to the system are given through an array of cameras, sensors and microphones. The input signals are generated by human (or pet) presence and involvement. A set of software modules is invoked based on a “rule” framework created by the user, e.g., artist or designer, of the interactive system. The “rules” define which set of input signals are connected to certain portions, i.e., impressible regions, of an image on a display of the system or connected to certain portions on a mosaiced display. The inventive system allows the user to build a set of “rules” as to how the impressible regions of the displayed images can change based on motion, color, and/or texture of a “visitor” interacting with the system.

Abstract: An adaptive memory system remembers facts, e.g. names the user has asked for, and the depth of information a user has requested on a certain topic. The adaptive memory builds a data structure of the user's knowledge base and behavior. The system applies this knowledge when retrieving segments of interests, augmenting video programs, recommending new programs. Also the system can use this knowledge and select items for a personal TV channel.

Abstract: Techniques are disclosed for detecting commercials or other particular types of video content in a video signal. In an illustrative embodiment, color histograms are extracted from frames of the video signal. For each of at least a subset of the extracted color histograms, the extracted color histogram is compared to a family histogram. If the extracted color histogram falls within a specified range of the family histogram, the family histogram is updated to include the extracted color histogram as a new member. If the extracted color histogram does not fall within the specified range of the family histogram, the family histogram is considered complete and the extracted color histogram is utilized to generate a new family histogram for use in processing subsequent extracted color histograms. The resulting family histograms are utilized to detect commercials or other particular type of video content in the video signal.

Abstract: A visualization system captures and analyzes a video signal to extract features in the video signal to render a graphical multi-dimensional visual representation of the program. The visualization system includes a memory and a processor and is programmed to extract features, augment the feature extraction with supplemental information, and render a visual summary to be displayed on a display device. Using the visual summary, a user can more easily determine the nature of a particular video program.

Abstract: A system and method for providing information to television viewers in order to assist them in shopping from an apparently conventional media, such as a television program, as well as making better and more educated shopping decisions are provided. The user selects the product of interest, and if desired, indicates the price, quantity and time frame within which he/she wants to buy the product. The system then acts as an agent on the user's behalf to get the best product under best price and shortest time frame as may be desired. When a user sees a product he wants while watching television, he simply can press a point on the screen, talk, or press a button on a remote control or similar keypad and the system shows the current price and availability. If the user wants the product, he can buy it instantly. If the user determines it to be too expensive, he can add the product to his wish-list with a price he would be willing to pay.

Abstract: An information tracking device receives content data, such as a video or television signal from one or more information sources and analyzes the content data according to a query criteria to extract relevant stories. The query criteria utilizes a variety of information, such as but not limited to a user request, a user profile, and a knowledge base of known relationships. Using the query criteria, the information tracking device calculates a probability of a person or event occurring in the content data and spots and extracts stories accordingly. The results are index, ordered, and then displayed on a display device.

Abstract: A system and method for delivering relevant information to recipients in a timely manner. Information from a plurality of sources is received through interface devices that permit a central processor to evaluate each information stream in light of the dynamic profiles that are stored in a subscriber database. A plurality of location sensors track and report the location of the various recipients. The processor determines the content that will be provided to the recipient and the delivery mode to be used for delivering the selected information to the subscriber.

Abstract: An information tracking device receives content data, such as a video or television signal from one or more information sources and analyzes the content data according to a query criteria to extract relevant stories. The query criteria utilizes a variety of information, such as but not limited to a user request, a user profile, and a knowledge base of known relationships. Using the query criteria, the information tracking device calculates a probability of a person or event occurring in the content data and spots and extracts stories accordingly. The results are index, ordered, and then displayed on a display device.

Abstract: A system and method for collecting, analyzing, and using sensory reactions and involuntary or spontaneous movements by members of a television viewing (or listening) audience. While known programming is displayed on a television receiver, a plurality of sensors monitor the viewer or viewers for recognizable evidence of an emotional response that can be associated with a discrete program segment. Where positive (or negative) responses can be associated with a certain type of program content, the system monitors subsequent programs for the opportunity to notify the viewer or simply present (or avoid presenting) the program automatically.

Abstract: A video indexing system analyzes contents of source video and develops a visual table of contents using selected images. A system for detecting significant scenes detects video cuts from one scene to another, and static scenes based on DCT coefficients and macroblocks. A keyframe filtering process filters out less desired frames including, for example, unicolor frames, or those frames having a same object as a primary focus or one primary focuses. Commercials may also be detected and frames of commercials eliminated. The significant scenes and static scenes are detected based on a threshold which is set based on the category of the video.

Abstract: There is disclosed an apparatus and method for conducting exclusive and inclusive metadata searches to identify and select multimedia programs. The apparatus of the invention comprises a metadata search controller that compares user specified search words with metadata words to find programs that meet user specified search criteria. The metadata search controller executes an inclusive metadata search to search for matches between a user specified search word and a metadata word that is related to the user specified search word in a word pair contained within a word pair database. The metadata search controller calculates a rank value for each program that is found by a metadata search and creates a ranked list of such programs.

Abstract: A multi-lingual transcription system for processing a synchronized audio/video signal containing an auxiliary information component from an original language to a target language is provided. The system filters text data from the auxiliary information component, translates the text data into the target language and displays the translated text data while simultaneously playing an audio and video component of the synchronized signal. The system additionally provides a memory for storing a plurality of language databases which include a metaphor interpreter and thesaurus and may optionally include a parser for identifying parts of speech of the translated text. The auxiliary information component can be any language text associated with an audio/video signal, i.e., video text, text generated by speech recognition software, program transcripts, electronic program guide information, closed caption text, etc.

Abstract: There is disclosed a video playback device for playing back a recorded television program in time-compressed mode on a display screen coupled to the video playback device. The video playback device comprises: 1) a disk storage system for storing the recorded television program as a plurality of audio-video segments; and 2) a fast playback controller for determining a limited time period during which the recorded television program must be played back and further capable of using at least one priority value associated with each of the plurality of audio-video segments to identify selected ones of the plurality of audio-video segments capable of being sequentially played-back during the limited time period.