Abstract: A digital capture system, such as a digital still camera or digital video recorder, has an embedded real-time content-based analysis function in the capture device to extract metadata from the digital signals. In one embodiment, metadata (descriptive information about the digital content) is formatted and stored separately from the content. In another embodiment, the metadata may be formatted and combined with the digital content in a container format such as MPEG-7, QuickTime, or FlashPix.

Abstract: A system and method that processes video to extract a keyframe-based adequate visual representation. The method utilizes a hierarchical processing technique. The first stage in the hierarchy extracts a chromatic difference metric from a pair of video frames. An initial set of frames is chosen based on the chromatic metric and a threshold. A structural difference measurement is extracted from this initial set of frames. A second threshold is used to select key frames from the initial set. The first and second thresholds are user selectable. The output of this process is the visual representation. The method is extensible to any number of metrics and any number of levels.

Abstract: One aspect of the invention is directed to a system and method for video cataloging. The video is cataloged according to predefined or user definable metadata. The metadata is used to index and then retrieve encoded video. In another aspect of the invention, video metadata track processors convert metadata tracks of the video information to produce displayable frames containing hyperlinking between displayable data. Another aspect of the invention is directed to a method of browsing stored video information, including displaying hyperlinked frames of metadata track representations, and selecting and displaying links between displayable frames.

Abstract: One aspect of the invention is directed to a system and method for video cataloging. The video is cataloged according to predefined or user definable metadata. The metadata is used to index and then retrieve encoded video. Video feature extractors produce metadata tracks from the video information, and each metadata track indexes the stored video information. A feature extractor registration interface registers the video feature extractors, providing for registration with the video engine of new video feature extractors with new metadata tracks.

Abstract: One aspect of the invention is directed to a system and method for video cataloging. The video is cataloged according to predefined or user definable metadata. The metadata is used to index and then retrieve encoded video.

Abstract: One aspect of the invention is directed to a search engine having indexed retrieval to improve computational efficiency of searching large databases of rich objects such as images. Feature vectors are extracted from images and stored in a feature vector database. When a query is submitted to the engine, a query feature vector Q will be specified, as well as a distance threshold T, indicating the maximum distance that is of interest for this query. All images within a distance of T will be identified by the query. Range constraints are defined such that all feature vectors within a distance of T of the query feature vector, satisfy all of the range constraints. The constraint is dependent on the specific primitive being indexed. The constraint is also defined such that any feature vector which is within a distance of T of the query feature vector also satisfies a functional constraint.

Abstract: A system and method for improving the retrieval performance of a query engine in a visual information retrieval (VIR) system by encoding domain-specific knowledge into the VIR system through a visual dictionary or "victionary". The victionary is a dictionary-like information-mapping module that is used to retrieve visual information at a "semantic" level. A VIR system that performs generic image processing is enhanced by adding a query transformation unit and a query expansion unit, i.e., the victionary. With these additional components, a user may present a query either as a text term (such as a keyword or phrase), or as an image (with weights) and execute a "semantic query". During semantic query processing, the victionary-enhanced system transforms the user's original term (or image query) to a set of equivalent queries, and internally executes all the equivalent queries before presenting the results to the user.

Abstract: A system and method for content-based search and retrieval of visual objects. A base visual information retrieval (VIR) engine utilizes a set of universal primitives to operate on the visual objects. An extensible VIR engine allows custom, modular primitives to be defined and registered. A custom primitive addresses domain specific problems and can utilize any image understanding technique. Object attributes can be extracted over the entire image or over only a portion of the object. A schema is defined as a specific collection of primitives. A specific schema implies a specific set of visual features to be processed and a corresponding feature vector to be used for content-based similarity scoring. A primitive registration interface registers custom primitives and facilitates storing of an analysis function and a comparison function to a schema table. A heterogeneous comparison allows objects analyzed by different schemas to be compared if at least one primitive is in common between the schemas.

Abstract: A system and method for content-based search and retrieval of visual objects. A base visual information retrieval (VIR) engine utilizes a set of universal primitives to operate on the visual objects. An extensible VIR engine allows custom, modular primitives to be defined and registered. A custom primitive addresses domain specific problems and can utilize any image understanding technique. Object attributes can be extracted over the entire image or over only a portion of the object. A schema is defined as a specific collection of primitives. A specific schema implies a specific set of visual features to be processed and a corresponding feature vector to be used for content-based similarity scoring. A primitive registration interface registers custom primitives and facilitates storing of an analysis function and a comparison function to a schema table. A heterogeneous comparison allows objects analyzed by different schemas to be compared if at least one primitive is in common between the schemas.

Abstract: A system and method for content-based search and retrieval of visual objects. A base visual information retrieval (VIR) engine utilizes a set of universal primitives to operate on the visual objects. An extensible VIR engine allows custom, modular primitives to be defined and registered. A custom primitive addresses domain specific problems and can utilize any image understanding technique. Object attributes can be extracted over the entire image or over only a portion of the object. A schema is defined as a specific collection of primitives. A specific schema implies a specific set of visual features to be processed and a corresponding feature vector to be used for content-based similarity scoring. A primitive registration interface registers custom primitives and facilitates storing of an analysis function and a comparison function to a schema table. A heterogeneous comparison allows objects analyzed by different schemas to be compared if at least one primitive is in common between the schemas.

Abstract: A system and method for content-based search and retrieval of visual objects. A base visual information retrieval (VIR) engine utilizes a set of universal primitives to operate on the visual objects. An extensible VIR engine allows custom, modular primitives to be defined and registered. A custom primitive addresses domain specific problems and can utilize any image understanding technique. Object attributes can be extracted over the entire image or over only a portion of the object. A schema is defined as a specific collection of primitives. A specific schema implies a specific set of visual features to be processed and a corresponding feature vector to be used for content-based similarity scoring. A primitive registration interface registers custom primitives and facilitates storing of an analysis function and a comparison function to a schema table. A heterogeneous comparison allows objects analyzed by different schemas to be compared if at least one primitive is in common between the schemas.