Abstract: A system that represents a video sequence comprising of a plurality of video clips as number of images. The plurality of video clips is represented as a reduced representation of video images. A video clip is represented as a keyframe, wherein multiple keyframes may then be arranged according to chronological order. All or only a representative portion of the video clips can be represented as keyframes. The size of the keyframe may be configured to represent the length or importance of the video clip. The keyframe may depict an entire frame of a video clip, or a region of meaningful information within a frame of a video clip. Multiple keyframes may be arranged in a two dimensional array, in an S-shaped curve, or some other pattern. The keyframes may depict motion of an object occurring overtime in the video clip by configuring groups of pixels in the key frame.

Abstract: A method, system, and apparatus for easily creating a video collage from a video is provided. By segmenting the video into a set number of video segments and providing an interface for a user to select images which represent the video segments and insert the selected images into a video collage template, a video collage may be easily created in a short amount of time. The system is designed to assign values to the video inserted in a video collage and compact the video based on these values thereby creating a small file which may be easily stored or transmitted.

Abstract: A system for automatically generating indexes for handwritten notes captured as digital ink in a computer is disclosed. Ink words are identified, and features of the ink words are computed. Pairwise distances or match scores, which measure the distance in the features between two ink words, are calculated. A clustering technique selects equivalence classes of ink words. Index terms, which are non-uniform through-out the notes, are selected from the equivalence classes of ink words. The system generates an index from the index terms, including displaying pages numbers where the index terms are located in the notes as well as hyper-linking the index terms. A technique to identify a threshold for use in clustering the ink words is also disclosed.

Abstract: A method of extracting audio excerpts comprises: segmenting audio data into a plurality of audio data segments; setting a fitness criteria for the plurality of audio data segments; analyzing the plurality of audio data segments based on the fitness criteria; and selecting one of the plurality of audio data segments that satisfies the fitness criteria. In various exemplary embodiments, the method of extracting audio excerpts further comprises associating the selected one of the plurality of audio data segments with video data. In such embodiments, associating the selected one of the plurality of audio data segments with video data may comprise associating the selected one of the plurality of audio data segments with a keyframe.

Abstract: An audio data system assigns properties to audio data and keywords to audio data files. The properties indicate the type of audio data and the keywords indicate the general content of an audio data file. The system provides an updatable database of audio data that is capable of being searched based on the properties and the keywords.

Abstract: A method for recognizing user specified pen-based gestures uses Hidden Markov Models. A gesture recognizer is implemented which includes a fast pruning procedure. In addition, an incremental training method is utilized.

Abstract: A document organizing system extracts annotations made to a document along with the context surrounding each annotation and organizes the annotations based upon the annotation attributes and/or context. The annotations are created by grouping marks based upon their proximity in time and space. The document is segmented to determine a minimum context associated with each annotation. A list of the annotations sorted by the attributes are then displayed to the user. The context provided by the invention for each annotation allows the user to fully understand the annotation.

Abstract: Detection of video shot boundaries using a Video Segmenting Hidden Markov Model to model the sequence of states of a video. The Video Segmenting Hidden Markov Model determines the state sequence based on feature values. Using Hidden Markov Model techniques allows for automatic learning and use of multiple features including motion vectors, audio differences and histogram differences, without the need for manual adjustments of these thresholds.

Abstract: An electronic notebook assigns properties to digital ink of note marks and keywords to digital pages. The properties indicate the type of note mark and the keywords indicate general content of the note marks on a digital page. The notebook also synchronizes audio data with the note marks and permits audio highlighting. The electronic notebook provides an updatable database of note marks and audio that is capable of being searched based on the properties, keywords, and audio highlights.

Abstract: When dynamically grouping a plurality of graphic objects, such as displayed on a graphic input display apparatus, a cluster tree is formed for the plurality of graphic objects. The cluster tree is based on a plurality of different types of distance measures. These include a time distance and a spatial distance. These distances are combined to form a distance metric indicting a distance between a pair of the graphic objects. Each level of the cluster tree defines a new cluster of the graphic objects. At least one of the graphic objects is selected. The different cluster levels of the cluster tree containing the selected graphic object are displayable. The displayed cluster of the graphic objects can be modified to increase or decrease the cluster level of the cluster containing the selected graphic object.

Abstract: Font-independent spotting of user-defined keywords in a scanned image. Word identification is based on features of the entire word without the need for segmentation or OCR, and without the need to recognize non-keywords. Font-independent character models are created using hidden Markov models (HMMs) and arbitrary keyword models are built from the character HMM components. Word or text line bounding boxes are extracted from the image, a set of features based on the word shape, (and preferably also the word internal structure) within each bounding box is extracted, this set of features is applied to a network that includes one or more keyword HMMs, and a determination is made. The identification of word bounding boxes for potential keywords includes the steps of reducing the image (say by 2.times.) and subjecting the reduced image to vertical and horizontal morphological closing operations.

Abstract: Font-independent spotting of user-defined keywords in a scanned image. Word identification is based on features of the entire word without the need for segmentation or OCR, and without the need to recognize non-keywords. Font-independent character models are created using hidden Markov models (HMMS) and arbitrary keyword models are built from the character HMM components. Word or text line bounding boxes are extracted from the image, a set of features based on the word shape, (and preferably also the word internal structure) within each bounding box is extracted, this set of features is applied to a network that includes one or more keyword HMMs, and a determination is made. The identification of word bounding boxes for potential keywords includes the steps of reducing the image (say by 2.times.) and subjecting the reduced image to vertical and horizontal morphological closing operations.

Abstract: A system for capturing and controlling access to concurrently recorded timestreams of a session. A session is of a collaborative activity such as a meeting. Timestreams of the session are captured by various recording devices. The timestreams are analyzed to create a set of events for each timestream. An event is subsequently used as an index for replaying the session. Playback of the timestreams is done in order to recall dialogs or other events that occurred during the meeting. The timestreams are played back contemporaneously. A timeline interface having the events mapped thereon is used to traverse through the different portions of the session.

Abstract: A system for the capture and playback of temporal data representing a collaborative activity such as a meeting. Such temporal data is captured by one or more capture devices (e.g. audio recordings or an electronic whiteboard). Temporal data is comprised of timestreams and events. Timestreams are sets of timestamped data which represent some recordable aspect of the meeting. Events are contained within a timestream that represent natural activities that occurred during the course of the session. Events are used to create indices which provide direct access to a point or span in time during the collaborative activity. Playback of a session is performed under the control of a session access device. Coupled to the session access device are a plurality of players for playing back timestreams. The session access device utilizes event information to create a user interface for controlling session replay. The user interface is comprised of a plurality of windows.

Abstract: A system and method for unsupervised clustering of audio data segments in an audio data recording containing speech from multiple speakers including the steps of: 1) providing a portion of the audio data containing speech from all of the speakers; 2) forming initial clusters by dividing the portion of the audio data into segments, each of which includes an ordered data set; 3) computing the pairwise distance between each pair of clusters using a likelihood ration independent of the order of data within the segments; and 4) combining into a new cluster the two clusters with a minimum pairwise distance. These steps are repeated until a number of clusters equal to the number of speakers is obtained.

Abstract: A method for segmenting audio data, comprising speech from a plurality of individual speakers, according to speaker is provided. The method comprises providing individual HMMs for each individual speaker, each individual HMM including at least one state, and constructing a speaker network HMM by connecting the individual HMMs in parallel. The audio data is then divided into segments by determining a most likely sequence of states through the speaker network HMM, each of the segments being associated with one of the individual HMMs. Afterward, the speaker of each of the segments is identified. The segmented data may be used to form an index into the audio data according to speaker.

Abstract: A processor controlled system for correlating an electronic index according to speaker for audio data being recorded in real time. The system includes a source of training data for each of the plurality of individual speakers and audio input system for providing real time audio data including speech for the individual speakers. The audio data is converted into spectral feature data by an audio processor, and is simultaneously recorded on a storage medium by a recording device. A system processor accepts the training data to create individual speaker models, which are combined in parallel to form a speaker network. The system processor then accepts the spectral feature data of the audio data and, using the speaker network, determines segments in the audio data corresponding to each speaker.

Abstract: A method for clustering speaker data from a plurality of unknown speakers. The method includes steps of providing a portion of audio data containing speech from at least all the speakers in the audio data and dividing the portion into data clusters. A pairwise distance between each pair of clusters is computed, the pairwise distance being based on a likelihood that two clusters were created by the same speaker, the likelihood measurement being biased by the prior probability of speaker changes. The two clusters with a minimum pairwise distance are combined into a new cluster and speakers models are trained for each of the remaining clusters including the new cluster. The likelihood that two clusters were created by the same speaker may be biased by a Markov duration model based on speaker changes over the length of the initial data clusters.

Abstract: Font-independent spotting of user-defined keywords in a scanned image. Word identification is based on features of the entire word without the need for segmentation or OCR, and without the need to recognize non-keywords. Font-independent character models are created using hidden Markov models (HMMs) and arbitrary keyword models are built from the character HMM components. Word or text line bounding boxes are extracted from the image, a set of features based on the word shape, (and preferably also the word internal structure) within each bounding box is extracted, this set of features is applied to a network that includes one or more keyword HMMs, and a determination is made. The identification of word bounding boxes for potential keywords includes the steps of reducing the image (say by 2x) and subjecting the reduced image to vertical and horizontal morphological closing operations.

Abstract: Font-independent spotting of user-defined keywords in a scanned image. Word identification is based on features of the entire word without the need for segmentation or OCR, and without the need to recognize non-keywords. Font-independent character models are created using hidden Markov models (HMMs) and arbitrary keyword models are built from the character HMM components. Word or text line bounding boxes are extracted from the image, a set of features based on the word shape, (and preferably also the word internal structure) within each bounding box is extracted, this set of features is applied to a network that includes one or more keyword HMMs, and a determination is made. The identification of word bounding boxes for potential keywords includes the steps of reducing the image (say by 2.times.) and subjecting the reduced image to vertical and horizontal morphological closing operations.