Abstract: A system and method for placing candidate spines into a display with the aid of a digital computer is provided. Candidate spines each having two or more clusters of documents are generated. Each cluster is associated with one or more concept scores for concepts identified within the documents of that cluster. A concept score vector is generated for each candidate spine by aggregating as spine concept scores, the cluster concept scores of the clusters associated with that candidate spine. The candidate spines are ordered by a number of the clusters associated with each candidate spine. Those candidate spines that are sufficiently dissimilar from the other candidate spines are identified based on the candidate spine concept score vectors. The sufficiently dissimilar candidate spines are placed in a display and at least a portion of the unplaced candidate spines are placed in relation to the placed candidate spines.

Abstract: A computer-implemented system and method for spine group generation and arrangement are provided. A set of spine groups each having one or more spines of clusters and at least one singleton cluster is obtained. A set of concentric rings are designated in a display and at least a portion of the spine groups are placed within the concentric rings. An innermost ring includes one or more of the spine groups that are unique from the other spine groups. A primary ring includes a majority of the spine groups placed within the display and an outermost ring delineates placement of descriptive labels for the placed spine groups.

Abstract: A computer-implemented system and method for generating document groupings is provided. A lexicon of terms extracted from a set of documents is generated. The lexicon includes a frequency of each extracted term within each document in the set. Concepts each having two or more of the extracted terms are generated. A subset of the documents in the set is selected based on the term frequencies. The subset of documents is grouped into clusters based on the concepts. A similarity of each document cluster is calculated with one or more documents based on a distance by summing the frequency of each term in that document and a weight of the cluster for each of the terms. The weights are updated until a rate of change for each cluster becomes constant.

Abstract: A computer-implemented system and method for generating and placing cluster groups is provided. A set of clusters each having one or more documents is maintained. A portion of the clusters is assigned to two or more spines. The cluster spines are ordered by cluster length. Those cluster spines that are unique are identified in decreasing cluster length order and placed. One or more of the remaining spines are placed in relation to at least one of the placed unique spines. At least one of the remaining clusters is placed in relation to one or more of the unique placed spines. Groups of clusters are displayed. Each group includes two or more of one such unique placed spine, the remaining spines placed in relation to that placed unique spine; and the remaining clusters placed in relation to the unique placed spine.

Abstract: A computer-implemented system and method for generating clusters for placement into a display is provided. A set of clusters is generated from a document set. A single cluster of related documents from the document set is obtained and at least one new cluster is added. One such document in the set is compared to the cluster. A difference in distance between the document and a common origin and the cluster and the common origin is determined. The document is designated as the new cluster when the difference fails to satisfy a predetermined threshold. One or more cluster spines each having two or more clusters placed along a vector are placed into a display. The clusters along each spine are identified as similar and the clusters of one such spine are also similar to further clusters located along a further spine having a small cosine rotation from that cluster spine.

Abstract: A computer-implemented system and method for placing groups of cluster spines into a display is provided. A shape is provided within a display and sectors are designated around the shape. One or more spine groups are placed into the display. Each spine group has two or more vectors of documents clusters and is placed adjacent to the shape within one of the sectors. A further spine group is placed into the display adjacent to one of the placed spine groups most similar to the further spine group. A determination is made that the further spine group overlaps the most similar placed spine group. The further spine group and the most similar placed spine group are translated away from the shape when rotation of the overlapping further spine group, away from the most similar placed spine group, places the further spine group into a different sector.

Abstract: A system and method for identifying related messages are provided. A set of messages, each having a header, sender and transmission time, is obtained. A message is selected from the set. A body of the selected message is compared to a body of a further message in the set. The further message is labeled as a duplicate of the selected message when the bodies match. The duplicate labeling of the further message is verified when the header, sender, and transmission time of the further message matches the header, sender, and transmission time of the selected message. The duplicate messages are removed from the set. The remaining messages are sorted in order of message length. A shorter message is compared with a longer message and is marked as a near duplicate of the longer message when the body of the shorter message is included in the body of the longer message.

Abstract: A computer-implemented system and method for placing groups of document clusters into a display is provided. One or more spines of document clusters are placed into a display and at least one of the document clusters for each placed spine is designated as an anchor cluster. At least one of the unplaced spines is compared with each of the placed spines in the display and one of the placed spines most similar to the unplaced spine is identified. The document clusters of the unplaced spine are compared with the anchor cluster of the most similar placed spine and the cluster on the unplaced spine that is most similar to the anchor cluster on the most similar placed spine is identified. The most similar cluster of the unplaced spine is grafted to the anchor cluster of the most similar placed spine.

Abstract: A computer-implemented system and method for visualizing message-related relationships within a social network is provided. Entities associated with a group of messages are identified. A social network is built based on a relationship of the entities and messages by connecting those entities that share at least one of the messages from the group. A representation of the social network is generated by representing each message with a node. The nodes are organized based on at least one characteristic of the messages.

Abstract: A computer-implemented system and method for identifying duplicate and near duplicate messages is provided. A set of messages is obtained. A body of one such message is compared with the body of each other message. Those messages having matching bodies are identified as exact duplicates. The exact duplicates are removed from the set. The remaining messages are sorted in order of message length and a shorter message is compared with a longer message. A determination is made that the body of the shorter message is included in the body of the longer message and the shorter message is marked as a near duplicate of the longer message.

Abstract: A computer-implemented system and method for identifying relevant documents for display are provided. Themes for a set of documents are generated. The documents are clustered based on the themes. A matrix including an inner product of document frequency occurrences and cluster concept weightings for each theme is generated for the documents. From the matrix, documents most relevant to a particular theme are identified, and the relevant documents are displayed.

Abstract: A computer-implemented system and method for providing a display of clusters is provided. A plurality of cluster spines is presented in a two-dimensional display. Each cluster spine includes a vector of document clusters. A compass is positioned over at least a portion of the clusters of one or more of the spines. A spine label for at least one of the spines within the compass is placed around a circumference of the compass. One of the spine labels is pinned to the compass at a fixed location. The compass is reoriented within the display and the pinned spine label is displayed at the fixed location on the reoriented compass.

Abstract: A computer-implemented system and method for organizing cluster groups within a display is provided. Cluster groups each having one or more spines of clusters are obtained. Each cluster includes at least one document. At least one of the cluster groups is placed around a ring centrally defined in a display. Circle sectors are defined around the ring and an initial target angle is identified within each of the sectors. The at least one cluster group is positioned within one of the circle sectors at the initial target angle. A further one of the cluster groups is positioned within a different circle sector up to one of a maximum and minimum angle relative to the initial target angle for that sector.

Abstract: A computer-implemented system and method for correcting a rendering of clusters is provided. A pair of clusters is selected within a representation. A span between centers of the clusters is determined. Radii for each of the clusters in the pair is identified. The radii are summed and at least one of the clusters in the pair is moved within the representation when the span exceeds the sum.

Abstract: A computer-implemented system and method for populating clusters of documents is provided. A set of clusters is placed in a display in relation to a common origin. One of a plurality of unclustered documents in the display is selected and an angle ? of the document from the common origin is determined. An angle ? of the cluster relative to the common origin is computed for each cluster. A difference is determined between the document angle ? and one such cluster angle ?. A predetermined variance is applied to the difference. The document is placed into the cluster when the difference is less than the variance.

Abstract: A computer-implemented system and method for organizing cluster groups within a display is provided. Cluster groups each having one or more spines of clusters are obtained. Each cluster includes at least one document. At least one of the cluster groups is placed around a ring centrally defined in a display. Circle sectors are defined around the ring and an initial target angle is identified within each of the sectors. The at least one cluster group is positioned within one of the circle sectors at the initial target angle. A further one of the cluster groups is positioned within a different circle sector up to one of a maximum and minimum angle relative to the initial target angle for that sector.

Abstract: A computer-implemented system and method for clustering similar documents is provided. Concepts are identified for a set of documents and occurrence frequencies are determined for each concept in the documents set. A distance quantifying a similarity for each of the documents in the set with one or more clusters of documents is calculated. Each document is mapped to at least one of the one or more document clusters.

Abstract: A computer-implemented system and method for clustering documents based on scored concepts is provided. A set of documents is obtained and concepts are extracted from the documents. A score is calculated for each concept. The score is determined as a function of summation of a frequency of occurrence, concept weight, structural weight, and corpus weight. The documents in the set are clustered based on the scores. A vector is formed for each document based on the concepts in that document and the scores associated with the concepts. A similarity is determined between each document and each of the other documents based on the formed vectors. Those documents that are sufficiently distinct from the other documents are identified as seed documents for separate document clusters. Each of the remaining documents are grouped into one of the clusters most similar to that remaining document.

Abstract: A computer-implemented system and method for identifying duplicate and near duplicate messages is provided. A set of messages is obtained. A body of one such message is compared with the body of each other message. Those messages having matching bodies are identified as exact duplicates. The exact duplicates are removed from the set. The remaining messages are sorted in order of message length and a shorter message is compared with a longer message. A determination is made that the body of the shorter message is included in the body of the longer message and the shorter message is marked as a near duplicate of the longer message.

Abstract: A system and method for providing a user-adjustable display of clusters and text is provided. A two-dimensional display of cluster spines is provided. The cluster spines each include clusters of documents proximately aligned. A compass is provided within the display and positioned over one or more clusters of at least one cluster spine based on instructions from a user. The display of the clusters positioned within the compass is altered. Spine labels are positioned circumferentially around the compass. Each spine label represents a concept associated with one of the cluster spines positioned within the compass.