Abstract: Techniques and systems are described to control a determination of document similarity. In one example, dimensionality of the documents is reduced through computation of a signature, e.g., via a hashing technique such as “minhashing” which is also known as min-wise independent permutations locality sensitive hashing. From these signatures, another hashing technique (e.g., locality sensitive hashing) is used to determine similarity of the signatures to each other. Identification of disjoint sets is then used as a basis to partition the documents for determination of document similarity by respective nodes of a plurality of computing devices. In this way, an amount of data shuffling between the nodes as part of the determination of document similarity may be reduced. In another example, a weighting is applied to attributes of documents as part of the determination of document similarity.

Abstract: Techniques and systems are described to control a determination of document similarity. In one example, dimensionality of the documents is reduced through computation of a signature, e.g., via a hashing technique such as “minhashing” which is also known as min-wise independent permutations locality sensitive hashing. From these signatures, another hashing technique (e.g., locality sensitive hashing) is used to determine similarity of the signatures to each other. Identification of disjoint sets is then used as a basis to partition the documents for determination of document similarity by respective nodes of a plurality of computing devices. In this way, an amount of data shuffling between the nodes as part of the determination of document similarity may be reduced. In another example, a weighting is applied to attributes of documents as part of the determination of document similarity.