Abstract: Methods, systems, and techniques for producing and using enhanced machine learning models and computer-implemented tools to investigate cybersecurity related data and threat intelligence data are provided. Example embodiments provide an Enhanced Predictive Security System, for building, deploying, and managing applications for evaluating threat intelligence data that can predict malicious domains associated with bad actors before the domains are known to be malicious. In one example, the EPSS comprises one or more components that work together to provide an architecture and a framework for building and deploying cybersecurity threat analysis application, including machine learning algorithms, feature class engines, tuning systems, ensemble classifier engines, and validation and testing engines.

Abstract: Methods, systems, and techniques for producing and using enhanced machine learning models and computer-implemented tools to investigate cybersecurity related data and threat intelligence data are provided. Example embodiments provide an Enhanced Predictive Security System, for building, deploying, and managing applications for evaluating threat intelligence data that can predict malicious domains associated with bad actors before the domains are known to be malicious. In one example, the EPSS comprises one or more components that work together to provide an architecture and a framework for building and deploying cybersecurity threat analysis application, including machine learning algorithms, feature class engines, tuning systems, ensemble classifier engines, and validation and testing engines.

Abstract: Methods, systems, and techniques for producing and using enhanced machine learning models and computer-implemented tools to investigate cybersecurity related data and threat intelligence data are provided. Example embodiments provide an Enhanced Predictive Security System, for building, deploying, and managing applications for evaluating threat intelligence data that can predict malicious domains associated with bad actors before the domains are known to be malicious. In one example, the EPSS comprises one or more components that work together to provide an architecture and a framework for building and deploying cybersecurity threat analysis application, including machine learning algorithms, feature class engines, tuning systems, ensemble classifier engines, and validation and testing engines.

Abstract: A computer-implemented system and method for generating document training sets is provided. Unclassified documents are provided to two or more classifiers. A classification code assigned to each unclassified document is received. A determination is made as to whether a disagreement exists between classification codes assigned to a common unclassified document via different classifiers. The common unclassified document with a disagreement in classification codes are provided for further review. Results of the further review include one of a new classification code and confirmation of one of the assigned classification codes. The unclassified documents for which a disagreement exists are grouped as a training set.

Abstract: A computer-implemented system and method for identifying similar documents is provided. A set of documents is obtained. Each document in the set is divided into segments and the segments are hashed. The hashed segments of at least two of the documents are compared. Hashed segments shared between the at least two documents are identified. A number of the hashed segments shared between the at least two documents is summed and a total number of hashed segments within the at least two documents is summed. A ratio of similarity between the at least two documents is determined based on the number of shared hashed segments and the total number of hashed segments.

Abstract: A computer-implemented system and method for identifying near duplicate documents is provided. A set of documents is obtained and each document is divided into segments. Each of the segments is hashed. A segment identification and sequence order is assigned to each of the hashed segments. The sequence order is based on an order in which the segments occur in one such document. The segments are compared based on the segment identification and those documents with at least two matching segments are identified. The sequence orders of the matching segments are compared and based on the comparison, a determination is made that the identified documents share a relative sequence of the matching segments. The identified documents are designated as near duplicate documents.

Abstract: A computer-implemented system and method for generating document training sets is provided. Unclassified documents are provided to two or more classifiers. A classification code assigned to each unclassified document is received. A determination is made as to whether a disagreement exists between classification codes assigned to a common unclassified document via different classifiers. The common unclassified document with a disagreement in classification codes are provided for further review. Results of the further review include one of a new classification code and confirmation of one of the assigned classification codes. The unclassified documents for which a disagreement exists are grouped as a training set.

Abstract: A computer-implemented system and method for generating a training set for use during document review is provided. Classification codes are assigned to a set of documents. Further classification codes are assigned to the same set of documents. The classification code for at least one document is compared with the further classification code for that document. A determination regarding whether a disagreement exists between the assigned classification code and the further classification code for at least one document is made. Those documents with disagreeing classification codes are identified as training set candidates. A stop threshold is applied to the training set candidates and the training set candidates are grouped as a training set when the stop threshold is satisfied.

Abstract: A computer-implemented system and method for generating a reference set via clustering is provided. A collection of unclassified documents is obtained and grouped into clusters. N-documents are selected from each cluster and are combined as reference set candidates. One of the n-documents from each cluster is located closest to a center of that cluster. A classification code is assigned to each of the reference set candidates. Two or more of the reference set candidates are grouped as a reference set of classified documents.

Abstract: A computer-implemented system and method for generating a reference set via seed documents is provided. A collection of documents is obtained. One or more seed documents are identified. The seed documents are compared with the document collection and those documents that are similar to the seed documents are identified as reference set candidates. A size threshold is applied to the reference set candidates, which are grouped as the reference set when the size threshold is satisfied.

Abstract: A computer-implemented system and method for generating a training set for use during document review is provided. Classification codes are assigned to a set of documents. Further classification codes are assigned to the same set of documents. The classification code for at least one document is compared with the further classification code for that document. A determination regarding whether a disagreement exists between the assigned classification code and the further classification code for at least one document is made. Those documents with disagreeing classification codes are identified as training set candidates. A stop threshold is applied to the training set candidates and the training set candidates are grouped as a training set when the stop threshold is satisfied.

Abstract: A computer-implemented system and method for generating a reference set via seed documents is provided. A collection of documents is obtained. One or more seed documents are identified. The seed documents are compared with the document collection and those documents that are similar to the seed documents are identified as reference set candidates. A size threshold is applied to the reference set candidates, which are grouped as the reference set when the size threshold is satisfied.

Abstract: A computer-implemented system and method for generating a reference set via clustering is provided. A collection of unclassified documents is obtained and grouped into clusters. N-documents are selected from each cluster and are combined as reference set candidates. One of the n-documents from each cluster is located closest to a center of that cluster. A classification code is assigned to each of the reference set candidates. Two or more of the reference set candidates are grouped as a reference set of classified documents.

Abstract: A computer-implemented system and method for identifying near duplicate documents is provided. A set of documents is obtained and each document is divided into segments. Each of the segments is hashed. A segment identification and sequence order is assigned to each of the hashed segments. The sequence order is based on an order in which the segments occur in one such document. The segments are compared based on the segment identification and those documents with at least two matching segments are identified. The sequence orders of the matching segments are compared and based on the comparison, a determination is made that the identified documents share a relative sequence of the matching segments. The identified documents are designated as near duplicate documents.

Abstract: A system and method for providing generating reference sets for use during document review is provided. A collection of unclassified documents is obtained. Selection criteria are applied to the document collection and those unclassified documents that satisfy the selection criteria are selected as reference set candidates. A classification code is assigned to each reference set candidate. A reference set is formed from the classified reference set candidates. The reference set is quality controlled and shared between one or more users.

Abstract: A system and method for providing generating reference sets for use during document review is provided. A collection of unclassified documents is obtained. Selection criteria are applied to the document collection and those unclassified documents that satisfy the selection criteria are selected as reference set candidates. A classification code is assigned to each reference set candidate. A reference set is formed from the classified reference set candidates. The reference set is quality controlled and shared between one or more users.