Abstract: Methods, systems and computer program products generating diverse and representative set of samples from a large amount of transaction data are disclosed. A data sampling system receives transaction records. Each transaction record has multiple text segments. The system selects a subset of transaction records that contain least frequently appeared text segments. The system determines a respective vector representation for each selected transaction record. The system can measure similarity between transaction records based on the vector representations. The system assigns the selected transaction records to multiple clusters based on the vector representations and designated dimensions of importance. The system identifies one or more anchors that include transaction records on boundaries between clusters. The system filters the subset of transaction records by removing transaction records that are close to the anchors.

Abstract: Methods, systems and computer program products generating diverse and representative set of samples from a large amount of transaction data are disclosed. A data sampling system receives transaction records. Each transaction record has multiple text segments. The system selects a subset of transaction records that contain least frequently appeared text segments. The system determines a respective vector representation for each selected transaction record. The system can measure similarity between transaction records based on the vector representations. The system assigns the selected transaction records to multiple clusters based on the vector representations and designated dimensions of importance. The system identifies one or more anchors that include transaction records on boundaries between clusters. The system filters the subset of transaction records by removing transaction records that are close to the anchors.

Abstract: Methods, systems and computer program products implementing data enrichment using global structure learning are disclosed. An information enrichment system predicts a likely canonical name from a transaction record in which names may be shortened, or extra token(s) inserted. In a training phase, the information enrichment system determines tag patterns based on labeled and unlabeled training transaction records. The tag patterns include co-occurrence probability and sequential order of co-occurrence of tags. In a testing phase, the information enrichment system receives a test transaction record. The information enrichment system predicts a likely tag sequence from the test transaction record based on the tag patterns. The information enrichment system predicts a canonical name based on likely tag values and token composition. The information enrichment system can then enrich the test transaction record with the predicted canonical name.