Abstract: In some embodiments, a system for automatically identifying synonyms within a token-based data management system includes a database configured to store a plurality of records, and a computing device having a synonym locator configured to create a token synonym mapping by automatically identifying token synonyms within the plurality of records based on a statistical analysis of the plurality of records. The token synonym mapping includes a first token linked to a second token, where the second token is a valid synonym of the first token. The computing device includes a synonym standardizer configured to standardize at least one record of the plurality of records based on the token synonym mapping such that, when the at least one record includes the second token, the synonym standardizer is configured to automatically replace the second token with the first token.

Abstract: A token-based database management system described herein may reduce an amount of comparisons during entity resolution of records. The system includes a token creator configured to create tokens from records, a token-record mapping creator configured to create a token-record mapping of tokens to records, a token importance calculator configured to calculate token importance values for the tokens, a token pruner configured to identify a token of the current record as unimportant based on token importance values of the tokens of the current record, and to remove the unimportant token from the token-record mapping, a record selector configured to select only records sharing at least one common token with the current record, and a record comparator configured to compare the current record with each of the selected records to determine whether the current record matches any of the selected records.

Abstract: A token-based database management system described herein may reduce an amount of comparisons during entity resolution of records. The system includes a token creator configured to create tokens from records, a token-record mapping creator configured to create a token-record mapping of tokens to records, a token importance calculator configured to calculate token importance values for the tokens, a token pruner configured to identify a token of the current record as unimportant based on token importance values of the tokens of the current record, and to remove the unimportant token from the token-record mapping, a record selector configured to select only records sharing at least one common token with the current record, and a record comparator configured to compare the current record with each of the selected records to determine whether the current record matches any of the selected records.

Abstract: In some embodiments, a system for automatically identifying synonyms within a token-based data management system includes a database configured to store a plurality of records, and a computing device having a synonym locator configured to create a token synonym mapping by automatically identifying token synonyms within the plurality of records based on a statistical analysis of the plurality of records. The token synonym mapping includes a first token linked to a second token, where the second token is a valid synonym of the first token. The computing device includes a synonym standardizer configured to standardize at least one record of the plurality of records based on the token synonym mapping such that, when the at least one record includes the second token, the synonym standardizer is configured to automatically replace the second token with the first token.