Abstract: An apparatus, computer-readable medium, and computer-implemented method for efficiently performing operations on distinct data values, including storing a tokenized column of data in a table by mapping each unique data value in a corresponding domain to a unique entity ID, and replacing each of the data values in the column with the corresponding entity ID to generate a column of tokenized data containing one or more entity IDs, receiving a query directed to the column of data, the query defining one or more group sets for grouping the data retrieved in response to the query, and generating an entity map vector for each group set, the length of each entity map vector equal to the number of unique entity IDs for the domain, and the value of each bit in the entity map vector indicating the presence or absence of a different unique entity ID in the group set.

Abstract: A data management service identifies sensitive data stored on enterprise databases according to record classification rules that classify a data record as having a sensitive data type if the data record includes fields matching at least one of the record classification rules. The data management service determines assessment scores for enterprise databases according to sensitive data records and protection policies on the enterprise databases. The data management service provides an interface that groups enterprise databases having common attributes or common sensitive data types and indicates aggregated assessment scores for the groups of enterprise databases. Through the interface with the grouped enterprise databases, an administrator apply protection policies to enterprise databases. To apply the protection policy, the data management service applies the protection policy to a source database from which dependent enterprise databases access the sensitive database.

Abstract: A data management service identifies sensitive data stored on enterprise databases according to record classification rules that classify a data record as having a sensitive data type if the data record includes fields matching at least one of the record classification rules. Methods and systems rely on a set of impact factors each having a set of set of value bands representing a range for the impact factor and a corresponding value (between 0 and 1). The factors, ranges, and values all are customizable for an organization. Impact scoring calculations take into account each of the impact factors, and each is weighted to represent a specific risk perception or assessment type. A similar impact scoring is applied to data quality using volume of data as a key attribute of the quality.

Abstract: A test data extraction and persistence technique that relies on a data domain based storage infrastructure is disclosed. In operation, a test data server receives a test data query that specifies selection parameters for selecting test data and any transformation operations to be performed on the test data. The test data server identifies domains associated with the selection parameters and traverses the tables in the database based on the identified domains to extract test data that satisfies the selection parameters. The test data server optionally performs transformation operations, such as masking operations, specified by the test data query on the extracted data. The identified domains are stored such that test data that satisfies the test data query may be extracted from the database repetitively without reevaluating the test data query each time.

Abstract: A method of searching all tables in a data model is disclosed, using a non-materializing virtual table interface that acts as a view into the underlying data model. The virtual table is virtually built on the fly at query execution time, and maps to all columns and rows within the data model. A query on the virtual table is translated into a set of data model queries for searching the data model, based on columns selected from the virtual table and other specified search parameters, as well as the virtual table definition. The search process works in conjunction with data domains, and uses compaction and tokenization of data.

Abstract: An apparatus, computer-readable medium, and computer-implemented method for efficiently performing operations on distinct data values, including storing a tokenized column of data in a table by mapping each unique data value in a corresponding domain to a unique entity ID, and replacing each of the data values in the column with the corresponding entity ID to generate a column of tokenized data containing one or more entity IDs, receiving a query directed to the column of data, the query defining one or more group sets for grouping the data retrieved in response to the query, and generating an entity map vector for each group set, the length of each entity map vector equal to the number of unique entity IDs for the domain, and the value of each bit in the entity map vector indicating the presence or absence of a different unique entity ID in the group set.

Abstract: A database keyword search technique that relies on a domain based storage infrastructure is disclosed. In operation, a keyword search string is processed to generate a set of search string permutations. Each string permutation specifies a different ordering of one or more portions of the search string. A domain based search process is then executed asynchronously for each string permutation. Each execution generates a search result set that identifies rows in the database that include data relevant to the string permutation. The results in each result set are scored and ranked based in part on the similarity between the string permutation and the search string provided by the user. The rankings determine which of the results are to be presented to the user.

Abstract: An apparatus, computer-readable medium, and computer-implemented method for generating a searchable archive, the method including receiving a set of tabular data comprising a plurality of rows, storing data corresponding to a group of rows in the plurality of rows in a compacted file, the compacted file comprising one or more compressed segments. The compressed segments can store data corresponding to a portion of the rows in the group of rows and can store the data corresponding to the group of rows in column-major order. The compressed segments can store one or more token values corresponding to one or more data values in the set of tabular data and the token values can be generated by dividing the set of tabular data into columns and assigning a different token to each unique data value within each of the columns.

Abstract: An apparatus, computer-readable medium, and computer-implemented method for data subsetting, including receiving a request for a subset of data from a plurality of tables, determining whether an entity graph corresponding the plurality of tables contains a cycle, and if so, performing cyclic subset processing, otherwise, expanding the entity graph and performing acyclic subset processing if the expanded entity graph does not have any cycles and cyclic subset processing if the expanded entity graph does have cycles.

Abstract: A method of searching all tables in a data model is disclosed, using a non-materializing virtual table interface that acts as a view into the underlying data model. The virtual table is virtually built on the fly at query execution time, and maps to all columns and rows within the data model. A query on the virtual table is translated into a set of data model queries for searching the data model, based on columns selected from the virtual table and other specified search parameters, as well as the virtual table definition. The search process works in conjunction with data domains, and uses compaction and tokenization of data.

Abstract: An apparatus, computer-readable medium, and computer-implemented method for generating a searchable archive, the method including receiving a set of tabular data comprising a plurality of rows, storing data corresponding to a group of rows in the plurality of rows in a compacted file, the compacted file comprising one or more compressed segments. The compressed segments can store data corresponding to a portion of the rows in the group of rows and can store the data corresponding to the group of rows in column-major order. The compressed segments can store one or more token values corresponding to one or more data values in the set of tabular data and the token values can be generated by dividing the set of tabular data into columns and assigning a different token to each unique data value within each of the columns.

Abstract: A searchable archiving system. A searchable archiving system includes one or more compacted files of archive data loosely coupled to a search process. To create a compacted file, an archiving process tokenizes the archive data, optimizes the tokenized archive data, and extracts archive metadata from the tokenized data. The tokenized data may then be compressed in a variety of ways into compressed segments that may be individually accessed and decompressed by the search agents. Before compression, segment metadata is extracted from the segments. The compressed segments and segment metadata are then combined to create a compacted file. The search process accesses the compacted files by consulting locally stored archive metadata extracted from the files during the compaction process. The search process then invokes one or more search agents that actively search the compacted files. The search agents do so by using the segment metadata to identify segments to decompress and search.

Abstract: A searchable archiving system. A searchable archiving system includes one or more compacted files of archive data loosely coupled to a search process. To create a compacted file, an archiving process tokenizes the archive data, optimizes the tokenized archive data, and extracts archive metadata from the tokenized data. The tokenized data may then be compressed in a variety of ways into compressed segments that may be individually accessed and decompressed by the search agents. Before compression, segment metadata is extracted from the segments. The compressed segments and segment metadata are then combined to create a compacted file. The search process accesses the compacted files by consulting locally stored archive metadata extracted from the files during the compaction process. The search process then invokes one or more search agents that actively search the compacted files. The search agents do so by using the segment metadata to identify segments to decompress and search.

Abstract: A searchable archiving system. A searchable archiving system includes one or more compacted files of archive data loosely coupled to a search process. To create a compacted file, an archiving process tokenizes the archive data, optimizes the tokenized archive data, and extracts archive metadata from the tokenized data. The tokenized data may then be compressed in a variety of ways into compressed segments that may be individually accessed and decompressed by the search agents. Before compression, segment metadata is extracted from the segments. The compressed segments and segment metadata are then combined to create a compacted file. The search process accesses the compacted files by consulting locally stored archive metadata extracted from the files during the compaction process. The search process then invokes one or more search agents that actively search the compacted files. The search agents do so by using the segment metadata to identify segments to decompress and search.

Abstract: A searchable archiving system. A searchable archiving system includes one or more compacted files of archive data loosely coupled to a search process. To create a compacted file, an archiving process tokenizes the archive data, optimizes the tokenized archive data, and extracts archive metadata from the tokenized data. The tokenized data may then be compressed in a variety of ways into compressed segments that may be individually accessed and decompressed by the search agents. Before compression, segment metadata is extracted from the segments. The compressed segments and segment metadata are then combined to create a compacted file. The search process accesses the compacted files by consulting locally stored archive metadata extracted from the files during the compaction process. The search process then invokes one or more search agents that actively search the compacted files. The search agents do so by using the segment metadata to identify segments to decompress and search.