Abstract: An apparatus, computer-readable medium, and computer-implemented method for efficiently performing operations on distinct data values, including receiving a query directed to a column of data, the query defining one or more group sets for grouping the data retrieved in response to the query, and for each of the one or more group sets, generating one or more entity map vectors, the length of each entity map vector being equal to the number of unique data values in a domain which corresponds to the column of data, the position of each bit in the entity map vector corresponding to the lexical position of a corresponding unique data value in a lexically ordered list of the unique data values, and the value of each bit in the entity map vector indicating the presence or absence of the corresponding unique data value in the group set.

Abstract: An apparatus, computer-readable medium, and computer-implemented method for efficiently performing operations on distinct data values, including receiving a query directed to a column of data, the query defining one or more group sets for grouping the data retrieved in response to the query, and for each of the one or more group sets, generating one or more entity map vectors, the length of each entity map vector being equal to the number of unique data values in a domain which corresponds to the column of data, the position of each bit in the entity map vector corresponding to the lexical position of a corresponding unique data value in a lexically ordered list of the unique data values, and the value of each bit in the entity map vector indicating the presence or absence of the corresponding unique data value in the group set.

Abstract: An apparatus, computer-readable medium, and computer-implemented method for efficiently performing operations on distinct data values, including storing a tokenized column of data in a table by mapping each unique data value in a corresponding domain to a unique entity ID, and replacing each of the data values in the column with the corresponding entity ID to generate a column of tokenized data containing one or more entity IDs, receiving a query directed to the column of data, the query defining one or more group sets for grouping the data retrieved in response to the query, and generating an entity map vector for each group set, the length of each entity map vector equal to the number of unique entity IDs for the domain, and the value of each bit in the entity map vector indicating the presence or absence of a different unique entity ID in the group set.

Abstract: An apparatus, computer-readable medium, and computer-implemented method for efficiently performing operations on distinct data values, including storing a tokenized column of data in a table by mapping each unique data value in a corresponding domain to a unique entity ID, and replacing each of the data values in the column with the corresponding entity ID to generate a column of tokenized data containing one or more entity IDs, receiving a query directed to the column of data, the query defining one or more group sets for grouping the data retrieved in response to the query, and generating an entity map vector for each group set, the length of each entity map vector equal to the number of unique entity IDs for the domain, and the value of each bit in the entity map vector indicating the presence or absence of a different unique entity ID in the group set.

Abstract: An apparatus, computer-readable medium, and computer-implemented method for generating a searchable archive, the method including receiving a set of tabular data comprising a plurality of rows, storing data corresponding to a group of rows in the plurality of rows in a compacted file, the compacted file comprising one or more compressed segments. The compressed segments can store data corresponding to a portion of the rows in the group of rows and can store the data corresponding to the group of rows in column-major order. The compressed segments can store one or more token values corresponding to one or more data values in the set of tabular data and the token values can be generated by dividing the set of tabular data into columns and assigning a different token to each unique data value within each of the columns.

Abstract: An apparatus, computer-readable medium, and computer-implemented method for generating a searchable archive, the method including receiving a set of tabular data comprising a plurality of rows, storing data corresponding to a group of rows in the plurality of rows in a compacted file, the compacted file comprising one or more compressed segments. The compressed segments can store data corresponding to a portion of the rows in the group of rows and can store the data corresponding to the group of rows in column-major order. The compressed segments can store one or more token values corresponding to one or more data values in the set of tabular data and the token values can be generated by dividing the set of tabular data into columns and assigning a different token to each unique data value within each of the columns.

Abstract: A searchable archiving system. A searchable archiving system includes one or more compacted files of archive data loosely coupled to a search process. To create a compacted file, an archiving process tokenizes the archive data, optimizes the tokenized archive data, and extracts archive metadata from the tokenized data. The tokenized data may then be compressed in a variety of ways into compressed segments that may be individually accessed and decompressed by the search agents. Before compression, segment metadata is extracted from the segments. The compressed segments and segment metadata are then combined to create a compacted file. The search process accesses the compacted files by consulting locally stored archive metadata extracted from the files during the compaction process. The search process then invokes one or more search agents that actively search the compacted files. The search agents do so by using the segment metadata to identify segments to decompress and search.

Abstract: A searchable archiving system. A searchable archiving system includes one or more compacted files of archive data loosely coupled to a search process. To create a compacted file, an archiving process tokenizes the archive data, optimizes the tokenized archive data, and extracts archive metadata from the tokenized data. The tokenized data may then be compressed in a variety of ways into compressed segments that may be individually accessed and decompressed by the search agents. Before compression, segment metadata is extracted from the segments. The compressed segments and segment metadata are then combined to create a compacted file. The search process accesses the compacted files by consulting locally stored archive metadata extracted from the files during the compaction process. The search process then invokes one or more search agents that actively search the compacted files. The search agents do so by using the segment metadata to identify segments to decompress and search.

Abstract: A searchable archiving system. A searchable archiving system includes one or more compacted files of archive data loosely coupled to a search process. To create a compacted file, an archiving process tokenizes the archive data, optimizes the tokenized archive data, and extracts archive metadata from the tokenized data. The tokenized data may then be compressed in a variety of ways into compressed segments that may be individually accessed and decompressed by the search agents. Before compression, segment metadata is extracted from the segments. The compressed segments and segment metadata are then combined to create a compacted file. The search process accesses the compacted files by consulting locally stored archive metadata extracted from the files during the compaction process. The search process then invokes one or more search agents that actively search the compacted files. The search agents do so by using the segment metadata to identify segments to decompress and search.

Abstract: A searchable archiving system. A searchable archiving system includes one or more compacted files of archive data loosely coupled to a search process. To create a compacted file, an archiving process tokenizes the archive data, optimizes the tokenized archive data, and extracts archive metadata from the tokenized data. The tokenized data may then be compressed in a variety of ways into compressed segments that may be individually accessed and decompressed by the search agents. Before compression, segment metadata is extracted from the segments. The compressed segments and segment metadata are then combined to create a compacted file. The search process accesses the compacted files by consulting locally stored archive metadata extracted from the files during the compaction process. The search process then invokes one or more search agents that actively search the compacted files. The search agents do so by using the segment metadata to identify segments to decompress and search.