Abstract: Systems and techniques are provided for query tree labeling and processing. A query may be received. A query tree may be generated from the query and identifiers may be generated for nodes of the query tree and stored in the nodes. Labels may be stored in nodes determined to meet at least one criteria for being labeled nodes to generate a labeled query tree. The nodes of the labeled query tree may be processed by receiving results from an entry in a results data structure and returning the results from the entry as the results for the node if the identifier of the nodes matches the identifier of an entry in the results data structure. If there is no match, results may be computed for the node. The results generated by computing the node may be stored in the results data structure if the node has a label.

Abstract: Systems and techniques are provided for flexible data ingestion. Data including a file including a database table may be received at a computing device. The file may be in a non-standard binary format. The data including the file may be stored unaltered as a source data chunk. A processed data chunk may be generated from the source data chunk by converting the file to a standard binary format and storing the file in the processed data chunk without altering the source data chunk. A materialized data chunk may be generated from the processed data chunk by performing, with a database server engine of the computing device, a database operation on the database table of the file of the processed data chunk and storing the file in the materialized data chunk without altering the processed data chunk. The database table of the file of the materialized data chunk may be made available for querying by the database server engine.

Abstract: Systems and techniques are provided for database table annotation processing. An annotation processing engine on a computing device may receive an instruction to build a database table. The annotation processing engine on the computing device may receive annotations made to the database table from a storage of the computing device. The annotations may include labels identifying features of the database table. The annotation processing engine on the computing device may determine a database server engine compatible with the database table. The annotation processing engine on the computing device may generate a query in a native language of the database server engine compatible with the database table based on the instruction to build the database table and the annotations made to the database table. The annotation processing engine on the computing device may send the query in the native language of the database server engine to the database server engine.

Abstract: Systems and techniques are provided for mutable data ingestion and storage. Data for a database may be received at a computing device. New data from the data for the database may be stored as a mutable data chunk. The data for the database may be determined to include updated data for a previously stored mutable data chunk. The updated data may be applied to the previously stored mutable data chunk to update the previously stored mutable chunk. A second previously stored mutable data chunk may be determined to have been initially stored a specified amount of time ago. In response to determining that the second previously stored mutable data chunk was initially stored the specified amount of time ago, the second previously stored mutable data chunk may be caused to become an immutable data chunk.

Abstract: Systems and techniques are provided for scoring parameter generation for identity resolution. A second data set that has a statistical profile that matches a statistical profile of a third data set may be generated from a first data set. The first data set, the second data set, and the third data set may include records that comprise data about individuals. Identity resolution scores generated for the first data set may be matched to pairs of records from the second data set. New scoring parameters may be generated from the second data set and the identity resolution scores matched to the pairs of records from the second data set using regression. Identity resolution scores for pairs of records from the third data set may be generated using the new scoring parameters.

Abstract: Systems and techniques are provided for query tree labeling and processing. A query may be received. A query tree may be generated from the query and identifiers may be generated for nodes of the query tree and stored in the nodes. Labels may be stored in nodes determined to meet at least one criteria for being labeled nodes to generate a labeled query tree. The nodes of the labeled query tree may be processed by receiving results from an entry in a results data structure and returning the results from the entry as the results for the node if the identifier of the nodes matches the identifier of an entry in the results data structure. If there is no match, results may be computed for the node. The results generated by computing the node may be stored in the results data structure if the node has a label.

Abstract: Systems and techniques are provided for mutable data ingestion and storage. Data for a database may be received at a computing device. New data from the data for the database may be stored as a mutable data chunk. The data for the database may be determined to include updated data for a previously stored mutable data chunk. The updated data may be applied to the previously stored mutable data chunk to update the previously stored mutable chunk. A second previously stored mutable data chunk may be determined to have been initially stored a specified amount of time ago. In response to determining that the second previously stored mutable data chunk was initially stored the specified amount of time ago, the second previously stored mutable data chunk may be caused to become an immutable data chunk.

Abstract: Systems and techniques are provided for flexible data ingestion. Data including a file including a database table may be received at a computing device. The file may be in a non-standard binary format. The data including the file may be stored unaltered as a source data chunk. A processed data chunk may be generated from the source data chunk by converting the file to a standard binary format and storing the file in the processed data chunk without altering the source data chunk. A materialized data chunk may be generated from the processed data chunk by performing, with a database server engine of the computing device, a database operation on the database table of the file of the processed data chunk and storing the file in the materialized data chunk without altering the processed data chunk. The database table of the file of the materialized data chunk may be made available for querying by the database server engine.

Abstract: Systems and techniques are provided for database table annotation processing. An annotation processing engine on a computing device may receive an instruction to build a database table. The annotation processing engine on the computing device may receive annotations made to the database table from a storage of the computing device. The annotations may include labels identifying features of the database table. The annotation processing engine on the computing device may determine a database server engine compatible with the database table. The annotation processing engine on the computing device may generate a query in a native language of the database server engine compatible with the database table based on the instruction to build the database table and the annotations made to the database table. The annotation processing engine on the computing device may send the query in the native language of the database server engine to the database server engine.

Abstract: Systems and techniques are provided for mutable data ingestion and storage. Data for a database may be received at a computing device. New data from the data for the database may be stored as a mutable data chunk. The data for the database may be determined to include updated data for a previously stored mutable data chunk. The updated data may be applied to the previously stored mutable data chunk to update the previously stored mutable chunk. A second previously stored mutable data chunk may be determined to have been initially stored a specified amount of time ago. In response to determining that the second previously stored mutable data chunk was initially stored the specified amount of time ago, the second previously stored mutable data chunk may be caused to become an immutable data chunk.

Abstract: Tools and techniques are described herein for checkpointing iterators during search. These tools may provide methods that include instantiating iterators in response to a search request. The iterators include fixed state information that remains constant over a life of the iterator, and further include dynamic state information that is updated over the life of the iterator. The iterators traverse through postings lists in connection with performing the search request. As the iterators traverse the posting lists, the iterators may update their dynamic state information. The iterators may then evaluate whether to create checkpoints, with the checkpoints including representations of the dynamic state information.

Abstract: Tools and techniques are described herein for checkpointing iterators during search. These tools may provide methods that include instantiating iterators in response to a search request. The iterators include fixed state information that remains constant over a life of the iterator, and further include dynamic state information that is updated over the life of the iterator. The iterators traverse through postings lists in connection with performing the search request. As the iterators traverse the posting lists, the iterators may update their dynamic state information. The iterators may then evaluate whether to create checkpoints, with the checkpoints including representations of the dynamic state information.

Abstract: Tools and techniques are described herein for checkpointing iterators during search. These tools may provide methods that include instantiating iterators in response to a search request. The iterators include fixed state information that remains constant over a life of the iterator, and further include dynamic state information that is updated over the life of the iterator. The iterators traverse through postings lists in connection with performing the search request. As the iterators traverse the posting lists, the iterators may update their dynamic state information. The iterators may then evaluate whether to create checkpoints, with the checkpoints including representations of the dynamic state information.

Abstract: Tools and techniques are described herein for checkpointing iterators during search. These tools may provide methods that include instantiating iterators in response to a search request. The iterators include fixed state information that remains constant over a life of the iterator, and further include dynamic state information that is updated over the life of the iterator. The iterators traverse through postings lists in connection with performing the search request. As the iterators traverse the posting lists, the iterators may update their dynamic state information. The iterators may then evaluate whether to create checkpoints, with the checkpoints including representations of the dynamic state information.