Abstract: Systems and techniques are provided for flexible data ingestion. Data including a file including a database table may be received at a computing device. The file may be in a non-standard binary format. The data including the file may be stored unaltered as a source data chunk. A processed data chunk may be generated from the source data chunk by converting the file to a standard binary format and storing the file in the processed data chunk without altering the source data chunk. A materialized data chunk may be generated from the processed data chunk by performing, with a database server engine of the computing device, a database operation on the database table of the file of the processed data chunk and storing the file in the materialized data chunk without altering the processed data chunk. The database table of the file of the materialized data chunk may be made available for querying by the database server engine.

Abstract: Systems and techniques are provided for database table annotation processing. An annotation processing engine on a computing device may receive an instruction to build a database table. The annotation processing engine on the computing device may receive annotations made to the database table from a storage of the computing device. The annotations may include labels identifying features of the database table. The annotation processing engine on the computing device may determine a database server engine compatible with the database table. The annotation processing engine on the computing device may generate a query in a native language of the database server engine compatible with the database table based on the instruction to build the database table and the annotations made to the database table. The annotation processing engine on the computing device may send the query in the native language of the database server engine to the database server engine.

Abstract: Systems and techniques are provided for mutable data ingestion and storage. Data for a database may be received at a computing device. New data from the data for the database may be stored as a mutable data chunk. The data for the database may be determined to include updated data for a previously stored mutable data chunk. The updated data may be applied to the previously stored mutable data chunk to update the previously stored mutable chunk. A second previously stored mutable data chunk may be determined to have been initially stored a specified amount of time ago. In response to determining that the second previously stored mutable data chunk was initially stored the specified amount of time ago, the second previously stored mutable data chunk may be caused to become an immutable data chunk.

Abstract: Systems and techniques are provided for scoring parameter generation for identity resolution. A second data set that has a statistical profile that matches a statistical profile of a third data set may be generated from a first data set. The first data set, the second data set, and the third data set may include records that comprise data about individuals. Identity resolution scores generated for the first data set may be matched to pairs of records from the second data set. New scoring parameters may be generated from the second data set and the identity resolution scores matched to the pairs of records from the second data set using regression. Identity resolution scores for pairs of records from the third data set may be generated using the new scoring parameters.

Abstract: Systems and techniques are provided for mutable data ingestion and storage. Data for a database may be received at a computing device. New data from the data for the database may be stored as a mutable data chunk. The data for the database may be determined to include updated data for a previously stored mutable data chunk. The updated data may be applied to the previously stored mutable data chunk to update the previously stored mutable chunk. A second previously stored mutable data chunk may be determined to have been initially stored a specified amount of time ago. In response to determining that the second previously stored mutable data chunk was initially stored the specified amount of time ago, the second previously stored mutable data chunk may be caused to become an immutable data chunk.