Abstract: A tree-based format may be implemented for data stored in a data store. A table may be maintained across one or multiple storage nodes in storage slabs. Storage slabs may be mapped to different nodes of a tree. Each node of the tree may be assigned a different range of distribution scheme values which identify what portions of the table are stored in the storage slab. Storage slabs mapped to child nodes in the tree may be assigned portions of the range of distribution scheme values assigned to a parent. Storage nodes may be added or removed for storing the table. Storage slabs may be moved from one storage node to another in order to accommodate the addition or removal of storage nodes.

Abstract: Using a graph representation of join history may be performed to distribute database data. Join history may be collected, captured, or tracked which describes the history of join operations between columns of different tables in a database. A graph representation of the join history may be generated. The graph representation may indicate a likelihood of different joins that may be performed between the columns of the tables of a database. An evaluation of the join history may be performed to identify columns for tables in the database to distribute the data of the tables amongst multiple storage locations according to the identified columns.

Abstract: The disclosed may include various systems and methods for improving the efficiency and scalability of large-scale systems. For example, the disclosed may include systems and methods for automatic privacy enforcement using privacy-aware infrastructure, scalable general-purpose low cost integer motion search, efficient scaler filter coefficients layout for flexible scaling quality control with limited hardware resources, hardware optimization for power saving with both different codecs enabled, optimizing storage overhead and performance for large distributed data warehouse, mass and volume efficient integration of intersatellite link terminals to a satellite bus, and overcoming retention limit for memory-based distributed database systems.

Abstract: A data transformation system includes a schema inference module and an export module. The schema inference module is configured to dynamically create a cumulative schema for objects retrieved from a first data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. Dynamically creating the cumulative schema includes, for each object of the retrieved objects, (i) inferring a schema from the object and (ii) selectively updating the cumulative schema to describe the object according to the inferred schema. The export module is configured to output the data of the retrieved objects to a data destination system according to the cumulative schema.

Abstract: A first column of a first database table may be transferred to a second table in a combined truncate and append operation in a transaction that may be atomic and reversible. References data for the first column may be removed from the first table and added to the second table. The combined operation may be reversed by removing, from the second table, references to blocks of data added to the second table prior to commencing the combined operation. A second column corresponding on a row-by-row basis to the first column may be added to the second table, where the data is indicative of visibility of data in the first column to other transactions on the second table.

Abstract: Data stores may implement prioritized scheduling of data store access requests. When new access requests are received, the new access requests may be scheduled for prioritized execution on processing resources. Access requests that are currently being executed with prioritized execution may be reprioritized to make additional capacity for prioritized execution of the new access requests. Prioritized execution may be automatically enabled or disabled for a data store based on monitoring of performance metrics for executing access requests.

Abstract: A method of operating a data analysis system includes retrieving objects from a data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. The method further includes dynamically creating a cumulative schema by, for each object of the retrieved objects: (i) inferring a schema from the object based on the metadata of the object and inferred data types of elements of the data of the object, (ii) creating a unified schema, wherein the unified schema describes both (a) the object described by the inferred schema and (b) a cumulative set of objects described by the cumulative schema, and (iii) storing the unified schema as the cumulative schema. The method further includes exporting the data of each of the retrieved objects to a data warehouse.

Abstract: A method of operating a query system includes retrieving objects from a data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. The method further includes dynamically creating a cumulative schema. The dynamically creating includes, for each object of the retrieved objects, (i) inferring a schema from the object based on the metadata of the object and inferred data types of elements of the data of the object, (ii) creating a unified schema, and (iii) storing the unified schema as the cumulative schema. The unified schema describes both (a) the object described by the inferred schema and (b) a cumulative set of objects described by the cumulative schema. The method further includes storing the data of each of the retrieved objects in a storage service.

Abstract: A data transformation system includes a schema inference module and an export module. The schema inference module is configured to dynamically create a cumulative schema for objects retrieved from a first data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. Dynamically creating the cumulative schema includes, for each object of the retrieved objects, (i) inferring a schema from the object and (ii) selectively updating the cumulative schema to describe the object according to the inferred schema. The export module is configured to output the data of the retrieved objects to a data destination system according to the cumulative schema.

Abstract: An embodiment provides a system and method for transaction commitment and replication. The method includes receiving a minitransaction from a client node at one or more memory nodes, wherein each memory node includes a number of replicas. The minitransaction is a type of transaction which atomically executes any combination of reading, comparing, and writing to any of a number of memory locations. The method also includes determining, for a leader of the replicas within a memory node, whether the leader is able to commit the minitransaction and stabilizing state changes of the minitransaction within a transaction log using a consensus procedure to update the replicas. The method further includes committing the minitransaction if, at each memory node, a quorum of the replicas is able to stabilize the minitransaction, or aborting the minitransaction otherwise.

Abstract: A data transformation system includes a schema inference module and an export module. The schema inference module is configured to dynamically create a cumulative schema for objects retrieved from a first data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. Dynamically creating the cumulative schema includes, for each object of the retrieved objects, (i) inferring a schema from the object and (ii) selectively updating the cumulative schema to describe the object according to the inferred schema. The export module is configured to output the data of the retrieved objects to a data destination system according to the cumulative schema.

Abstract: A shiftable memory supporting bimodal data storage includes a memory having built-in shifting capability to shift a contiguous subset of data stored in the memory from a first location to a second location within the memory. The shiftable memory further includes a bimodal data storage operator to operate on a data structure comprising the contiguous subset of data words and to provide in-place insertion of a data value using the built-in shifting capability.

Abstract: A computer cluster with objectives-based resource sharing. The cluster includes cloud nodes each with one or more resources, a terminal, data storage, and an allocation node to monitor cloud node resources, provide information descriptive of the cloud node resources to a customer through the terminal, receive a reservation for cloud node resources from the customer, store the reservation in the data storage, determine assignments of the cloud node resources for the reservation and any other pending reservations according to one or more objectives, and allocate the cloud node resources to customers according to the resource assignments.

Abstract: Computer-implemented systems and associated operating methods implement a fast join for databases which is adapted for usage with flash storage. A system comprises a processor that performs a join of two tables stored in a storage in pages processed in a column orientation wherein column values for all rows on a page are co-located in mini-pages within the page. The processor reduces input/output operations of the join by accessing only join columns and mini-pages containing join results.

Abstract: A method of assigning resources of a computer duster with resource sharing according to objectives. The method includes monitoring resources of each of a plurality of cloud nodes, providing information descriptive of the cloud node resources, receiving a reservation, determining whether resources are available to satisfy the reservation and any other pending reservations, if resources are available, using a rapid search to determine resource assignments for the reservation and any other pending reservations according to one or more objectives, and allocating resources according to the resource assignments.

Abstract: A shiftable memory supporting bimodal data storage includes a memory having built-in shifting capability to shift a contiguous subset of data stored in the memory from a first location to a second location within the memory. The shiftable memory further includes a bimodal data storage operator to operate on a data structure comprising the contiguous subset of data words and to provide in-place insertion of a data value using the built-in shifting capability.

Abstract: A method of operating a data analysis system includes retrieving objects from a data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. The method further includes dynamically creating a cumulative schema by, for each object of the retrieved objects: (i) inferring a schema from the object based on the metadata of the object and inferred data types of elements of the data of the object, (ii) creating a unified schema, wherein the unified schema describes both (a) the object described by the inferred schema and (b) a cumulative set of objects described by the cumulative schema, and (iii) storing the unified schema as the cumulative schema. The method further includes exporting the data of each of the retrieved objects to a data warehouse.

Abstract: A data transformation system includes a schema inference module and an export module. The schema inference module is configured to dynamically create a cumulative schema for objects retrieved from a first data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. Dynamically creating the cumulative schema includes, for each object of the retrieved objects, (i) inferring a schema from the object and (ii) selectively updating the cumulative schema to describe the object according to the inferred schema. The export module is configured to output the data of the retrieved objects to a data destination system according to the cumulative schema.

Abstract: A method of operating a query system includes retrieving objects from a data source. Each of the retrieved objects includes (i) data and (ii) metadata describing the data. The method further includes dynamically creating a cumulative schema. The dynamically creating includes, for each object of the retrieved objects, (i) inferring a schema from the object based on the metadata of the object and inferred data types of elements of the data of the object, (ii) creating a unified schema, and (iii) storing the unified schema as the cumulative schema. The unified schema describes both (a) the object described by the inferred schema and (b) a cumulative set of objects described by the cumulative schema. The method further includes storing the data of each of the retrieved objects in a storage service.

Abstract: A method of operating a query system includes retrieving objects from a data source, wherein each of the retrieved objects includes (i) data and (ii) metadata describing the data. The method includes dynamically creating a cumulative schema by inferring a schema from each of the retrieved objects and merging the inferred schema with the cumulative schema. The method includes storing the data of each of the retrieved objects in a storage service. The method includes receiving, from a user, a query, and responding to the query based on data stored by the storage service.