Abstract: A method for dynamically building a column store database from a row store database. The method includes establishing the row store database for storing data, wherein each row includes a plurality of attributes, and wherein data in row store database is current to a temporal point in time. The method includes establishing the column store database including data structured to satisfy received analytic queries. The method includes beginning from an initial state of the column store database, for each subsequently received analytic query, importing a targeted amount of data from a corresponding temporal state of the row store database into the column store database to satisfy the corresponding subsequently received analytic query.

Abstract: A multiple disk loader apparatus includes a plurality of rods. Each rod has a pair of pins extending radially from a side of the rod. The pair of pins are spaced circumferentially around the rod with respect to each other. Each pin has a top surface wherein the top surface of a first pin is longitudinally separated from the top surface of a second pin by a predetermined gap. A rotation device is coupled to the plurality of rods. The rotation device rotates the plurality of rods individually through a respective predetermined arc and in a respective rotational direction in order to extract a disk stack from storage and to separate a disk from the stack to load into a drive.

Abstract: A method comprising receiving one or more data storage specifications from a tenant, determining that a plurality of physical storage units in a physical storage system comprises a sufficient system capacity to provision data storage for the tenant, sending a storage request message to request creation of a network storage entity for the tenant according to the data storage specifications, and receiving a storage response message indicating a first of a plurality of logical storage units allocated to the network storage entity according to the data storage specifications, wherein the plurality of logical storage units are distributed across the plurality of physical storage units, and wherein the plurality of logical storage units are arranged in a sequential order to form a logical circular buffer.

Abstract: A method implemented by a network element (NE) in a network, comprising composing a first network storage entity by mapping a plurality of logical storage units to a plurality of physical storage units in a physical storage system according to a first storage metric associated with the plurality of physical storage units, arranging the plurality of logical storage units sequentially to form a logical circular buffer, and designating a current logical storage unit for writing data and an upcoming logical storage unit for writing data after the current storage unit is fully written, and rebalancing the physical storage system while the physical storage system is actively performing network storage operations by relocating at least one of the logical storage units to a different physical storage unit according to a second storage metric associated with the plurality of physical storage units.

Abstract: A method comprising receiving a plurality of data records, storing the data records as data segments in a storage element, receiving a plurality of descriptors for each data segment, wherein each descriptor describes an aspect of data contained in the data segments, employing a first user-defined function to resolve a first minimum descriptor for each data segment and a first maximum descriptor for each data segment, composing a lightweight index for the data segments, wherein the lightweight index comprises the first minimum descriptor for each data segment and the first maximum descriptor for each data segment, and appending the lightweight index to the data segments in the storage element.

Abstract: The present invention is directed to a novel, a topic-based messaging architecture (including schema, protocols, naming conventions, etc.) to be used in a distributed data-oriented OLTP environment. According to an aspect of the claimed subject matter, the topic-based messaging architecture can be implemented as a type of publication-subscription (“pub-sub”) messaging pattern. In one or more embodiments of the topic-based system, messages are published to “topics,” or named logical channels. Subscribers in a topic-based system will receive all messages published to the topics to which they subscribe, and all subscribers to a topic will receive the same messages. The publisher is responsible for defining the classes of messages to which subscribers can subscribe. The topic-based messaging interface improves the scalability of a distributed database management system and provides a robust mechanism for message delivery.

Abstract: Method, system, and programs for dynamic control of a processing system having a plurality of tiers. Queue lengths of a plurality of nodes in one of the plurality of tiers are received. A control objective is received from a higher tier. One or more requests from the higher tier are processed by the plurality of nodes in the tier. A control model of the tier is computed based on the received queue lengths. One or more parameters of the control model are adjusted based on the received control objective. At least one control action is determined based on the control model and the control objective.

Abstract: A shingled magnetic recording (SMR) append-only file system includes a disk comprising a plurality of concentric append-only shingled data bands having partially overlapping data tracks, wherein the data bands are associated with a circular linked list having a head data band and a tail data band, the head data band and the tail data band each comprising a plurality of data blocks. The system also includes a processor configured to write data blocks to the disk, and create a new file wherein an empty data band of the plurality of data bands is removed from the circular linked list and added to a single linked list of the new file.

Abstract: A method for parallel processing of data, including accessing a sub-graph of an execution plan for a query used for accessing a database by a host processor, wherein the execution plan includes operators organized into sub-graphs. The method includes generating a transformed sub-graph by transforming operators from the sub-graph for parallel execution on discrete computing devices. The method includes adding at least one first operator to the transformed sub-graph configured for allocating memory on the discrete computing devices and copying ingress variables to the allocated memory. The method includes adding at least one second operator to the transformed sub-graph for copying egress variables, wherein the transformed sub-graph comprises a plurality of intermediate variables generated during execution of the sub-graph that are fully contained within the transformed sub-graph. The method includes off-loading the transformed sub-graph to the discrete devices for execution.

Abstract: A method for adaptively building and updating a column store database from a row store database using queries. The method includes establishing the row store database for storing data, wherein at least one row of the row store includes attributes. The method includes establishing the column store including data structured to satisfy received analytic queries, wherein the column store includes attribute vectors corresponding to attributes in the row store, wherein at least one of the attribute vectors includes data used to satisfy previously received analytic queries. The method includes receiving change transactions directed to the row store, wherein the change transactions are ordered. The method includes when a first referenced attribute referenced by an analytic query corresponds to an attribute vector in the column store, updating the attribute vector based on log information corresponding to change transactions directed to the referenced attribute to satisfy the analytic query.

Abstract: A method for dynamically building a column store database from a row store database. The method includes establishing the row store database for storing data, wherein each row includes a plurality of attributes, and wherein data in row store database is current to a temporal point in time. The method includes establishing the column store database including data structured to satisfy received analytic queries. The method includes beginning from an initial state of the column store database, for each subsequently received analytic query, importing a targeted amount of data from a corresponding temporal state of the row store database into the column store database to satisfy the corresponding subsequently received analytic query.

Abstract: A method for parallel processing of data, including accessing a sub-graph of an execution plan for a query used for accessing a database by a host processor, wherein the execution plan includes operators organized into sub-graphs. The method includes generating a transformed sub-graph by transforming operators from the sub-graph for parallel execution on discrete computing devices. The method includes adding at least one first operator to the transformed sub-graph configured for allocating memory on the discrete computing devices and copying ingress variables to the allocated memory. The method includes adding at least one second operator to the transformed sub-graph for copying egress variables, wherein the transformed sub-graph comprises a plurality of intermediate variables generated during execution of the sub-graph that are fully contained within the transformed sub-graph. The method includes off-loading the transformed sub-graph to the discrete devices for execution.

Abstract: An apparatus comprising an input/output (IO) port configured to couple to a large-scale storage device, a memory configured to store a plurality metadata databases (DBs) for a file system of the large-scale storage device, wherein the plurality of metadata DBs comprise key-value pairs with empty values, and a processor coupled to the IO port and the memory, wherein the processor is configured to partition the file system into a plurality of partitions by grouping directories in the file system by a temporal order, and index the file system by storing metadata of different partitions as keys in separate metadata DBs.

Abstract: Method, system, and programs for balancing work load in a distributed system. A plurality of resource units in the distributed system are ranked based a plurality of multi-dimensional load metrics determined from each of the plurality of resource units. The plurality of resource units are divided into a plurality of resource unit groups based on the ranking of the plurality of resource units. At least one pair of resource unit groups is formed. Each of the at least one pair of resource unit groups is assigned with a probability of load balancing action. For each pair of resource unit groups, a load balancing action is determined based on the assigned probability of load balancing action.

Abstract: A method of pipelining re-shuffled data of a distributed column oriented relational database management system (RDBMS). A request is received from a consumer process that requires RDBMS column data to be shuffled in a specific order according to an order that each of a plurality of columns will be used by the consumer process. For each of the plurality of columns, the method re-shuffles the RDBMS column data according to the specific order to form re-shuffled RDBMS column data, and sends the re-shuffled RDBMS column data to the consumer process.

Abstract: Various disclosed embodiments include methods and systems for managing lock or latch chains in concurrent execution of database queries. A method includes receiving a plurality of transactions, each transaction associated with one or more queuing requests. The method includes, for each transaction, determining one or more partition sets. Each partition set corresponds to one or more database partitions needed for the transaction. The one or more database partitions are included within a partitioned database. The method includes, for each transaction, determining one or more queues needed for the transaction and storing a bitmap representation of the one or more queues needed for the transaction. The one or more queues needed for the transaction correspond to the one or more database partitions needed for the transaction.

Abstract: An apparatus comprising a plurality of storage nodes comprising a plurality of corresponding storage disks and configured to store data in a distributed manner between the storage disks that achieves a Redundant Array of Independent Disks-0 (RAID0) like performance based on positioning information and without indexing the distributed data. A network component comprising a storage disk configured to maintain a plurality of physical files for different user data that are mapped to different volumes, wherein the volumes are distributed between the storage disk and a second storage disk based on a RAID0 like data distribution scheme without being indexed.

Abstract: A method that comprises receiving a logical execution plan for a database query corresponding to a plurality of tables of the database, wherein the logical execution plan comprises one or more operators, receiving an operator cost for each of the operators in the logical execution plan, computing a first accumulated processing cost for a first of the tables based on the logical execution plan, operator selectivity, and operator costs corresponding to the first table, computing a second accumulated processing cost for a second of the tables based on the logical execution plan, operator selectivity, and operator costs corresponding to the second table, comparing the first accumulated processing cost and the second accumulated processing cost to determine a table with the highest accumulated processing cost, and responsive to comparing the accumulated processing costs, computing a physical execution plan that requires partitioning the table with the highest accumulated processing cost.

Abstract: A ProbeSystem for one or more Replicas is disclosed. For each replica, the Probe System includes a ProbeHead that updates a Head field in a Target Record pertaining to the Replica and a ProbeEcho that updates an Echo field in the TargetRecord. The Head field includes a time stamp and an epoch value. The Echo field includes a latency value and an epoch value. The fields are such that, during reporting, the ProbeHead can analyze the TargetRecord for the replica to determine the latency between one or more paths between the Replica and other Replicas. The ProbeSystem has a peer-to-peer shared nothing architecture and applies to a large class of high-replica, distributed and high-scale cloud storage systems and services.

Abstract: Method, system, and programs for resolving data inconsistency in a distributed system having a plurality of replica instances. In response to a read request of at least one data element, a key-specific probability value for the at least one data element of the read request is retrieved. Based on the retrieved key-specific probability value and a system load of the distributed system, a self-healing probability of the at least one data element of the read request is computed. A self-healing transaction that includes the at least one data element of the read request is then generated based on the self-healing probability of the at least one data element. The generated self-healing transaction is further applied to at least some of the plurality of replica instances in the distributed system to resolve data inconsistency.