Abstract: The disclosed embodiments provide a system for processing and storing data. During operation, the system obtains a set of records sorted by a first integer value, with each record containing the first integer value and one or more additional integer values. The system sets, for each integer value in each record, a delta encoding bit that indicates whether the integer value is delta encoded with respect to at least one other integer value in the records. Next, the system uses a delta encoding state represented by the delta encoding bit to calculate an encoded value from the integer value. The system then encodes a length of the encoded value in a length tag for the integer value and replaces the integer value in the record with an encoded version containing the delta encoding bit, the length tag, and the encoded value.

Abstract: A system for cluster resource allocation includes an interface and a processor. The interface is configured to receive a process and input data. The processor is configured to determine an estimate for resources required for the process to process the input data; determine existing available resources in a cluster for running the process; determine whether the existing available resources are sufficient for running the process; in the event it is determined that the existing available resources are not sufficient for running the process, indicate to add new resources; determine an allocated share of resources in the cluster for running the process; and cause execution of the process using the share of resources.

Abstract: A system for directory level atomic commits includes an interface and a processor. The interface is configured to receive an indication to provide a set of files. The processor is configured to determine whether a file in a directory has been either 1) atomically committed or 2) written by a non-atomic process and not designated as deleted and provide the file as one file of the set of files in the event that the file in the directory has been either 1) atomically committed or 2) written by a non-atomic process and not designated as deleted.

Abstract: The disclosed embodiments provide a system for processing queries of a graph database. During operation, the system executes a set of processes for processing queries of a graph database storing a graph, wherein the graph comprises a set of nodes, a set of edges between pairs of nodes in the set of nodes, and a set of predicates. When a query of the graph database is received, the system performs a lookup of a hash map to obtain one or more offsets into an edge store for the graph database, which includes a one-linkage structure and a two-linkage structure. Next, the system accesses the offset(s) in the edge store to obtain a subset of the edges matching the query. The system then uses the subset of the edges to generate a result of the query and provides the result in a response to the query.

Abstract: The disclosed embodiments provide a system for processing queries of a graph database. During operation, the system executes one or more processes for processing queries of a graph database storing a graph, wherein the graph comprises a set of nodes, a set of edges between pairs of nodes in the set of nodes, and a set of predicates. When a query of the graph database is received, the system processes the query by matching a query time of the query to a virtual time in a log-based representation of the graph database. Next, the system uses an edge store for the graph database to access a subset of the edges matching the query. The system then generates a result of the query by materializing updates to the subset of the edges before the virtual time and provides the result in a response to the query.

Abstract: The disclosed embodiments provide a system for processing queries of a graph database. During operation, the system executes one or more processes for providing the graph database storing a graph, wherein the graph includes a set of nodes, a set of edges between pairs of nodes in the set of nodes, and a set of predicates. Next, the system receives a query containing a missing value parameter, wherein the missing value parameter designates an attribute in the graph database with a nonexistent value. The system then produces, from the query, a transformed query comprising one or more parameters that do not include the missing value parameter. Finally, the system uses the transformed query and the graph database to generate a result of the query and provides the result in a response to the query.

Abstract: The present invention extends to methods, systems, and computer program products for formulating global statistics for parallel databases. In general, embodiments of the invention merge (combine) information in multiple compute node level histograms to create a global histogram for a table that is distributed across a number of compute nodes. Merging can include aligning histogram step boundaries across the compute node histograms. Merging can include aggregating histogram step-level information, such as, for example, equality rows and average range rows (or alternately equality rows, range rows, and distinct range rows), across the compute node histograms into a single global step. Merging can account for distinct values that do not appear at one or more compute nodes as well as distinct values that are counted at multiple compute nodes. A resulting global histogram can be coalesced to reduce the step count.

Abstract: The present invention extends to methods, systems, and computer program products for optimizing parallel queries using interesting distributions. For each logical operator in an SQL server MEMO, in a top down manner from a root operator to the leaf operators, interesting distributions for the operators can be identified based on the properties of the operators. Identified interesting distributions can be propagated down to lower operators by annotating the lower operators with the interesting distributions. Thus, a SQL server MEMO can be annotated with interesting distributions propagated top down from root to leaf logical operators to generate an annotated SQL server MEMO. Parallel query plans can then be generated from the annotated SQL server MEMO in a bottom up manner from leaf operators to a root operator. Annotated interesting properties can be used to prune operators, thereby facilitating a more tractable search space for a parallel query plan.

Abstract: In order to facilitate the requesting of desired information from a graph database that stores a graph, a system may translate an initial query into a query that can be executed against the graph database. In particular, using primitives, the system may translate a query associated with a type of database (such as a relational database) into the query. The graph may include nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the primitives may include: a rule based on edges in the graph that expresses a relational schema in the type of database, and information associated with a compound key that specifies a relationship between nodes, edges and predicates in the graph corresponding to a table in the type of database. Then, the system may execute the query against the graph database, and may receive a result that includes a subset of the graph.

Abstract: A system obtains desired information from a database by dynamically modifying a query plan while executing a query against the database. In particular, the system accesses predefined cardinality information associated with the query for the database (such as a number of occurrences of information associated with the query in the database), and identifies query constraints based on the predefined cardinality information. Then, the system determines an initial query plan based on the query constraints. After executing an initial query against the database based on the initial query plan, the system revises the initial query and the initial query plan, based on partial results of the initial query, to produce a revised query and a revised query plan. Next, the system executes the revised query against the database based on the revised query plan to obtain additional partial results, and the operations are repeated until a total result is obtained.

Abstract: In order to facilitate the requesting of desired information from a graph database that stores a graph, a system may translate an initial query into a query that can be executed against the graph database. In particular, using primitives, the system may translate a query associated with a type of database (such as a relational database) into the query. The graph may include nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the primitives may include: a rule based on edges in the graph that expresses a relational schema in the type of database, and information associated with a compound key that specifies a relationship between nodes, edges and predicates in the graph corresponding to a table in the type of database. Then, the system may execute the query against the graph database, and may receive a result that includes a subset of the graph.

Abstract: Distributed query processing is often performed by a set of nodes that apply MapReduce to a data set and materialize partial results to storage, which are then aggregated to produce the query result. However, this architecture requires a preconfigured set of database nodes; can only fulfill queries that utilize MapReduce processing; and may be slowed down by materializing partial results to storage. Instead, distributed query processing can be achieved by choosing a node for various portions of the query, and generating customized code for the node that only performs the query portion that is allocated to the node. The node executes the code to perform the query portion, and rather than materializing partial results to storage, streams intermediate query results to a next selected node in the distributed query. Nodes selection may be involve matching the details of the query portion with the characteristics and capabilities of the available nodes.

Abstract: In order to request desired information from a graph database, a system executes a query against the graph database. The graph database may store a graph that includes nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the query may identify a first edge associated with a predicate that specifies one or more of the nodes in the graph. In response to the query, the system receives a result that includes a subset of the graph. In particular, the subset of the graph may include the desired information expressed within an associated structure of the graph.

Abstract: In order to request desired information from a graph database, a system executes a query against the graph database. The graph database may store a graph that includes nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the query may identify a first edge associated with a predicate that specifies one or more of the nodes in the graph. In response to the query, the system receives a result that includes a subset of the graph. Then, the system may verify the subset of the graph. In particular, the system may verify that the subset of the graph includes one or more particular edges, such as an edge with the predicate in the query. Furthermore, the system may verify that the subset of the graph includes data and/or associated structure in the portion of the graph.

Abstract: In order to facilitate efficient storage and extraction of data, a system may represent a compound relationship in a graph stored in a graph database. In particular, the system (or a user) may define the compound relationship based on two or more predicates associated with two or more edges between two or more nodes in the graph. Then, the system may generate, in the graph, a hub node that corresponds to the compound relationship and that has an identifier that is hidden from users of the graph so that the identifier of the hub node is not visible external to the graph database. Moreover, the identifier of the hub node may be specified in a query using the two or more predicates, thereby allowing data associated with the compound relationship to be readily identified, modified and/or extracted.

Abstract: In order to request desired information from a graph database, a system executes a query against the graph database. The graph database may store a graph that includes nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the query may identify a first edge associated with a predicate that specifies one or more of the nodes in the graph. In response to the query, the system receives a result that includes a subset of the graph. Then, the system may use the result of the query in a concatenated set of queries. In particular, the system may execute a second query against the subset of the graph. Furthermore, in response to the second query, the system may receive a second result that includes a second subset of the graph.

Abstract: In order to facilitate the requesting of desired information from a graph database that stores a graph, a system may translate an initial query into a query that can be executed against the graph database. In particular, using primitives, the system may translate a query associated with a type of database (such as a relational database) into the query. The graph may include nodes, edges between the nodes, and predicates to represent and store data with index-free adjacency. Moreover, the primitives may include: a rule based on edges in the graph that expresses a relational schema in the type of database, and information associated with a compound key that specifies a relationship between nodes, edges and predicates in the graph corresponding to a table in the type of database. Then, the system may execute the query against the graph database, and may receive a result that includes a subset of the graph.

Abstract: Presented is a system and method for evaluating relational database queries in a distributed system. An optimized query plan is received by a control node. The query plan is decoded to a SQL statement that is semantically equivalent to the query plan, even though the query plan contains elements that have no direct analog in SQL. The decoded SQL is transmitted to a compute node for execution.

Abstract: Distributing rows of data in a distributed table distributed across a plurality of nodes. A method includes identifying skewed rows of a first table to be distributed in a distributed database system. The skewed rows include a common data value in a column such that the skewed rows are skewed, according to a predetermined skew factor, with respect to other rows in the first table not having the common data value. Non-skewed rows of the first table that are not skewed according to the skew factor are identified. The skewed rows of the first table are distributed across nodes in a non-deterministic fashion. The non-skewed rows of the first table are distributed across nodes in a deterministic fashion. The rows of the first table distributed across the nodes, whether distributed in a deterministic fashion or non-deterministic fashion, are stored in a single table at each of the nodes.

Abstract: The present invention extends to methods, systems, and computer program products for optimizing parallel queries using interesting distributions. For each logical operator in an SQL server MEMO, in a top down manner from a root operator to the leaf operators, interesting distributions for the operators can be identified based on the properties of the operators. Identified interesting distributions can be propagated down to lower operators by annotating the lower operators with the interesting distributions. Thus, a SQL server MEMO can be annotated with interesting distributions propagated top down from root to leaf logical operators to generate an annotated SQL server MEMO. Parallel query plans can then be generated from the annotated SQL server MEMO in a bottom up manner from leaf operators to a root operator. Annotated interesting properties can be used to prune operators, thereby facilitating a more tractable search space for a parallel query plan.