Abstract: Herein is fast and memory efficient iteration of elements of a graph that has (e.g. topological) changes. In an embodiment, a computer generates an element iterator that is based on a graph that contains many elements (e.g. vertices, edges, and their properties) and a delta log that represents modification(s) of the graph. The delta log only records changes, such that only some graph elements are modified and occur in the delta log. Thus, iteration of only some graph elements may need to retrieve data from the delta log. Based on the element iterator and the delta log, a first graph element is accessed during iteration. Based on the element iterator but not the delta log, a second graph element is accessed. For example, a result may be generated that is based on the first element and the second element that were iterated based on different respective data structures.

Abstract: Techniques are provided for updating in-memory property graphs in a fast manner, while minimizing memory consumption. A graph is represented as delta compressed sparse rows (CSR), in which its data structure stores forward edge offsets that map reverse edges to forward edges, enabling fast traversals of graph edges in forward and reverse directions. To support fast graph updates, delta logs are used to store changes to the graph. In an embodiment, a base version of the graph data structure is initially loaded or created, and subsequent versions of the graph are created from the reference to the initial graph and a delta log data structure that records the changes compared to the base version of the graph.

Abstract: Techniques herein optimally distribute graph query processing across heterogeneous tiers. In an embodiment, a computer receives a graph query to extract a query result (QR) from a graph in a database operated by a database management system (DBMS). The graph has vertices interconnected by edges. Each vertex has vertex properties, and each edge has edge properties. The computer decomposes the graph query into filter expressions (FE's). Each FE is processed as follows. A filtration tier to execute the FE is selected from: the DBMS which sends at least the QR to a stream, a stream evaluator that processes the stream as it arrives without waiting for the entire QR to arrive and that stores at least the QR into memory, and an in-memory evaluator that identifies the QR in memory. A translation of the FE executes on the filtration tier to obtain vertices and/or edges that satisfy the FE.

Abstract: Techniques are provided herein for efficient representation of heterogeneous graphs in memory. In an embodiment, vertices and edges of the graph are segregated by type. Each property of a type of vertex or edge has values stored in a respective vector. Directed or undirected edges of a same type are stored in compressed sparse row (CSR) format. The CSR format is more or less repeated for edge traversal in either forward or reverse direction. An edge map translates edge offsets obtained from traversal in the reverse direction for use with data structures that expect edge offsets in the forward direction. Subsequent filtration and/or traversal by type or property of vertex or edge entails minimal data access and maximal data locality, thereby increasing efficient use of the graph.

Abstract: Techniques are provided herein for efficient representation of heterogeneous graphs in memory. In an embodiment, vertices and edges of the graph are segregated by type. Each property of a type of vertex or edge has values stored in a respective vector. Directed or undirected edges of a same type are stored in compressed sparse row (CSR) format. The CSR format is more or less repeated for edge traversal in either forward or reverse direction. An edge map translates edge offsets obtained from traversal in the reverse direction for use with data structures that expect edge offsets in the forward direction. Subsequent filtration and/or traversal by type or property of vertex or edge entails minimal data access and maximal data locality, thereby increasing efficient use of the graph.

Abstract: Techniques herein are for navigation data structures for graph traversal. In an embodiment, navigation data structures that a computer stores include: a source vertex array of vertices; a neighbor array of dense identifiers of target vertices terminating edges; a bidirectional map associating, for each vertex, a sparse identifier of the vertex with a dense identifier of the vertex; and a vertex array containing, when a dense identifier of a source vertex is used as an offset, a pair of offsets defining an offset range, for use with the neighbor array. The source vertex array, using the dense identifier of a particular vertex as an offset, contains an offset, into a neighbor array, of a target vertex terminating an edge originating at the particular vertex. The neighbor array contiguously stores dense identifiers of target vertices terminating edges originating from a same source vertex.

Abstract: Techniques are provided for mapping tables and columns of a legacy relational schema into synthetic tables that are dedicated for graph analysis. In an embodiment, a computer receives a mapping of relational tables to node tables and edge tables. The node tables contain columns and rows. The edge tables contain columns and rows. The rows of the node tables and the rows of the edge tables define a graph. Based on the mapping and the relational tables, the computer calculates a value of at least one column of at least one row of the node tables. Based on an execution of a query of the graph, the computer returns the value.

Abstract: Techniques herein minimize memory needed to store distances between vertices of a graph for use during a multi-source breadth-first search (MS-BFS). In an embodiment, during each iteration of a first sequence of iterations of a MS-BFS, a computer updates a first matrix that contains elements that use a first primitive integer type having a first width to record a distance from a source vertex of a graph to another vertex. The computer detects that a count of iterations of the first sequence of iterations exceeds a threshold. Responsively, the computer creates a second matrix that contains elements that use a second primitive integer type having a second width that is larger than the first width to record a distance from a source vertex of the graph to another vertex. During each iteration of a second sequence of iterations of the MS-BFS, the computer updates the second matrix.

Abstract: Techniques are described herein for allocating and rebalancing computing resources for executing graph workloads in manner that increases system throughput. According to one embodiment, a method includes receiving a request to execute a graph processing workload on a dataset, identifying a plurality of graph operators that constitute the graph processing workload, and determining whether execution of each graph operator is processor intensive or memory intensive. The method also includes assigning a task weight for each graph operator of the plurality of graph operators, and performing, based on the assigned task weights, a first allocation of computing resources to execute the plurality of graph operators. Further, the method includes causing, according to the first allocation, execution of the plurality of graph operators by the computing resources, and monitoring computing resource usage of graph operators executed by the computing resources according to the first allocation.

Abstract: Techniques are described herein for concurrently evaluating graph processing tasks in a fair and efficient manner. In an embodiment, a request to execute a graph processing task is received. A first mapping associates each graph processing task of a plurality of graph processing tasks to a set of workload characteristics of a plurality of sets of workload characteristics. A second mapping associates each set of workload characteristics of the plurality of sets of workload characteristics to a set of execution parameters of a plurality of sets of execution parameters. Using the first mapping, a set of workload characteristics is determined based on the graph processing task. Using the second mapping, a set of execution parameters is determined based on the determined set of workload characteristics. The graph processing task is executed based on the determined set of execution parameters.

Abstract: Embodiments perform real-time vertex connectivity checks in graph data representations via a multi-phase search process. This process includes an efficient first search phase using landmark connectivity data that is generated during a preprocessing phase. Landmark connectivity data maps the connectivity of a set of identified landmarks in a graph to other vertices in the graph. Upon determining that the subject vertices are not closely related via landmarks, embodiments implement a second search phase that performs a brute-force search for connectivity, between the subject vertices, among the graph's non-landmark vertices. This brute-force search prevents exploration of cyclical paths by recording the vertices on a currently-explored path in a stack data structure. The second search phase is automatically aborted upon detecting that the non-landmark vertices in the graph are over a threshold density.

Abstract: Techniques are described herein for allocating and rebalancing computing resources for executing graph workloads in manner that increases system throughput. According to one embodiment, a method includes receiving a request to execute a graph processing workload on a dataset, identifying a plurality of graph operators that constitute the graph processing workload, and determining whether execution of each graph operator is processor intensive or memory intensive. The method also includes assigning a task weight for each graph operator of the plurality of graph operators, and performing, based on the assigned task weights, a first allocation of computing resources to execute the plurality of graph operators. Further, the method includes causing, according to the first allocation, execution of the plurality of graph operators by the computing resources, and monitoring computing resource usage of graph operators executed by the computing resources according to the first allocation.

Abstract: Techniques are provided for strategy-based graph simplification. In an embodiment, a computer provides configurable strategies that simplify edges of a graph. A client selects and configures a strategy subset of the configurable strategies to define a particular simplification scheme. The computer simplifies a graph by applying the strategy subset to the graph. In embodiments, predefined classes or other application programming interface (API) is provided to clients to obtain and customize strategy instances, such as with a factory or builder. Strategy instances may be imperative or declarative. A service implementation, such as a graph engine, may be embedded or remoted. Techniques herein provide for reuse and optimization.

Abstract: Embodiments perform real-time vertex connectivity checks in graph data representations via a multi-phase search process. This process includes an efficient first search phase using landmark connectivity data that is generated during a preprocessing phase. Landmark connectivity data maps the connectivity of a set of identified landmarks in a graph to other vertices in the graph. Upon determining that the subject vertices are not closely related via landmarks, embodiments implement a second search phase that performs a brute-force search for connectivity, between the subject vertices, among the graph's non-landmark vertices. This brute-force search prevents exploration of cyclical paths by recording the vertices on a currently-explored path in a stack data structure. The second search phase is automatically aborted upon detecting that the non-landmark vertices in the graph are over a threshold density.

Abstract: Techniques herein minimize memory needed to store distances between vertices of a graph for use during a multi-source breadth-first search (MS-BFS). In an embodiment, during each iteration of a first sequence of iterations of a MS-BFS, a computer updates a first matrix that contains elements that use a first primitive integer type having a first width to record a distance from a source vertex of a graph to another vertex. The computer detects that a count of iterations of the first sequence of iterations exceeds a threshold. Responsively, the computer creates a second matrix that contains elements that use a second primitive integer type having a second width that is larger than the first width to record a distance from a source vertex of the graph to another vertex. During each iteration of a second sequence of iterations of the MS-BFS, the computer updates the second matrix.

Abstract: Techniques herein generate, such as during compilation, polymorphic dispatch logic (PDL) to switch between specialized implementations of a polymorphic graph algorithm. In an embodiment, a computer detects, within source logic of a graph algorithm, that the algorithm processes an instance of a generic graph type. The computer generates several alternative implementations of the algorithm. Each implementation is specialized to process the graph instance as an instance of a respective graph subtype. The computer generates PDL that performs dynamic dispatch as follows. At runtime, the PDL receives a graph instance of the generic graph type. The PDL detects which particular graph subtype is the graph instance. The PDL then invokes whichever alternative implementation that is specialized to process the graph instance as an instance of the detected particular graph subtype. In embodiments, the source logic is expressed in a domain specific language (DSL), e.g. for analysis, traversal, or querying of graphs.

Abstract: Techniques herein minimize memory needed to store distances between vertices of a graph for use during a multi-source breadth-first search (MS-BFS). In an embodiment, during each iteration of a first sequence of iterations of a MS-BFS, a computer updates a first matrix that contains elements that use a first primitive integer type having a first width to record a distance from a source vertex of a graph to another vertex. The computer detects that a count of iterations of the first sequence of iterations exceeds a threshold. Responsively, the computer creates a second matrix that contains elements that use a second primitive integer type having a second width that is larger than the first width to record a distance from a source vertex of the graph to another vertex. During each iteration of a second sequence of iterations of the MS-BFS, the computer updates the second matrix.

Abstract: Techniques herein optimally distribute graph query processing across heterogeneous tiers. In an embodiment, a computer receives a graph query to extract a query result (QR) from a graph in a database operated by a database management system (DBMS). The graph has vertices interconnected by edges. Each vertex has vertex properties, and each edge has edge properties. The computer decomposes the graph query into filter expressions (FE's). Each FE is processed as follows. A filtration tier to execute the FE is selected from: the DBMS which sends at least the QR to a stream, a stream evaluator that processes the stream as it arrives without waiting for the entire QR to arrive and that stores at least the QR into memory, and an in-memory evaluator that identifies the QR in memory. A translation of the FE executes on the filtration tier to obtain vertices and/or edges that satisfy the FE.

Abstract: Techniques herein decouple available results, from graph analysis execution, to adapt to various deployment configurations. In an embodiment, a graph engine is deployed that has multiple mutually-exclusive configuration modes that include being embedded within a software application, centrally serving software applications, or distributed amongst a cluster of computers. Based on a current configuration mode of the graph engine, a software application receives or generates an analysis request to process a graph. The software application provides the analysis request to the graph engine in exchange for access to a computational future, of the graph engine, that is based on the analysis request and the graph. Based on a proxy of said computational future, the software application accesses a result of the analysis request. In an embodiment, a remote proxy exchanges representational state transfer (REST) messages.

Abstract: Techniques are provided herein for efficient representation of heterogeneous graphs in memory. In an embodiment, vertices and edges of the graph are segregated by type. Each property of a type of vertex or edge has values stored in a respective vector. Directed or undirected edges of a same type are stored in compressed sparse row (CSR) format. The CSR format is more or less repeated for edge traversal in either forward or reverse direction. An edge map translates edge offsets obtained from traversal in the reverse direction for use with data structures that expect edge offsets in the forward direction. Subsequent filtration and/or traversal by type or property of vertex or edge entails minimal data access and maximal data locality, thereby increasing efficient use of the graph.