Abstract: Herein are machine learning (ML) feature processing and analytic techniques to detect anomalies in parse trees of logic statements, database queries, logic scripts, compilation units of general-purpose programing language, extensible markup language (XML), JAVASCRIPT object notation (JSON), and document object models (DOM). In an embodiment, a computer identifies an operational trace that contains multiple parse trees. Values of explicit features are generated from a single respective parse tree of the multiple parse trees of the operational trace. Values of implicit features are generated from more than one respective parse tree of the multiple parse trees of the operational trace. The explicit and implicit features are stored into a same feature vector. With the feature vector as input, an ML model detects whether or not the operational trace is anomalous, based on the explicit features of each parse tree of the operational trace and the implicit features of multiple parse trees of the operational trace.

Abstract: Techniques are provided for dynamically self-balancing communication and computation. In an embodiment, each partition of application data is stored on a respective computer of a cluster. The application is divided into distributed jobs, each of which corresponds to a partition. Each distributed job is hosted on the computer that hosts the corresponding data partition. Each computer divides its distributed job into computation tasks. Each computer has a pool of threads that execute the computation tasks. During execution, one computer receives a data access request from another computer. The data access request is executed by a thread of the pool. Threads of the pool are bimodal and may be repurposed between communication and computation, depending on workload. Each computer individually detects completion of its computation tasks. Each computer informs a central computer that its distributed job has finished. The central computer detects when all distributed jobs of the application have terminated.

Abstract: Techniques are provided for vectorizing Heapsort. A K-heap is used as the underlying data structure for indexing values being sorted. The K-heap is vectorized by storing values in a contiguous memory array containing a beginning-most side and end-most side. The vectorized Heapsort utilizes horizontal aggregation SIMD instructions for comparisons, shuffling, and moving data. Thus, the number of comparisons required in order to find the maximum or minimum key value within a single node of the K-heap is reduced resulting in faster retrieval operations.

Abstract: According to an embodiment, a method includes converting a knowledge base into a graph. In this embodiment, the knowledge base contains a plurality of entities and specifies a plurality of relationships among the plurality of entities, and entities in the knowledge base correspond to vertices in the graph, and relationships between entities in the knowledge base correspond to edges between vertices in the graph. The method may also include extracting a plurality of vertex embeddings from the graph. An example vertex embedding of the plurality of vertex embeddings represents, for a particular vertex, a proximity of the particular vertex to other vertices of the graph. Further, the method may include performing, based at least in part on the plurality of vertex embeddings, entity linking between input text and the knowledge base.

Abstract: Database techniques are provided that use state machines to manage polyglot subroutine bindings for database commands. In an embodiment, a computer receives a database command that contains call sites (CSs). Each CS is associated with a user defined logic (UDL). The computer associates an initial operational state with each of the CSs. During a first invocation of a particular CS, the CS becomes initialized and transitions to an optimized state that is configured for streamlined invocation of the UDL. The UDL is invoked to contribute data to a partial result for the database command. Eventually, command execution stalls and causes the CS to transition to an unready state, which entails releasing shared resources. Later execution resumes and during another invocation of the CS, resources are reacquired, the CS is made ready and transitioned back to the optimized state. The CS may again be repeatedly invoked while revisiting the optimized state.

Abstract: Techniques are provided for mapping tables and columns of a legacy relational schema into synthetic tables that are dedicated for graph analysis. In an embodiment, a computer receives a mapping of relational tables to node tables and edge tables. The node tables contain columns and rows. The edge tables contain columns and rows. The rows of the node tables and the rows of the edge tables define a graph. Based on the mapping and the relational tables, the computer calculates a value of at least one column of at least one row of the node tables. Based on an execution of a query of the graph, the computer returns the value.

Abstract: Techniques herein minimize memory needed to store distances between vertices of a graph for use during a multi-source breadth-first search (MS-BFS). In an embodiment, during each iteration of a first sequence of iterations of a MS-BFS, a computer updates a first matrix that contains elements that use a first primitive integer type having a first width to record a distance from a source vertex of a graph to another vertex. The computer detects that a count of iterations of the first sequence of iterations exceeds a threshold. Responsively, the computer creates a second matrix that contains elements that use a second primitive integer type having a second width that is larger than the first width to record a distance from a source vertex of the graph to another vertex. During each iteration of a second sequence of iterations of the MS-BFS, the computer updates the second matrix.

Abstract: Techniques are described for generating and re-using reachability graphs for efficient execution of queries. In an embodiment, a query is received for execution on a data graph. Such a query may include one or more expressions for edges in the data graph, which when executed select one or more paths in the data graph to generate results for the query. The system uses a repository to store reachability graphs and may determine whether a reachability graph for an expression of the query for the data graph is stored in a repository. Such a reachability graph is generated by applying the expression on the data graph to qualify or disqualify the edges in the data graph to be included as part of edges of the reachability graph. For example, an edge in a reachability graph exists between two vertices when at least one edge of the data graph has qualified between two vertices of the data graph that correspond to the two vertices of the reachability graph.

Abstract: Herein are techniques that extend a software system to embed new guest programing languages (GPLs) that interoperate in a transparent, modular, and configurable way. In embodiments, a computer inserts an implementation of a GPL into a deployment of the system. A command registers the GPL, define subroutines for the GPL, generates a guest virtual environment, and adds a binding of a dependency to a guest module. In an embodiment, a native programing language invokes a guest programing language to cause importing intra- or inter-language dependencies. An embodiment defines a guest object that is implemented in a first GPL and accessed from a second GPL. In an embodiment, dependencies are retrieved from a virtual file system having several alternative implementation mechanisms that include: an archive file or an actual file system, and a memory buffer or a column of a database table.

Abstract: Herein are techniques that extend a software system to embed new guest programing languages (GPLs) that interoperate in a transparent, modular, and configurable way. In embodiments, a computer inserts an implementation of a GPL into a deployment of the system. A command registers the GPL, define subroutines for the GPL, generates a guest virtual environment, and adds a binding of a dependency to a guest module. In an embodiment, a native programing language invokes a guest programing language to cause importing intra- or inter-language dependencies. An embodiment defines a guest object that is implemented in a first GPL and accessed from a second GPL. In an embodiment, dependencies are retrieved from a virtual file system having several alternative implementation mechanisms that include: an archive file or an actual file system, and a memory buffer or a column of a database table.

Abstract: Herein are techniques that extend a software system to embed new guest programing languages (GPLs) that interoperate in a transparent, modular, and configurable way. In embodiments, a computer inserts an implementation of a GPL into a deployment of the system. A command registers the GPL, define subroutines for the GPL, generates a guest virtual environment, and adds a binding of a dependency to a guest module. In an embodiment, a native programing language invokes a guest programing language to cause importing intra- or inter-language dependencies. An embodiment defines a guest object that is implemented in a first GPL and accessed from a second GPL. In an embodiment, dependencies are retrieved from a virtual file system having several alternative implementation mechanisms that include: an archive file or an actual file system, and a memory buffer or a column of a database table.

Abstract: The present invention relates to execution optimization of database queries. Herein are techniques for optimal execution based on query interpretation by translation to a domain specific language (DSL), with optimizations such as partial evaluation, abstract syntax tree (AST) rewriting, just in time (JIT) compilation, dynamic profiling, speculative logic, and Futamura projection. In an embodiment, a database management system (DBMS) that is hosted on a computer generates a query tree that represents a database query that contains an expression that is represented by a subtree of the query tree. The DBMS generates a sequence of DSL instructions that represents the subtree. The sequence of DSL instructions is executed to evaluate the expression during execution of the database query. In an embodiment, an AST is generated from the sequence of DSL instructions. In an embodiment, the DSL AST is optimally rewritten based on a runtime feedback loop that includes dynamic profiling information.

Abstract: Techniques herein minimally communicate between computers to repartition a graph. In embodiments, each computer receives a partition of edges and vertices of the graph. For each of its edges or vertices, each computer stores an intermediate representation into an edge table (ET) or vertex table. Different edges of a vertex may be loaded by different computers, which may cause a conflict. Each computer announces that a vertex resides on the computer to a respective tracking computer. Each tracking computer makes assignments of vertices to computers and publicizes those assignments. Each computer that loaded conflicted vertices transfers those vertices to computers of the respective assignments. Each computer stores a materialized representation of a partition based on: the ET and vertex table of the computer, and the vertices and edges that were transferred to the computer. Edges stored in the materialized representation are stored differently than edges stored in the ET.

Abstract: Techniques are described herein for performing named entity disambiguation. According to an embodiment, a method includes receiving input text, extracting a first mention and a second mention from the input text, and selecting, from a knowledge graph, a plurality of first candidate vertices for the first mention and a plurality of second candidate vertices for the second mention. The present method also includes evaluating a score function that analyzes vertex embedding similarity between the plurality of first candidate vertices and the plurality of second candidate vertices. In response to evaluating and seeking to optimize the score function, the method performs selecting a first selected candidate vertex from the plurality of first candidate vertices and a second selected candidate vertex from the plurality of second candidate vertices.

Abstract: Techniques herein optimize subgraph pattern matching. A computer receives a graph vertex array and a graph edge array. Each vertex and each edge has labels. The computer stores an array of index entries and an array of edge label sets. Each index entry corresponds to a respective vertex originating an edge and associates an offset of the edge with an offset of the respective vertex. Each edge label set contains labels of a respective edge. The computer selects a candidate subset of edges originating at a current vertex. The edge labels of each candidate edge of the candidate subset include a same particular query edge labels. The computer selects the candidate subset based on the index array and afterwards selects a result subset of vertices from among the terminating vertices of the candidate edges. The labels of each vertex of the result subset include a same particular query vertex labels.

Abstract: Techniques are described herein for allocating and rebalancing computing resources for executing graph workloads in manner that increases system throughput. According to one embodiment, a method includes receiving a request to execute a graph processing workload on a dataset, identifying a plurality of graph operators that constitute the graph processing workload, and determining whether execution of each graph operator is processor intensive or memory intensive. The method also includes assigning a task weight for each graph operator of the plurality of graph operators, and performing, based on the assigned task weights, a first allocation of computing resources to execute the plurality of graph operators. Further, the method includes causing, according to the first allocation, execution of the plurality of graph operators by the computing resources, and monitoring computing resource usage of graph operators executed by the computing resources according to the first allocation.

Abstract: Techniques are described herein for concurrently evaluating graph processing tasks in a fair and efficient manner. In an embodiment, a request to execute a graph processing task is received. A first mapping associates each graph processing task of a plurality of graph processing tasks to a set of workload characteristics of a plurality of sets of workload characteristics. A second mapping associates each set of workload characteristics of the plurality of sets of workload characteristics to a set of execution parameters of a plurality of sets of execution parameters. Using the first mapping, a set of workload characteristics is determined based on the graph processing task. Using the second mapping, a set of execution parameters is determined based on the determined set of workload characteristics. The graph processing task is executed based on the determined set of execution parameters.

Abstract: Techniques herein minimally communicate between computers to repartition a graph. In embodiments, each computer receives a partition of edges and vertices of the graph. For each of its edges or vertices, each computer stores an intermediate representation into an edge table (ET) or vertex table. Different edges of a vertex may be loaded by different computers, which may cause a conflict. Each computer announces that a vertex resides on the computer to a respective tracking computer. Each tracking computer makes assignments of vertices to computers and publicizes those assignments. Each computer that loaded conflicted vertices transfers those vertices to computers of the respective assignments. Each computer stores a materialized representation of a partition based on: the ET and vertex table of the computer, and the vertices and edges that were transferred to the computer. Edges stored in the materialized representation are stored differently than edges stored in the ET.

Abstract: Embodiments perform real-time vertex connectivity checks in graph data representations via a multi-phase search process. This process includes an efficient first search phase using landmark connectivity data that is generated during a preprocessing phase. Landmark connectivity data maps the connectivity of a set of identified landmarks in a graph to other vertices in the graph. Upon determining that the subject vertices are not closely related via landmarks, embodiments implement a second search phase that performs a brute-force search for connectivity, between the subject vertices, among the graph's non-landmark vertices. This brute-force search prevents exploration of cyclical paths by recording the vertices on a currently-explored path in a stack data structure. The second search phase is automatically aborted upon detecting that the non-landmark vertices in the graph are over a threshold density.

Abstract: Techniques are provided for obtaining generic vectorized d-heaps for any data type for which horizontal aggregation SIMD instructions are not available, including primitive as well as complex data types. A generic vectorized d-heap comprises a prefix heap and a plurality of suffix heaps. Each suffix heap of the plurality of suffix heaps comprises a d-heap. A plurality of key values stored in the heap are split into key prefix values and key suffix values. Key prefix values are stored in the prefix heap and key suffix values are stored in the plurality of suffix heaps. Each entry in the prefix heap includes a key prefix value of the plurality of key values and a reference to the suffix heap of the plurality of suffix heaps that includes all key suffix values of the plurality of key values that share the respective key prefix value.