Abstract: A query generation and processing system includes a relational data store, a query generator, and a query processor. The relational data store stores data ingested from data sources in a first and second datasets. The query generator interprets a data expression in a simplified query language to generate a query in a structured query language based on identifying quads corresponding to the first and second datasets in the data expression and determining an implicit join between the quads based on an unambiguous relationship obtainable from a schema of the first and datasets, in which the data expression does not expressly identify a join between the first quad and the second quad. The query processor generates a query pipeline that uses the data of the first and second datasets stored by the relational data store to execute the query generated by the query processor.

Abstract: Data expressions in a simplified query language are processed to generate queries in a structured query language which can then be executed against data ingested from one or more data sources. The data expression is parsed to determine quads and to produce a tree of the quads. A derivation graph including nodes representing the quads and including at least one edge representing a derivation relationship between two of the quads determined based on attributes of the quads is generated based on the tree of quads and a data schema. The derivation graph is then queried based on a grain of the quads to generate the query. The simplified query language does not require an expression of a join relationship between the quads within the data expression when an unambiguous relationship between the quads is obtainable from the data schema.

Abstract: Data is ingested from one or more data sources directly into a low-latency memory buffer. In response to ingesting the data, the ingested data is accessed within the low-latency memory buffer to execute a query without requiring creation of a copy of the ingested data and thus without first writing the ingested data to a warm or cold storage. At some point subsequent to executing the query, the ingested data may be purged from the low-latency memory buffer, such as based on a recency of use of a dataset corresponding to the ingested data for query execution. The purging of the ingested data moves the ingested data to a warm or cold storage and clears space in the low-latency memory buffer for later ingested data to be accessed directly within the memory buffer for query execution also without requiring creation of a copy thereof.

Abstract: Data corresponding to first and second datasets are stored within a low-latency buffer. A first query is executed by computing a join between the first and second datasets to produce a first output using the data stored in the low-latency buffer. Following execution of the first query, data corresponding to the first dataset is maintained in the low-latency buffer and data corresponding to the second dataset is purged from the low-latency buffer based on a determination that the first dataset is a static dataset and a determination that the second dataset is not a static dataset. A second query is then executed using the first dataset to produce a second output while the data corresponding to the first dataset is maintained in the low-latency buffer. The second query may be the same as or different from the first query.

Abstract: A derivation graph including nodes representing quads identified within a data expression in a simplified query language is queried using deferred join processing. A derivation graph is generated based on a first data expression that includes a join between a second data expression and a third data expression, in which the derivation graph includes at least one node representative of the second data expression and at least one node representative of the third data expression. A root node is identified within the derivation graph by determining that the nodes representative of the second data expression and the third data expression are derivable from the root node using the derivation graph. Query language instructions representing the join between the second data expression and the third data expression written in a second query language are then generated using the root node.

Abstract: Improved systems and methods for database analysis are described herein. A method includes generating a graph-based ontological data structure including nodes connected by edges in a low-latency database analysis system, wherein a respective node represents an object in the low-latency database analysis system, and wherein the respective node comprises a body comprising content of the respective object and a header comprising information about the object, receiving a modification request to the graph-based ontological data structure, wherein the modification request comprises data representing a change to the graph-based ontological data structure is received from a component of the low-latency database analysis system, verifying that the change clears conflicts, and applying the change to the graph-based ontological data structure after verifying that the change clears conflicts.

Abstract: Improved systems and methods for database analysis are described herein. A method includes generating a graph-based ontological data structure including nodes connected by edges in a low-latency database analysis system, wherein each node represents a respective analytical-object in the low-latency database analysis system, maintaining versions for each of the nodes in the graph-based ontological data structure, maintaining versions for each of the edges in the graph-based ontological data structure, maintaining a transaction log for each transaction with respect to the graph-based ontological data structure, reverting to an earlier version of at least a portion of the graph-based ontological data structure using the transaction log, versioned nodes, and versioned edges in response to an event, and outputting a version of the graph-based ontological data structure in a defined form for presentation to a user or for use by a client.

Abstract: Improved systems and methods for database analysis are described herein. A method includes generating a graph-based ontological data structure including nodes connected by edges in a low-latency database analysis system, wherein each node represents a respective analytical-object in the low-latency database analysis system, maintaining versions for each of the nodes in the graph-based ontological data structure, maintaining versions for each of the edges in the graph-based ontological data structure, maintaining a transaction log for each transaction with respect to the graph-based ontological data structure, reverting to an earlier version of at least a portion of the graph-based ontological data structure using the transaction log, versioned nodes, and versioned edges in response to an event, and outputting a version of the graph-based ontological data structure in a defined form for presentation to a user or for use by a client.

Abstract: An information retrieval system converts unstructured ad-hoc search queries into structured search instructions that retrieve data in a structured relational database or an unstructured database. Data from the database is uploaded into a distributed in-memory database system. Tokens are automatically generated based on attributes, measures, and other metadata extracted from the relational database. The tokens are then compared with the non-structured ad-hoc user search queries. The information retrieval system uses the tokens to identify or predict what structured data is associated with user search queries. The tokens guide the user through a set of search terms that the system then uses to generate the structured query instructions. The structured query instructions retrieve specific data and answers from in the database system.

Abstract: An information retrieval system converts unstructured ad-hoc search queries into structured search instructions that retrieve data in a structured relational database or an unstructured database. Data from the database is uploaded into a distributed in-memory database system. Tokens are automatically generated based on attributes, measures, and other metadata extracted from the relational database. The tokens are then compared with the non-structured ad-hoc user search queries. The information retrieval system uses the tokens to identify or predict what structured data is associated with user search queries. The tokens guide the user through a set of search terms that the system then uses to generate the structured query instructions. The structured query instructions retrieve specific data and answers from in the database system.