Abstract: Semantic analysis to resolve ambiguous user input data with respect to a request for data includes identifying tokens based on the text string, identifying a tables in a database, wherein a token indicates a column from a table and a token indicates a column from another table, identifying candidate join paths for joining tables, obtaining respective candidate results using the candidate join paths, outputting combined candidate results including values from the respective candidate results, obtaining second user input data indicating a selected value, identifying the request for data as unambiguous and identifying a selected join path based on the selected value, and, in response to identifying the request for data as unambiguous, outputting data responsive to the request for data using the selected join path.

Abstract: Semantic analysis to resolve ambiguous user input data with respect to a request for data includes identifying tokens based on the text string, identifying a tables in a database, wherein a token indicates a column from a table and a token indicates a column from another table, identifying candidate join paths for joining tables, obtaining respective candidate results using the candidate join paths, outputting combined candidate results including values from the respective candidate results, obtaining second user input data indicating a selected value, identifying the request for data as unambiguous and identifying a selected join path based on the selected value, and, in response to identifying the request for data as unambiguous, outputting data responsive to the request for data using the selected join path.

Abstract: A query generation and processing system includes a relational data store, a query generator, and a query processor. The relational data store stores data ingested from data sources in a first and second datasets. The query generator interprets a data expression in a simplified query language to generate a query in a structured query language based on identifying quads corresponding to the first and second datasets in the data expression and determining an implicit join between the quads based on an unambiguous relationship obtainable from a schema of the first and datasets, in which the data expression does not expressly identify a join between the first quad and the second quad. The query processor generates a query pipeline that uses the data of the first and second datasets stored by the relational data store to execute the query generated by the query processor.

Abstract: Data corresponding to first and second datasets are stored within a low-latency buffer. A first query is executed by computing a join between the first and second datasets to produce a first output using the data stored in the low-latency buffer. Following execution of the first query, data corresponding to the first dataset is maintained in the low-latency buffer and data corresponding to the second dataset is purged from the low-latency buffer based on a determination that the first dataset is a static dataset and a determination that the second dataset is not a static dataset. A second query is then executed using the first dataset to produce a second output while the data corresponding to the first dataset is maintained in the low-latency buffer. The second query may be the same as or different from the first query.

Abstract: Data is ingested from one or more data sources directly into a low-latency memory buffer. In response to ingesting the data, the ingested data is accessed within the low-latency memory buffer to execute a query without requiring creation of a copy of the ingested data and thus without first writing the ingested data to a warm or cold storage. At some point subsequent to executing the query, the ingested data may be purged from the low-latency memory buffer, such as based on a recency of use of a dataset corresponding to the ingested data for query execution. The purging of the ingested data moves the ingested data to a warm or cold storage and clears space in the low-latency memory buffer for later ingested data to be accessed directly within the memory buffer for query execution also without requiring creation of a copy thereof.

Abstract: Semantic analysis to resolve ambiguous user input data with respect to a request for data includes identifying tokens based on the text string, identifying a tables in a database, wherein a token indicates a column from a table and a token indicates a column from another table, identifying candidate join paths for joining tables, obtaining respective candidate results using the candidate join paths, outputting combined candidate results including values from the respective candidate results, obtaining second user input data indicating a selected value, identifying the request for data as unambiguous and identifying a selected join path based on the selected value, and, in response to identifying the request for data as unambiguous, outputting data responsive to the request for data using the selected join path.

Abstract: Improved systems and methods for database analysis are described herein. A method includes generating a graph-based ontological data structure including nodes connected by edges in a low-latency database analysis system, wherein a respective node represents an object in the low-latency database analysis system, and wherein the respective node comprises a body comprising content of the respective object and a header comprising information about the object, receiving a modification request to the graph-based ontological data structure, wherein the modification request comprises data representing a change to the graph-based ontological data structure is received from a component of the low-latency database analysis system, verifying that the change clears conflicts, and applying the change to the graph-based ontological data structure after verifying that the change clears conflicts.

Abstract: Systems and methods for multi-layered key-value storage are described. For example, methods may include receiving two or more put requests that each include a respective primary key and a corresponding respective value; storing the two or more put requests in a buffer in a first datastore; determining whether the buffer is storing put requests that collectively exceed a threshold; responsive to the determination that the threshold has been exceeded, transmitting a write request to a second datastore, including a subsidiary key and a corresponding data file that includes the respective values of the two or more put requests at respective offsets in the data file; for the two or more put requests, storing respective entries in an index in the first datastore that associate the respective primary keys with the subsidiary key and the respective offsets; and deleting the two or more put requests from the buffer.

Abstract: Data-query execution with distributed machine-language query management in a low-latency database analysis system may include obtaining, at a distributed in-memory database, a data-query expressing a request for data in a defined structured query language associated with the distributed in-memory database, automatically generating a high-level language query representing at least a portion of the data-query, obtaining a machine language query corresponding to the high-level language query, executing the machine language query to obtain results data, and outputting the results data. Obtaining the machine language query may include determining whether the machine language query is cached, and in response to a determination that the machine language query is unavailable, sending a request for the machine language query to a distributed machine-language-query management instance.

Abstract: Improved systems and methods for database analysis are described herein. A method includes generating a graph-based ontological data structure including nodes connected by edges in a low-latency database analysis system, wherein each node represents a respective analytical-object in the low-latency database analysis system, maintaining versions for each of the nodes in the graph-based ontological data structure, maintaining versions for each of the edges in the graph-based ontological data structure, maintaining a transaction log for each transaction with respect to the graph-based ontological data structure, reverting to an earlier version of at least a portion of the graph-based ontological data structure using the transaction log, versioned nodes, and versioned edges in response to an event, and outputting a version of the graph-based ontological data structure in a defined form for presentation to a user or for use by a client.

Abstract: Indexing in a low-latency data access and analysis system includes accessing, by an indexing unit of a low-latency data access and analysis system, constituent data from a data source of the low-latency data access and analysis system and indexing the constituent data in an index of the low-latency data access and analysis system by an indexing unit of the low-latency data access and analysis system. Indexing includes partitioning the constituent data based on a characteristic of the constituent data into at least a first partition and a second partition, segmenting the first partition into a first segment of the first partition, sharding the first segment into a first shard of the first segment of the first partition, segmenting, using hash-partitioning, the second partition into one or more segments of the second partition, and for respective segments of the second partition, sharding the respective segment into one or more respective shards.

Abstract: Index sharding in a low-latency database analysis system includes obtaining index configuration data for indexing constituent data, the constituent data including a plurality of logical tables, and indexing, by an indexing unit, the constituent data by partitioning the constituent data based on a characteristic of the constituent data into at least a first partition and a second partition, segmenting the first partition into a first segment of the first partition, sharding the first segment into a first shard of the first segment of the first partition, segmenting, using hash-partitioning, the second partition into one or more segments of the second partition, and for each segment of the second partition, sharding the segment into one or more respective shards.

Abstract: Semantic analysis to resolve ambiguous user input data with respect to a request for data includes identifying tokens based on the text string, identifying a tables in a database, wherein a token indicates a column from a table and a token indicates a column from another table, identifying candidate join paths for joining tables, obtaining respective candidate results using the candidate join paths, outputting combined candidate results including values from the respective candidate results, obtaining second user input data indicating a selected value, identifying the request for data as unambiguous and identifying a selected join path based on the selected value, and, in response to identifying the request for data as unambiguous, outputting data responsive to the request for data using the selected join path.

Abstract: Token based dynamic data indexing with integrated security includes generating attribute tokens based on a database, each attribute token including an ordered sequence of symbols, generating an index indexing the attribute tokens, such that the index includes a root node, a first branch depending from the root node representing a first symbol from a first attribute token, and a security bitmask for the first attribute token, such that a security bitmask for the first branch at the root node is a hierarchical logical disjunction based on the security bitmask for the first attribute token, such that generating a response receiving an information retrieval request indicating a search symbol and a search context, is based on a determination that a horizontal logical disjunction of a vertical logical conjunction of the security bitmask for the first branch and a security bitmask for the information retrieval request indicates authorization.

Abstract: Query execution on compressed in-memory data includes receiving, at a processor of an instance of a distributed in-memory database, a query for data from a table stored in the distributed in-memory database as compressed table data, obtaining results data responsive to the query from the table, and outputting the results data for presentation to a user. Obtaining results data includes allocating memory to identify allocated memory for decompressing the compressed table data, obtaining uncompressed table data by decompressing the compressed table data into the allocated memory, and obtaining the results data from the uncompressed table data. The allocated memory is deallocated in response to obtaining the results data. Compressing a table to form compressed table data is also described.

Abstract: Systems and methods for multi-layered key-value storage are described. For example, methods may include receiving two or more put requests that each include a respective primary key and a corresponding respective value; storing the two or more put requests in a buffer in a first datastore; determining whether the buffer is storing put requests that collectively exceed a threshold; responsive to the determination that the threshold has been exceeded, transmitting a write request to a second datastore, including a subsidiary key and a corresponding data file that includes the respective values of the two or more put requests at respective offsets in the data file; for the two or more put requests, storing respective entries in an index in the first datastore that associate the respective primary keys with the subsidiary key and the respective offsets; and deleting the two or more put requests from the buffer.

Abstract: Data-query execution with distributed machine-language query management in a low-latency database analysis system may include obtaining, at a distributed in-memory database, a data-query expressing a request for data in a defined structured query language associated with the distributed in-memory database, automatically generating a high-level language query representing at least a portion of the data-query, obtaining a machine language query corresponding to the high-level language query, executing the machine language query to obtain results data, and outputting the results data. Obtaining the machine language query may include determining whether the machine language query is cached, and in response to a determination that the machine language query is unavailable, sending a request for the machine language query to a distributed machine-language-query management instance.

Abstract: Index sharding in a low-latency database analysis system includes obtaining index configuration data for indexing constituent data, the constituent data including a plurality of logical tables, and indexing, by an indexing unit, the constituent data by partitioning the constituent data based on a characteristic of the constituent data into at least a first partition and a second partition, segmenting the first partition into a first segment of the first partition, sharding the first segment into a first shard of the first segment of the first partition, segmenting, using hash-partitioning, the second partition into one or more segments of the second partition, and for each segment of the second partition, sharding the segment into one or more respective shards.

Abstract: Improved systems and methods for database analysis are described herein. A method includes generating a graph-based ontological data structure including nodes connected by edges in a low-latency database analysis system, wherein each node represents a respective analytical-object in the low-latency database analysis system, maintaining versions for each of the nodes in the graph-based ontological data structure, maintaining versions for each of the edges in the graph-based ontological data structure, maintaining a transaction log for each transaction with respect to the graph-based ontological data structure, reverting to an earlier version of at least a portion of the graph-based ontological data structure using the transaction log, versioned nodes, and versioned edges in response to an event, and outputting a version of the graph-based ontological data structure in a defined form for presentation to a user or for use by a client.

Abstract: A search engine performs semantic analysis of unstructured search terms to generate relational database queries. By understanding the semantics, the search engine more effectively identifies and predicts what information the user is searching for and provides more in-depth user assistance. The search engine uses different ranking schemes to identify which data is most likely associated with search terms and provides both row level and object level data security. The search engine may identify ambiguous search requests and allows the user to select different join paths that create different mappings between the data. The search engine may generate worksheets containing the selected mappings.