Abstract: In a method for caching a generated query plan for time series data, a query plan for time series data is generated based on a query comprising an expression, the query plan including a path of execution for resolving the query. The path of execution of the query plan to resolve the query is executed. A result of the path of execution is returned as a response to the query. The query plan is cached for retrieval and execution responsive to receiving another query that matches the query, such that the query plan can be used to resolve the another query.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: The present disclosure relates to systems, methods, and computer-readable media for implementing a device management system that controls a plurality of remotely managed computing devices by way of one or more delta proxies. The systems disclosed herein can set up, generate, or otherwise implement a delta proxy that includes interfaces for facilitating communication between a master control system and any number of managed devices. The systems disclosed herein can implement a delta proxy for controlling multiple managed devices via a single instance of the delta proxy. The delta proxy may include a plug-in layer including software plug-ins that facilitate control of devices of different types and functionalities. The delta proxy can additionally include a namespace layer for allocating memory on a per-device basis.

Abstract: Systems and methods are disclosed for optimizing full-spectrum cardinality approximations on big data by exploiting an underlying relationship between LogLog counting estimation techniques and order statistics-based estimation techniques. To accomplish the foregoing, a multiset of objects that each corresponds to one of a plurality of objects associated with a resource are obtained by a computing device. A compound data object is populated by the computing device with data that is derived based on generated hash values that correspond to each object in the obtained multiset. The populated compound data object is processed utilizing a processor with a full-spectrum unified estimation operation that can accurately determine a cardinality estimate for the obtained multiset, utilizing considerably less resources when compared to traditional and state of the art techniques.

Abstract: Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Abstract: Systems, methods, and devices, for managing data skew during a join operation are disclosed. A method includes computing a hash value for a join operation and detecting data skew on a probe side of the join operation at a runtime of the join operation using a lightweight sketch data structure. The method includes identifying a frequent probe-side join key on the probe side of the join operation during a probe phase of the join operation. The method includes identifying a frequent build-side row having a build-side join key corresponding with the frequent probe-side join key. The method includes asynchronously distributing the frequent build-side row to one or more remote servers.

Abstract: Methods, systems, and computer program products for a database processing system are described. A core database comprises common procedures for processing information, at least one of the common procedures configured to provide a common functionality to an analysis application. A content import database comprises content-specific procedures, at least one of the content-specific procedures configured to provide access for at least one of the common procedures to content of the content import database that conforms to a content-specific data model.

Abstract: A method, a computer program product and a computer system are provided. Attribute value information contains at least a minimum value representing a smallest value of a first attribute and a maximum value representing a largest value of the first attribute, thereby defining a first range of values of the first attribute. A received query against a data table requests one or more values of at least the first attribute that are covered by the first range of values. The attribute value information may be used for selecting a data block of the data table as a candidate potentially including at least part of the requested one or more values and scanning the data block. In response to determining that the data block does not include the one or more requested values, the attribute value information may be updated accordingly.

Abstract: Methods and apparatuses for determining set-membership using Single Instruction Multiple Data (“SIMD”) architecture are presented herein. Specifically, methods and apparatuses are discussed for determining, in parallel, whether multiple values in a first set of values are members of a second set of values. Many of the methods and systems discussed herein are applied to determining whether one or more rows in a dictionary-encoded column of a database table satisfy one or more conditions based on the dictionary-encoded column. However, the methods and systems discussed herein may apply to many applications executed on a SIMD processor using set-membership tests.

Abstract: Methods and apparatuses for determining set-membership using Single Instruction Multiple Data (“SIMD”) architecture are presented herein. Specifically, methods and apparatuses are discussed for determining, in parallel, whether multiple values in a first set of values are members of a second set of values. Many of the methods and systems discussed herein are applied to determining whether one or more rows in a dictionary-encoded column of a database table satisfy one or more conditions based on the dictionary-encoded column. However, the methods and systems discussed herein may apply to many applications executed on a SIMD processor using set-membership tests.

Abstract: Examples disclosed herein relate to accessing electronic databases. Some examples disclosed herein may include partitioning a computation task into subtasks. A processing node of a computation engine may generate a database query for retrieving an electronic data segment associated with at least one of the subtasks from a database. The database query may include pre-processing instructions for a database management system (DBMS) associated with the database to pre-process the electronic data segment before providing the electronic data segment to the processing node. The pre-processing instructions may include at least one of: filtering, projection, join, aggregation, count, and user-defined instructions. The generated query may be provided to the DBMS.

Abstract: A method including calculating processing cost data of the general data table of the data common layer; determining a general data table on which the external data table of the non-data common layer is dependent; and calculating use cost data of the external data table according to the processing cost data of the general data table. In evaluation of the cost of each general data table of the data common layer, instead of taking into account the storage and computation consumption of the current data table in isolation, several upstream data tables and multiple siblings of the data table are also taken into account comprehensively. The processing cost of the general data table is reasonably and accurately assessed, thus reflecting the advantages and disadvantages of data model construction of the data common layer, and providing decision-making support for the optimization and operation of the models of the data common layer.

Abstract: A database engine receives a human-readable database query that includes a plurality of conditional expressions. The database engine parses the database query to build an operator tree that includes a subtree corresponding to each of the conditional expressions. The database engine identifies a subexpression that appears in two or more of the conditional expressions. The subexpression is executed conditionally. The database engine hoists the subexpression outside of the conditional expression so that it is executed unconditionally. The database engine modifies the operator tree to specify computation of a value for the subexpression a first time and to reuse the computed value when the subexpression is subsequently encountered. The database engine executes the modified operator tree to form a result set corresponding to the database query, thereby evaluating the subexpression only a single time for each row of input data and returns the result set.

Abstract: A big data sampling system indexed by at least one key and including a big data table comprising complete historical data corresponding to at least one financial instrument, a query table indexed by a query identifier associated with each of the at least one financial instrument and including references to the complete historical data of the big data table, and an inner join corresponding to the query table and the big data table. The inner join takes in a sample of the complete historical data corresponding to an input query. The sample is extracted by byte range download from a file server and is associated with the query identifier of the query table. The sample corresponds with the complete historical data of the at least one financial instrument. An output of the inner join identifies the complete historical data of the financial instrument in response to the query.

Abstract: A system and computer implemented method for execution of aggregation expressions on a distributed non-relational database system is provided. According to an aspect, an operation is provided to perform the aggregation operation. An indication of at least one foreign collection of documents is accepted. An indication of at least one variable for at least one local field of at least one local collection of documents is accepted. An indication of a pipeline for execution on the at least one local collection of documents and the at least one foreign collection of documents is accepted. An operation comparing the local field of the at least one local collection of documents and the foreign field of the at least one foreign collection of documents is performed. Responsive to the operation, data from the at least one local collection of documents and the at least one foreign collection of documents is aggregated.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: In an example, a database system may be configured to receive configuration data including a mapping of objects of a relational database to query generation parameters; compare values in a received first query to the mapping to identify column definition data of the query generation parameters; ascertain whether to identify table definition data of the configuration data based on the column definition data; determine whether to create one or more subqueries for one or more tables based on the table definition data; in response to determining to create the one or more subqueries, join a table of the one or more tables to a root table based on table relationship data of the configuration data; and build a second relational datastore query using the one or more subqueries.

Abstract: A system, computer-implemented method, and a computer-readable storage medium for determining estimations to statistical questions. Data statistics used in evaluating estimations to statistical questions are gathered from multiple sources in a database management system. A statistical question comprising, for example, cardinality estimations is received, wherein the question is posed by different database components, such as query optimizer, or execution engine. Estimation for the statistical question is computed, where the estimation is based on multiple data statistics and without using the data referred to in the question.

Abstract: A facility for querying user-defined database columns is described. The facility receives a database query referring to one or more database columns, at least one of the referred-to database columns being a user-defined database column. The facility provides access to a database query result generated for the perceived database query via a programmatic object.

Abstract: Technologies are disclosed herein for providing asynchronous application store event notifications. An application developer registers with an application store to receive asynchronous push notifications for application store events of interest, such as events relating to in-application purchases, payment events, fulfillment events, application publishing events, or other types of events. When an event for which a notification is to be provided is detected, an application store event message is generated for the event and provided to a notification service. The notification service generates an application store event notification for the event on a topic defined in advance by the application developer. The application store event notification may then be delivered to an endpoint defined by the application developer, such as a hypertext transfer protocol (HTTP) endpoint.

Abstract: Techniques and solutions are provided for analyzing constraints. A disclosed embodiment provides for determining whether a table contains a valid tuple, and if so, the minimum index of the first valid tuple. Determining whether the table contains a valid tuple can provide an early indication of whether a set of constraints is violated or entailed, for positive and negative constraints, respectively. Determining the minimum index of the first valid tuple can expedite subsequent constraint analysis. Further disclosed embodiments provide for removal of duplicate and wildcard values from a set of constraints, which can facilitate negative constraint analysis. An indication can be provided, such as through a user interface, of whether a set of constraints is violated or entailed. Software can restrict selection of values that may result in constraint violation based on the provided indication.

Abstract: In an example, a top-k function is associated with a top-k projection for a data storage system. Input data to be loaded into the data storage system is divided and ranked according to the top-k function and stored in the top-k projection.

Abstract: Performing a join of first and second database tables for a query includes applying a predicate of the query to the first table as a first predicate and determining how many distinct join key values the first table has that survive the applying of the first predicate, wherein a join key value of the first table that survives the applying of the first predicate is a surviving join key value for a second predicate. A selection includes selecting among applying the second predicate to the second table, probing the second table with the second predicate, and neither applying the second predicate to the second table nor probing the second table with the second predicate, wherein the selecting is responsive to the number of distinct, surviving join key values.

Abstract: Techniques related to efficient evaluation of queries with multiple predicate expressions are disclosed. A first predicate expression (PE) is evaluated against a plurality of rows in a first column vector (CV) to determine that a subset of rows does not satisfy the first PE. The subset comprises less than all of the plurality of rows. When a query specifies the first PE in conjunction with a second PE, a selectivity of the first PE is determined. If the selectivity meets a threshold, the second PE is evaluated against all of the plurality of rows in a second CV. If the selectivity does not meet the threshold, the second PE is evaluated against only the subset of rows in the second CV. When a query specifies the first PE in disjunction with the second PE, the second PE may be evaluated against only the subset of rows in the second CV.

Abstract: Performing a join of first and second database tables for a query includes applying a predicate of the query to the first table as a first predicate and determining how many distinct join key values the first table has that survive the applying of the first predicate, wherein a join key value of the first table that survives the applying of the first predicate is a surviving join key value for a second predicate. A selection includes selecting among applying the second predicate to the second table, probing the second table with the second predicate, and neither applying the second predicate to the second table nor probing the second table with the second predicate, wherein the selecting is responsive to the number of distinct, surviving join key values.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: A batch data query method and apparatus are provided. According to the method, multiple logical query plan trees corresponding to multiple query statements are obtained. The multiple logical query plan trees are divided into multiple query groups according to operators of each logical query plan tree. Logical query plan trees in each query group have the following relationship: an inter-query write-read dependency relationship, an operator overlap relationship, or an operand overlap relationship. Then, inter-query optimization is performed on the logical query plan trees in each query group, and multiple corresponding physical query plan trees are generated according to the multiple optimized logical query plan trees. Finally, physical queries are executed according to the multiple physical query plan trees, to obtain query results of the multiple query statements.

Abstract: Aspects automatically provisioning of cloud resources for ETL job execution as a function of optimizer output data. Access paths identify overall cost and individual step operator costs and statistics relative to cloud resources for executing an SQL commands against a relational database. In one aspect operator steps having high step costs are identified and amounts, types or speeds of a cloud resources associated with the step are adjusted to thereby reduce the step cost and the access path overall cost. In another aspect SQL text descriptors and step costs and function statistics are compared to knowledge base criteria for matches to criteria associated inefficient configurations cloud resources for revision to improve corresponding operation efficiency. In another aspect usages of cloud resources by operator steps are monitored and logged and compared to historic data of similar steps to determine under or over-utilization of resources.

Abstract: Example implementations relate to automatic generation of sub-queries. For example, a computing device may include a processor. The processor may receive a query defining a particular set of data stored in at least one database. The processor may access metadata associated with the particular set of data and may select a data attribute associated with the particular set of data based on the metadata. The processor may automatically generate sub-queries of the query using the data attribute selected based on the metadata. The processor may process the sub-queries and provide a query result of the query that is a combination of sub-query results of the sub-queries.

Abstract: Techniques are described herein for processing queries comprising joins specifying a plurality of tables. The techniques involve partitioning the tables by assigning rows to partitions. One or more partition maps may be generated to indicate the partitions. Subsequent tables may be partitioned based on the generated partition maps. The partitions may be used to determine results for the joins.

Abstract: A method, computer system, and a computer program product for transforming a scalar sub-query to join with non-provable singleton output, responsive to receiving a parsed representation of a statement, containing a scalar operation, which is correlated to predetermined types of objects, by analyzing the parsed representation to determine eligibility for transformation of any scalar operations. Responsive to a determination of eligibility, a query engine injects additional processing into the parsed representation to defer checking of a 1 row condition required by statement semantics. The query engine further injects processing to utilize a result of the additional processing to validate a single row is produced for each GROUPING/PARTITION key. The parsed representation is modified to remove the 1 row condition. The query engine decorrelates the parsed representation using a predetermined method absent a requirement for a unique key to be present in a predicate of the scalar operation.

Abstract: A method for transforming a scalar sub-query to join with non-provable singleton output, responsive to receiving a parsed representation of a statement, containing a scalar operation, which is correlated to predetermined types of objects, by analyzing the parsed representation to determine eligibility for transformation of any scalar operations. Responsive to a determination of eligibility, a query engine injects additional processing into the parsed representation to defer checking of a 1 row condition required by statement semantics. The query engine further injects processing to utilize a result of the additional processing to validate a single row is produced for each GROUPING/PARTITION key. The parsed representation is modified to remove the 1 row condition. The query engine decorrelates the parsed representation using a predetermined method absent a requirement for a unique key to be present in a predicate of the scalar operation.

Abstract: Systems and methods are disclosed which employ a centralized selection context having selection states spread across user interface elements. Methods are described in which the binding between widgets and data storage is isolated using the centralized selection context paradigm for a majority of data items represented within a user interface. This selection context is fully accessible by methods that access data while considering the current selection context. The centralized selection context simplifies event-handling logic by having selection context readily available thus removing the need to query individual widgets for their state.

Abstract: A reference data tool helps meet the technical challenges associated with providing important data to an application that needs the data. The tool causes execution of custom reports on an external data platform that extract the data needed by the application from the external data platform. The tool creates a custom data store for the data in a location accessible to the application. After the tool receives the extracted data, the tool automatically writes the data into the proper location in the custom data store.

Abstract: Salient sampling for query size estimation includes identifying two or more columns in a database table that have corresponding columns in one or more other tables. One or more hash functions are applied to domains of each of the identified columns. A first hash function is applied to a domain of the first column and a second hash function to a domain of the second column. A subset of the rows in the database table is selected. The selecting includes selecting rows in the database table where results of the first hash function meet a first numeric threshold and selecting rows in the database table where results of the second hash function meet a second numeric threshold. A sample database table corresponding to the database table is created. The sample database table includes the selected subset of the rows in the database table.

Abstract: The disclosure relates to posting auditable, immutable data to a blockchain. The method includes receiving a blockchain comprised of a plurality of blocks, each block having at least a block header and one or more transaction values, where each transaction value includes at least a transaction hash, and receiving a data file associated with a specific transaction hash included in a block in the blockchain, where the data file includes one or more variables. The method includes modifying at least one of the one or more variables included in the data file, generating a new hash value via application of one or more hashing algorithms to the modified data file, and generating a new transaction value based on at least the generated new hash value and the specific transaction hash. The method includes digitally signing the generated new transaction value, and electronically transmitting the signed new transaction value.

Abstract: A calculation engine of a database management system is described. In some implementations, the calculation engine receives a calculation scenario including a plurality of join operations defining an intersection between at least two nodes. The calculation engine optimizes a first join which is of a certain cardinality, and for which no attributes are requested, other than a join attribute. The optimization includes determining whether a static filter is present for a first node or a second node of the first join, and pruning the first node and/or the second node from the hierarchical join when the attribute is not requested from the first node or the second node and/or when the static filter is not present for the first node or the second node. Related systems, methods, and articles of manufacture are also described.

Abstract: A method for spooling data for use in joining a small table with a large table in a relational database system. The method analyzes a join condition for combining records from the small and large tables, selects qualified rows from the large table, and writes the qualified rows to a spool file. The spool file includes a first partition containing hash values of all bind terms for the join condition; a second partition including a join column with a best selective bind term; and at least one additional partition including additional join columns used in bind terms. The partitions are grouped together within a container row in the spool file, and multiple container rows are written together within a super-container row in the spool file.

Abstract: Methods and systems disclosed herein may optimize write operations in a transaction to reduce the number of operations to the point where each of the write operations may be performed in parallel. The writer optimizer may review a first write operation and a second write operation to determine whether the first write operation and the second write operation write to the same row in the same table. When the first write operation and the second write operation write to the same row in the same table, the first write operation and the second write operation may be combined to generate a functionally-equivalent third write operation. The third write operation may reduce the number of operations in the transaction by replacing both the first write operation and the second write operation.

Abstract: Techniques and structures for providing a multitenant environment having both a relational database and a non-relational database. A database query is received with at least one server computing device providing a multitenant environment having a relational database system and a non-relational database system, the database query in a format corresponding to the relational database system. The query is parsed with the at least one server computing device to find one or more components of the query directed to data stored in the non-relational database system. One or more components of the query directed to data stored in the non-relational database system are converted to one or more scans of the non-relational database system. Results from the query of the relational database system and results from the one or more scans of the non-relational database system are collected. The results from the query and the results of the one or more scans to a combined result are combined.

Abstract: A received query is analyzed to provide a set of potential bushy tree operation plans that include the creation of one or more “sub-joins” that provide intermediate resultant “sub-tables”, and using these sub-tables as operands in the other predicates of the query. The received query is analyzed to identify “satellite” tables and “seed” tables. A satellite table is a table of the database that is filtered by a selective predicate, and is connected to only one other table in the database via at least one linking predicate, and a seed table is a table of the database that is connected to a satellite table and at least one other table. Multiple candidate operation plans are formulated as bushy trees that include a sub-join of each seed table and its connected satellite table. A least cost operation plan is selected for execution from among these candidate operation plans and the conventional least cost linear operation plan based on the costs associated with each operation plan.

Abstract: Embodiments of the invention improve over conventional or routine technologies by generating a separate data structure or organization, other than those temporary search files that are typically used, that facilitates the processing of the datasets. In one embodiment, instead of using sorted temp files or running sorting algorithms, a meta-join and/or meta-group-by indexes may be generated or created that provide a quick access to the records in the datasets. Moreover, embodiments of the invention eliminate the need to sort or pre-sort datasets before or during a query is conducted on the dataset. Aspects of the invention use, for example, the meta-join and/or meta-group-by indexes to pre-organize the data such that, when executing a query against the dataset, the query is executed against the meta-join and/or meta-group-by indexes.

Abstract: Methods and systems for synthesizing mapping tables using table corpus is provided. A functional dependency between at least two items of an input table is determined. A plurality of two-column tables are extracted from the table corpus. The extracted plurality of two-column tables are synthesized to determine at least one mapping table having a first column having the functional dependency with a second column. A next item of the input table is provided from the determined at least one mapping table.

Abstract: A system and method allowing customization of a user-interface into a data repository are disclosed. Customization options are provided via a hierarchy and/or compartmentalization of structured control files such as Extensible Markup Language (XML) files. The control files may be defined in a hierarchical fashion allowing a particular end-user or group of end-users to customize their interface without impacting other users of the same data and user-interface system. Compartmentalization of functions to specific files may also aid in ease of maintenance of customizations. Customization may include an ability to customize what data is shown on a screen and which capabilities are available on each screen for data interaction pertaining to a particular job responsibility. Capabilities to provide these and other customization options to an end-user are disclosed in a non-limiting embodiment of an import export trade transaction storage and retrieval system.

Abstract: Systems, methods, and devices for generating and updating cross-account materialized views in multiple tenant database systems. A methods includes defining a share object in a first account wherein the share object includes data associated with the first account. The method includes granting cross-account access rights to the share object to a second account such that the second account has access to the share object without copying the share object. The method includes generating a materialized view over the share object. The method includes updating the data associated with the first account. The method includes identifying whether the materialized view is stale with respect to the share object by merging the materialized view and the share object.

Abstract: A pattern can be identified in at least part of a query whose definition is received in a query request. The identified pattern can be matched with a set of pre-defined patterns, each of which has associated therewith at least one pre-compiled query execution sub-component of a plurality of pre-compiled query execution sub-components retained in a library. A plan for executing the query can be generated, for example by incorporating the pre-compiled query execution sub-component associated with the matched pattern into the plan based on a pseudo code representation of the plan derived from the definition.

Abstract: According to an aspect, a system includes a query engine configured to obtain a multi-provider query model for a query specifying a filter to be applied on provider partitions. The obtained multi-provider query model incorporates a pruning plan. The query engine includes a query optimizer configured to execute the pruning plan during optimization of the multi-provider query model. The pruning plan determines if each of the provider partitions includes content that is required to execute the filter based on pruning information. The query optimizer configured to reduce a scope of the query by excluding at least one provider partition that is determined as not required to execute the filter.

Abstract: A first data table and a second table to be joined is determined. The first data table and the second data table have a join attribute. Data blocks of the first data table are stored on a storage device. An attribute value information for the join attribute for the data block of the first data table is determined. At least one partition for the first data table and the second table is defined using at least the attribute value information on the join attribute. Each partition of the at least one partition has a respective partition range of values of the join attribute. A pair wise partition join is processed on a first partition range of the determined partition ranges.

Abstract: This disclosure proposes systems, methods, and apparatus that identify raw SQL queries that are likely to cause a double counting error, and if such a SQL query is identified, then convert the raw SQL query into SQL queries that account for and avoid double counting. In some embodiments, this process uses queries and subqueries that refer back to a common table expression (CTE) in order to reduce code length and increase query execution speed.