Abstract: An in-vehicle/out-vehicle cooperation device includes a data receiving unit configured to receive sensor data from the plurality of sensors via an in-vehicle network, an inter-end delay time estimation unit configured to estimate a transmission delay time of sensor data between ends from each of the plurality of sensors to a predetermined device communicable by a wireless communication device, a value determination unit configured to determine a value of sensor data output from the plurality of sensors based on a state of the vehicle and states of the plurality of sensors, and a selection unit configured to select a part of the sensor data based on transmission delay time estimated by the delay time estimation unit and the value determined by the value determination unit, and transmit a copy of the selected sensor data to the predetermined device via the wireless communication device.

Abstract: A database system is operable to determine a request to implement a K Nearest Neighbors (KNN) algorithm to generate classification data for a set of new records. A query operator execution flow is determined for the request that includes a KNN-join operator. A query resultant that indicates classification data for the set of new records is generated by performing a plurality of operator executions in accordance with the query operator execution flow based on, for each record of the set of new records, generating a plurality of similarity measures by performing a similarity function on the each record and each of a set of previously-classified records; identifying a proper subset of the set of previously-classified records that includes exactly a predefined number of records; and joining the each record with the proper subset of the set of previously-classified records.

Abstract: The description relates to executing an inference query relative to a database management system, such as a relational database management system. In one example a trained machine learning model can be stored within the database management system. An inference query can be received that applies the trained machine learning model on data local to the database management system. Analysis can be performed on the inference query and the trained machine learning model to generate a unified intermediate representation of the inference query and the trained model. Cross optimization can be performed on the unified intermediate representation. Based upon the cross-optimization, a first portion of the unified intermediate representation to be executed by a database engine of the database management system can be determined, and, a second portion of the unified intermediate representation to be executed by a machine learning runtime can be determined.

Abstract: A query is received. It is determined that the query does not fit a profile for a run-the-business set of queries, where the profile for the run-the-business set of queries excludes queries that are not routine parts of running a business and that do not require priority processing. The query is executed with a dynamically-created compute capacity that is not part of a compute capacity used to run the run-the-business set of queries.

Abstract: Method and apparatus for optimized data partition selection for related queries. A query to be executed over a plurality of data records is received. The plurality of data records comprises a first set of data records stored in a first database distributed across a plurality of partitions and a second set of data records stored in a second database. An input vector is generated based on the query, the first and second databases. The input vector is processed using a machine learning (ML) model to predict a cost for executing the query if one or more data records in the second set of data records are loaded to a first partition of the first database. A plan is selected for loading the one or more data records from the second set of data records to the first partition based on the predicted cost for executing the query.

Abstract: Described is intelligent security based on the detection of the presence of mobile devices within a defined region, determined via accurate mobile device location sensing, e.g., within on the order of one meter. Upon detection of the presence of a mobile device within a defined region via mobile device millimeter wave-based position sensing, an identity associated with the mobile device is obtained. If the identity associated with the mobile device is not a trusted identity, action can be taken based on the presence of the identity within the defined region. The intelligent security can be implemented within an application program of a registered user's mobile device and/or a security system, such as a commercially available security system with alarm monitoring services. Various criteria can determine the action(s) to take, including by which region (if more than one), time of day, identity trust level, duration of presence, and the like.

Abstract: System and method for interacting with a plurality of data sources are provided. A request may be parsed and an identification parameter identifying a data set may be determined. A field included in the request may be designated as a distribution key. At least one data source may be selected based on a value associated with the distribution key. At least a portion of the request may be sent to a selected data source. Other embodiments are described and claimed.

Abstract: A real-time data validation system prevents problematic updates from being replicated from a primary database to a replica database. The data validation system may inject a time delay into the replication process, so that when an update in a replication stream fails a specified data validation, the data validation system will prevent the update from being applied to the replica. The replication process may be halted to take snapshot of the replica database during the halt, and resumed after the snapshot. Accordingly, if the problematic update is later confirmed to be invalid, the replica database can be rolled back to a point before the problematic update. Advantageously, the disclosed system can be used to implement a variety of data validations on top of a database replication process to protect the replica database from receiving undesired updates.

Abstract: A system and method for providing local approximations of query results are provided. The method includes querying a primary neural network with at least one test query, wherein the at least one test query includes a real test result derived from executing the at least one training query on a data set; receiving from the primary neural network a predicted test result in response to the at least one test query; sending, based on the predicted test result, a model of a primary neural network to a local machine; and storing the model of a local neural network of the local machine, wherein the local neural network is configured to generate a prediction in response to a user query received by the local machine.

Abstract: Systems and methods for query generation based on merger of subqueries are described. For example, methods may include accessing a first join graph representing tables in a database, wherein the first join graph has vertices corresponding to respective tables in the database and directed edges corresponding to join relationships; receiving a first query specification that references data in two or more of the tables of the database to specify multiple subqueries in a set of subqueries; checking that two or more subqueries from the set of subqueries have the same join graph; checking that the two or more subqueries have the same set of grouping columns; responsive, at least in part, to the two or more subqueries having the same join graph and the same set of grouping columns, merging the two or more subqueries to obtain a consolidated query.

Abstract: A data layout model generation system generates, with reinforcement learning, a node configuration and a data layout key in a distributed parallel database. This system includes a sample acquisition processor that acquires, on the basis of a predetermined acquisition method, sample data from data stored in the distributed parallel database, a data layout estimator having, as states in the reinforcement learning, the node configuration and the data layout key including information regarding an order of sorting columns that constitute the data and information regarding a method for distribution between nodes, the data layout estimator estimating layout of the data on the basis of the state and the sample data, a reward calculator that calculates a reward in the reinforcement learning on the basis of a result obtained by estimating the layout of the data, the node configuration, and a processing cost of a query executed on the distributed parallel database.

Abstract: Fog computing systems are provided comprising edge-nodes and middle-nodes between edge-nodes and cloud-node. These nodes form a hierarchical structure with each cloud, middle node having children nodes, and each middle, edge node having a parent-node. Each edge-node receives data from sensors, assigns reception-timestamp to each data indicating when data has been received to produce series of timestamp-ordered data, trains local model through machine-learning based on said series of timestamp-ordered data, and sends said series to parent-node of the edge-node. Each middle-node collects series of timestamp-ordered data from children nodes of the middle-node, trains supra-local model through machine-learning based on said collected series of timestamp-ordered data, and sends said collected series to parent-node of the middle-node. Parent-children structures, edge-nodes and middle-nodes for such fog computing systems are also provided.

Abstract: A method, including receiving, from a client, a unified query, and extracting, from the unified query, an endpoint query for a first data source on a first server and an endpoint query for a second data source on a second server. The extracted endpoint query for the first data source is forwarded to the first server. Upon receiving a response to the endpoint query forwarded to the first server, one or more parameters are extracted from the response. The endpoint query for the second data source is updated so as to include the extracted one or more parameters, and the updated endpoint query for the second data source is forwarded to the second server. Upon receiving, from the second server, a response to the forwarded endpoint query, a result for the received unified query is generated based on the receive responses, and the generated result is conveyed to the client.

Abstract: A method, system and computer program product for join graph generation based upon a log of previously executed database queries includes method for generating a join graph for relational database queries. The method includes loading into memory of a computer, a log of a set of database queries previously executed against data in a database and sequentially parsing each of the queries in the log to identify different semantically characterizable components of each of the queries. The method further includes generating a join graph for each of the queries from corresponding ones of the components. Finally, the method includes selectively adding each of the generated join graphs to a set of join graphs in a data model for the data in the database.

Abstract: A data platform that implements memoizable functions for database objects. The data platform detects a first execution of a memoizable function and generates a first key based on metadata of one or more database objects operated on by the memoizable function and generates a first result for the memoizable function based on the one or more database objects. The data platform detects a second execution of the memoizable function and generates a second key based on the metadata of the one or more database objects operated on by the memoizable function. When the first key and the second key are equal, the data platform reuses the first result of the memoizable function. When the first key and second key do not match, the data platform generates a second result for the second execution of the memoizable function.

Abstract: Systems, methods, and devices for implementing secure views for zero-copy data sharing in a multi-tenant database system are disclosed. A method includes generating a share object in a first account comprising a share role. The method includes associating view privileges for the share object such that an underlying detail of the share object comprises a secure view definition. The method includes granting, to a second account, cross-account access rights to the share role or share object in the first account. The method includes receiving a request from the second account to access data or services of the first account and providing a response to the second account based on the data or services of the first account. The method is such that the underlying detail of the share object that comprises the secure view definition is hidden from the second account and visible to the first account.

Abstract: Even when one refactoring operation cannot establish a target software structure, an appropriate refactoring operation establishes the target software structure. An improvement proposing device includes: a structure comparator to output, as an improvement object, a difference between a first software structure and a second software structure different in software structure from the first software structure; and an improvement plan examining unit to examine an improvement plan for each improvement portion in the improvement object, the improvement plan being a method for bringing the first software structure closer to the second software structure.

Abstract: Provided is a method of indexing in a network key value indexing system. The method includes retrieving a first key name from a storage device of the network key value indexing system, the first key name identifying a first prefix, a first bucket, and a first key, the first prefix indicating the first bucket, parsing the first key name into the first prefix, the first bucket, and the first key, determining the first prefix, the first bucket, and the first key based on a first delimiter, and generating a hash table in a memory cache of the network key value indexing system to associate the first prefix with the first key.

Abstract: A method for dynamic filter pushdown for massive parallel processing databases on the cloud, including acquiring one or more filters corresponding to a query, acquiring statistics information of one or more database tables, determining a selectivity of the one or more database tables based on the statistics information, determining whether the selectivity satisfies a threshold condition, and pushing down the one or more filters to the one or more database tables based on the determination of whether the selectivity satisfies a threshold condition.

Abstract: One example method includes evaluating code of a quantum circuit, estimating one or more runtime statistics concerning the code, generating a recommendation based on the one or more runtime statistics, and the recommendation identifies one or more resources recommended to be used to execute the quantum circuit, checking availability of the resources for executing the quantum circuit, allocating resources, when available, sufficient to execute the quantum circuit, and using the allocated resources to execute the quantum circuit.

Abstract: Methods and system are presented for monitoring database processes to generate machine learning predictions. A plurality of database processes executed on database implementations can be monitored, wherein the monitoring includes determining a start time, an end time, and a number of rows impacted by portions of the database processes, and the monitored database processes generate instances of machine learning data including at least the number of rows impacted and an associated duration of time. Using a machine learning component and the machine learning data, a duration of time can be predicted for a candidate database process for execution on a database implementation.

Abstract: Examples of dynamic database query efficiency improvement are provided herein. Query portions of a received database query can be identified as candidates for replacement. The candidates for replacement can be query portions that reduce the efficiency of the query. Alternative queries can be determined that include substitute query portion(s) in place of candidate(s) for replacement. An expected performance of the alternative queries can be determined. Based at least in part on the expected performance of the alternative queries, one or more alternative queries can be selected as replacement database queries for the received database query.

Abstract: Techniques for massively-parallel real-time database-integrated machine learning (ML) inference are described. An ML model is deployed as one or more model serving units behind an endpoint. The ML model can be associated with a virtual table or function, and a query that is received that references the virtual table or function can be processed by issuing inference requests to the endpoint by the query execution engine(s).

Abstract: Mechanisms for resolving a database query are provided. In some instances, these mechanisms include identifying a connected component in a query graph corresponding to the database query. In some instances, these mechanisms further include determining a longest path length for the connected component. In some instances, these mechanisms further include selecting a path having the longest path length. In some instances, these mechanisms still further include building an algebraic expression for the path. In some instances, these mechanisms still further include solving the algebraic expression using matrix-matrix multiplication to provide a solution. And, in some instances, these mechanisms still further include responding to the query based on the solution.

Abstract: Operations include estimating, in real time, a runtime of a query. The query optimization system receives set of query definitions for defining a target query. The system uses the set of query definition elements to determine an estimated runtime for the target query. If the estimated runtime exceeds some acceptable threshold value, then the system determines a modification to the set of query definition elements. The system uses the modification to generate a modified query, corresponding to a lower estimated runtime.

Abstract: The invention relates to a commitment process for efficiently generating a jointly supported decision.

Abstract: Context dependent execution time prediction may be applied to redirect queries to additional query processing resources. A query to a database may be received at a first query engine. A prediction model for executing queries at the first query engine may be applied to determine predicted query execution time for the first query engine. A prediction model for executing queries at a second query engine may also be applied to determine predicted query execution time for the second query engine. One of the query engines may be selected to perform the query based on a comparison of the predicted query execution times.

Abstract: A complex data type is encoded over columns of a table of a columnar database by mapping fields of the complex data type to the columns. An optimized query can be generated for a query specifying the complex data type. The optimized query specifies the columns to which the fields specified within the query are mapped, instead of specifying the fields. The optimized query can be processed against the database in a late materialization manner to fulfill the query.

Abstract: A method for execution by a query processing system includes receiving a query request that indicates a query for execution by a database system. A plurality of query execution mode options for execution of the query via the database system can be determined. A plurality of execution success conditions corresponding to the plurality of query execution mode options can be determined. A plurality of resultant correctness guarantee data corresponding to the plurality of query execution mode options based on the plurality of execution success conditions can be generated. Query execution mode selection data can be generated by selecting a query execution mode from the plurality of query execution mode options based having resultant correctness guarantee data that compares favorably to determined resultant correctness requirement data. A resultant for the query can be generated by facilitating execution of the query in accordance with the selected execution mode.

Abstract: A method of operating a communications network is disclosed. Modern communications networks produce vast amounts of network operational data which have the potential to provide a useful summary of the operational state of the network. Whilst processes such as clustering are known for arranging the vast amount of data into groups, the clusters themselves do not provide data which might be easily interpreted by network elements or administrators. Network operational data often comprises a plurality of data items, each of which gives a value for each of a set of attributes. By processing a cluster to identify attributes in the cluster whose values vary less in the cluster then they vary outside of the cluster, and then generating a cluster description which is based on a measure of the central tendency of the values of those attribute in the cluster, an easily interpretable general description of the data items in the cluster is provided.

Abstract: A method, system and computer program product for providing early diagnosis of hardware, software or configuration problems in a data warehouse system. A received query is parsed to determine the properties of the query. The query may then be joined to existing groups of queries if those groups have shared properties of the query. After executing the query according to an execution plan, results from the execution of the query is received, which may include problem(s) that occurred during execution of the query. For those problems that reach a pre-defined threshold of becoming a “group problem” in those groups joined by the query, the problem is reported to the end user concerning those groups where the problem exceeds the pre-defined threshold. In this manner, an early diagnosis of the problems in the data warehouse system that can cause delay and failure of the processing of queries is able to occur.

Abstract: Methods and systems for generating database statistics. Table statistics in a metadata catalog of a source database system are observed, statistics generation costs utilizing a target database system are estimated, and source statistics generation costs utilizing a source database system are estimated. The statistics generation costs are compared and statistics generation queries by the target database system are triggered in response to the statistics generation costs utilizing the target database system having a predefined relationship with the source statistics generation costs utilizing the source database system. The statistics generation queries are performed by the target database system in response to the triggering by the source database system. The generated statistics are sent from the target database system to the source database system, the table statistics in a metadata catalog are updated based on the generated statistics, and the updated table statistics are used to optimize a query plan.

Abstract: Various embodiments relate generally to data science and data analysis, computer software and systems, and wired and wireless network communications to provide an interface between repositories of disparate datasets and computing machine-based entities that seek access to the datasets, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby a collaborative data layer and associated logic facilitate, for example, efficient access to, and implementation of, collaborative datasets. In some examples, a method may include receiving data representing a query of a consolidated dataset that may include datasets formatted atomized datasets, analyzing the query to classify portions of the query to form classified query portions, partitioning the query into sub-queries as a function of a classification type for each of the classified query portions, and retrieving data representing a query result from distributed data repositories.

Abstract: In general, embodiments of the present invention provide systems, methods and computer readable media for automatic cleaning of entity resolution (ER) data persistently stored in a data repository.

Abstract: In accordance with one aspect of the present disclosure, a request to provide recommendations of data enrichments for a database is received at a recommendation engine. The recommendation engine may perform static and dynamic analysis of data associated with the database and may further refine recommendations based on policies. The recommendation engine may then provide the recommendations, if any, of data enrichments to allow a software developer, for example, to indicate whether the data enrichments are to be used.

Abstract: Automated Index Management entails automated monitoring of query workload in a DBMS to determine a set of higher load queries to use to evaluate new potential indexes. Without the need of user approval or action, the potential indexes are automatically created, evaluated and tested, and then made available for system wide use for executing queries issued by end users. Indexes created by Automated Index Management are referred to herein as auto indexes.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for facilitating dynamic selection and application for rules for query processing for large datasets in an on-demand services environment according to one embodiment. In one embodiment and by way of example, a method comprises determining processing patterns of a query based on historical performances associated with the query placed on behalf of a tenant in a multi-tenant environment, and dynamically applying one or more rules to the query for processing of the query within a predictable amount of time, where the one or more rules are dynamically selected from sets of rules based on the processing patterns. The method may further include executing the query based on the one or more rules by scanning one or more portions of a database having contents pertinent to the query and generating results by processing the query based on the contents and within the predictable amount of time.

Abstract: Optimized query plans may be generated independent of the query engine that performs the optimized query plan. A request to generate an optimized query plan for a query may be received and a type of engine for performing the query may be identified. An initial plan may be generated in an engine-specific format for the type of engine that is translated into an optimization plan format. An analysis of the initial plan optimization plan format may be performed to generate an optimized query plan. The optimized query plan may be translated into the engine-specific format and sent in response to the request for the optimized query plan.

Abstract: Joining data using a disjunctive operator using a lookup table is described. An example computer-implemented method can include receiving a query with a set of conjunctive predicates and a set of disjunctive predicates. The method may also include generating a lookup table for each predicate in the sets of conjunctive predicates and disjunctive predicates. The method, for each row in a probe-side table, may also further include looking up a value associated with that row in each of the lookup tables and adding the row to a results set when there is a match. Additionally, the method may also include returning the results set.

Abstract: Embodiments of the present invention provide an index establishment method and device. The method can include determining, according to index status information of a column in a database within a preset time threshold, whether an index is to be established for the column; determining an index type according to the data information of the column; and establishing the index for the column according to the index type based on the determination that the index is to be established for the column.

Abstract: Querying a data set formed from a version of primary data and secondary data is facilitated. First and second versions of primary data are stored in a primary data version store. Secondary data is received. The secondary data is stored in a secondary data store. A query language statement is received. The query language statement is executed by selecting query results from a data set that includes the secondary data and elements of the first version of primary data not inconsistent with the secondary data.

Abstract: Systems and methods are provided for loading data in connection with testing an application, based on a hierarchical framework. An exemplary method includes receiving a data query from a requestor, identifying a dependency of the data query on a data structure, and determining whether said data structure is annotated with a poor quality indicator. The exemplary method further includes providing a warning to the requestor when the data structure is annotated with the poor quality indicator, thereby informing the requestor of a potential quality issuer with a report relying on the data structure query.

Abstract: A data analysis platform may be based on database views. A build module may receive, from a source code repository, information about a modified definition of a view. The build module may identify schema objects on which the view depends and form instructions for creating the view and the schema objects. The instructions may be executed to form the updated version of the view in a schema space separate from a production schema space. A deployment pipeline may coordinate replacing the production version of the view with the new version in response to validating the new version of the view.

Abstract: An example operation may include one or more of identifying a query from a requesting entity, where the query requests access to one or more blockchains, converting the query to an expression tree, creating one or more expression tree variations based on the expression tree, the one or more expression tree variations provide one or more different expressions than the expression tree and a same result as the expression tree, determining access conformity between one or more expression tree variations and the expression tree, selecting an expression tree variation with a greatest conformity rating, performing the query using the expression tree variation with the greatest conformity rating, and providing query results to a requesting entity.

Abstract: Disclosed are systems and methods for managing queries on on-line advertisement data. The system includes a query engine device for receiving queries from and outputting query results to query client devices and a training engine for generating and adjusting a model for predicting an estimation of resource usage for execution of each received query based on each query's corresponding feature vector having values pertaining to the query and a system status. The query engine device is further configured to provide the estimation of resource usage for each query to the corresponding query client device and, in response, receive input from such corresponding query client device and specifying whether to proceed with the corresponding query. A database system receives input from each query's corresponding query client device as to whether to proceed with the query and, in response, initiates or inhibits execution of such query with respect to a database storage system.

Abstract: A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes a query optimizer for optimizing data queries. The optimizer estimates the column cardinality of a set of rows based on estimated column cardinalities of disjoint subsets of the set of rows. For a particular column, the actual column cardinality of the set of rows is the sum of the actual column cardinalities of the two subsets of rows. The optimizer creates two respective Bloom filters from the two subsets, and then combines them to create a combined Bloom filter using logical OR operations. The actual column cardinality of the set of rows is estimated using a computation from the combined Bloom filter.

Abstract: Various embodiments virtualize data across heterogeneous formats. In one embodiment, a plurality of heterogeneous data sources is received as input. A local schema graph including a set of attribute nodes and a set of type nodes is generated for each of the plurality of heterogeneous data sources. A global schema graph is generated based on each local schema graph that has been generated. The global schema graph comprises each of the local schema graphs and at least one edge between at least one of two or more attributes nodes and two or more type nodes from different local schema graphs. The edge indicates a relationship between the data sources represented by the different local schema graphs comprising the two or more attributes nodes based on a computed similarity between at least one value associated with each of the two or more attributes nodes.

Abstract: Embodiments of the present invention relate to elimination of blocks such as splits in distributed processing systems such as MapReduce systems using the Hadoop Distributed Filing System (HDFS). In one embodiment, a method of and computer program product for optimizing queries in distributed processing systems are provided. A query is received. The query includes at least one predicate. The query refers to data. The data includes a plurality of records. Each record comprises a plurality of values in a plurality of attributes. Each record is located in at least one of a plurality of blocks of a distributed file system. Each block has a unique identifier. For each block of the distributed file system, at least one value cluster is determined for an attribute of the plurality of attributes. Each value cluster has a range. The predicate of the query is compared with the at least one value cluster of each block.

Abstract: Techniques herein optimize sort-merge join method for a band join. In an embodiment, for a query comprising a query block specifying a join between a first table and a second table, a band join condition is detected between the first table and the second table. Once the band join condition in detected, an execution plan is generated and executed. The execution of the execution plan includes: for a first row of at least a subset of first sorted rows, scanning second rows from a set of second sorted rows, joining each of said second rows with said first row, and ceasing to scan when encountering a row from the second sorted rows that falls outside a bound of said band join condition. Techniques also include parallelizing a workload by overlapping the distribution of rows to the same slave process and computing cost and cardinality estimation for enhanced band join.

Abstract: A method includes receiving, from a client device, an enhanced database query for a union operation of a first query and at least a second query, parsing the enhanced database query to identify two or more parameters controlling handling of duplicate rows in the first query and the second query, evaluating the enhanced database query utilizing the parameters to generate a result table, and providing the result table to the client device.