Abstract: Time-series columnar-based information is received and indexed in a compute infrastructure for cost-effective cloud-based object storage. The approach leverages a file format that enables highly-performant search and retrieval of the data stored in the cloud. In operation, an indexer receives the time-series information, indexes that information according to the file format, and forwards the indexed information for storage to the object store, where it is stored as a set of time-based partitions. A partition comprises a set of files, namely, a manifest file, a data file, and an index file. These files are structured as a compact instance of a set of raw unstructured data that comprises the given partition. Highly-performant information retrieval is enabled in response to a time-bounded query, because operations at a query peer (with respect to one or more partitions) are carried out in real-time during query processing and without requiring retrieval of the data file as a whole.

Abstract: The present disclosure involves systems, software, and computer implemented methods for configuring graph query parallelism for high system throughput. One example method includes receiving a query to be executed against a graph database. System properties are determined of a system in which the query is to be executed. Algorithmic properties are determined of at least one algorithm to be used to execute the query. Graph data statistics are determined for the graph database. Graph traversal estimations are determined for a first iteration of the graph query and an estimated cost model is determined for the first iteration based on the graph traversal estimations. Estimated thread boundaries are determined for performing parallel execution of the first iteration. Work packages of vertices to be processed during the execution of the first iteration are determined based on the first estimated cost model and the work packages are provided to a work package scheduler.

Abstract: Context dependent execution time prediction may be applied to redirect queries to additional query processing resources. A query to a database may be received at a first query engine. A prediction model for executing queries at the first query engine may be applied to determine predicted query execution time for the first query engine. A prediction model for executing queries at a second query engine may also be applied to determine predicted query execution time for the second query engine. One of the query engines may be selected to perform the query based on a comparison of the predicted query execution times.

Abstract: A method dynamically selects query execution operators. A database engine receives a query, parses the query to form a query execution tree, and compiles the tree to form a first executable plan that includes in-memory operators. The database engine executes the first plan, including executing in-memory operators in parallel. While executing a first in-memory operator, insufficient memory is detected. In response, the database engine aborts the execution, and recompiles the query tree in two ways, forming a second executable plan that replaces the first in-memory operator with a first spooling operator. The first spooling operator executes within a fixed volatile memory budget and swaps to non-volatile memory according to the budget. A third executable plan retains the first in-memory operator, but schedules it to run serially. The database engine selects either the second plan or the third plan, and executes the selected plan to return results for the query.

Abstract: A system enables an arbitrary number of items to be indexed in a geographic region that provides a predictable query response time across a sharded database. Items indexed to the geographic region are stored on a single shard and additional items are added to that shard as long as an overflow condition indicative of undesirable query response times is not met. If the overflow condition is met the system expands the storage of items indexed to the geographic region to one or more additional shards in order to maintain predictable query response times. The system may maintain a shard count representing the total number of shards being used to store items corresponding to a geographic region, which can be used to query one or more relevant shards. The system may apply deterministic hashing in order to evenly distribute shards across database nodes of the sharded database.

Abstract: The present disclosure provides a system for performing a method for scaling one or more query processes elastically. The method includes a first step of receiving a request for running the one or more query processes concurrently. In addition, the method includes another step of pre-determining amount of space required for efficient operation and processing of the one or more query processes. Further, the method includes yet another step of creating one or more virtual spaces in a query space outside the data space for efficient running of the one or more query processes. Furthermore, the method includes yet another step of decompressing compressed data stored in the one or more memory regions of the data space requested by the one or more query processes. Moreover, the method includes yet another step of compressing decompressed data present in the virtual space of the one or more virtual spaces.

Abstract: Performance measures are predicted for queries to prioritize query performance at a database system. A trained machine learning model for the database system may be applied to a query to determine a predicted performance measure for the query. The predicted performance measure may be compared with other predicted performance measures for other waiting queries to determine a priority for executing the query.

Abstract: Techniques are described to improve query evaluation in computer systems. In an embodiment, a system receives a full text query for evaluation against a collection of hierarchically marked data object sets. The query specifies token(s) and context(s) which indicate hierarchical location(s) to match within a queried hierarchical data structure. To evaluate the query, the system determines a) data object set(s) that contain the query specified token(s) using token list(s), and/or b) data object set(s) that contain the query specified context(s) using label list(s).

Abstract: Query execution status may be tracked to selectively route queries to resources for execution. The completion of queries executing at computing resources obtained from a pool of computing resources configured to execute queries may be detected. Instead of returning the computing resources to the pool, the computing resources may be identified as available in resource management data. When another query is received, the resource management data may be evaluated to select an available computing resource. The query may then be routed to the selected computing resource for execution.

Abstract: A method dynamically selects query execution operators. A database engine receives a query, parses the query to form a query execution tree, and compiles the tree to form a first executable plan that includes in-memory operators. The database engine executes the first plan, including executing in-memory operators in parallel. While executing a first in-memory operator, insufficient memory is detected. In response, the database engine aborts the execution, and recompiles the query tree in two ways, forming a second executable plan that replaces the first in-memory operator with a first spooling operator. The first spooling operator executes within a fixed volatile memory budget and swaps to non-volatile memory according to the budget. A third executable plan retains the first in-memory operator, but schedules it to run serially. The database engine selects either the second plan or the third plan, and executes the selected plan to return results for the query.

Abstract: The present disclosure generally relates to a cost-based optimizer for efficiently processing data through the use of multiple different data processing platforms. The cost-based optimizer may receive an input plan for processing data that includes a number of base operators. The cost-based optimizer may then determine execution operators for each base operator, where each execution operator corresponds to a different data processing platform. From the execution operators, the cost-based optimizer may determine possible subplans for executing the input plan on one or more data processing platforms. The cost-based optimizer may determine the cost of executing each possible subplan and choose the subplan with the lowest cost.

Abstract: A blockchain node receives a service request, where the service request comprises one or more data types and respective service data corresponding to the one or more data types that are stored in a blockchain. At least one of a service type or identification information is determined corresponding to the service request. The service request is parsed to obtain each data type of the service request and service data corresponding to each data type. Based on a mapping relationship between a data type and service data, the service data that is obtained through parsing in a relational database corresponding to the blockchain node is stored.

Abstract: A determination is made that a request associated with an application is a candidate for blockchain-based deferral. An insertion of a record representing at least the request at a blockchain is initiated. One or more records representing the request are accessed to perform an operation indicated in the request.

Abstract: A computer-implemented method includes identifying a query, including one or more predicates and one or more branches, wherein one or more branches includes one or more legs. The computer-implemented method further includes, for each branch, in parallel: determining a risk, determining a return row threshold, estimating a number of return rows; terminating access if the return rows exceed the threshold. The computer-implemented method further includes, for each leg, in parallel: determining a leg return row threshold; accessing the leg; fetching one or more return rows into one or more leg return row pages; terminating access if the return rows exceed the threshold; intersecting one or more leg return row pages into one or more intersected leg return row pages; and applying the one or more predicates to the one or more intersected leg return row pages. The method may be embodied in a corresponding computer system or computer program product.

Abstract: A method and apparatus for “ring fencing” an operating system, with any apps or programs running on that operating system, in such a way that all operations of the apps, programs, and operating system use real resources only with the permission or through the intermediation of a resource allocator programmed to allow only secure access to a known finite set of resources. The list of kinds of resources is short, and the list of resources is under the explicit control of the resource allocator, which is programmed with Communicating Sequential Processes techniques so as to give formally verifiable assurance of predefined security requirements. At the same time, the full features of the operating system are supported with minimal changes in its kernel and no changes in app or program binary code that accesses real resources via calls or bytecode that drives dynamic system libraries or a runtime environment.

Abstract: Techniques are described herein for generating and using in-memory data structures to represent columns in data block sets. In an embodiment, a database management system (DBMS) receives a query for a target data set managed by the DBMS. The query may specify a predicate for a column of the target data set. The predicate may include a filtering value to be compared with row values of the column of the target data set. Prior to accessing data block sets storing the target data set from persistent storage, the DBMS identifies an in-memory summary that corresponds to a data block set, in an embodiment. The in-memory summary may include in-memory data structures, each representing a column stored in the data block set. The DBMS determines that a particular in-memory data structure exists in the in-memory summary that represents a portion of values of the column indicated in the predicate of the query.

Abstract: A method for accessing information. A computer system receives a request for a piece of the information stored in a location between client computer systems and sources of the information. The information has formats for the information in the location. The formats for the information are controlled by the sources of the information. Each of the sources is assigned to control a group of portions of the information in the location. The computer system identifies the piece of the information corresponding to the request. The computer system returns a response to the request for the piece of the information, enabling simplifying access to the information in different formats.

Abstract: Techniques are described herein for generating and using in-memory data structures to represent columns in data block sets. In an embodiment, a database management system (DBMS) receives a query for a target data set managed by the DBMS. The query may specify a predicate for a column of the target data set. The predicate may include a filtering value to be compared with row values of the column of the target data set. Prior to accessing data block sets storing the target data set from persistent storage, the DBMS identifies an in-memory summary that corresponds to a data block set, in an embodiment. The in-memory summary may include in-memory data structures, each representing a column stored in the data block set. The DBMS determines that a particular in-memory data structure exists in the in-memory summary that represents a portion of values of the column indicated in the predicate of the query.

Abstract: A query is received by a database server from a remote application server. The query encapsulates an on-the-fly calculation scenario that defines a data flow model that includes one or more calculation nodes. Thereafter, the database server instantiates the on-the-fly calculation scenario. The database server then executes the operations defined by the calculation nodes of the instantiated calculation scenario to result in a responsive data set so that the database server can provide the data set to the application server. Related apparatus, systems, methods, and articles are also described.

Abstract: Described are techniques for performing system management such as a system including one or more data storage systems or appliances. User input may be received by a chat processing engine. The user input may be a natural language input to perform a request. The user input may be received from a management application used to perform data storage management. The chat processing engine may perform first processing of the user input to determine a response to the request. The response may be sent to the management application. The natural language input from a user may be in the form of text or voice that is converted to text for processing by the chat processing engine.

Abstract: A query is received by a database server from a remote application server. The query is associated with a calculation scenario that defines a data flow model that includes one or more calculation nodes and that includes at least one hierarchy filter. Each hierarchy filter is a filter of at least one hierarchy table. Each calculation node defines one or more operations to execute on the database server. Thereafter, the database server instantiates the calculation scenario. The instantiating includes optimizing the calculation scenario by transforming at least one hierarchy filter and pushing down the at least one transformed hierarchy filter to a lower calculation node. The database server then executes the operations defined by the calculation nodes of the optimized calculation scenario to result in a responsive data set. The database server can subsequently provide the data set to the application server.

Abstract: The streams environment includes a plurality of operators coupled with processing elements including a first processing element coupled with a first operator instructed with a first programming instructions, and a second processing element coupled with a second operator instructed with a second programming instructions. A workload of the first processing element and a workload of the second processing element are measured. A first threshold of the workload of the first processing element, and second threshold of the workload of the second processing element are determined. The first programming instructions and the second programming instructions are compared to determine if the first operator and the second operator are susceptible to fusion. The first operator is de-coupled and fused to the second processing element, in response to determining the first threshold and the determination that the first operator and the second operator are susceptible to fusion.

Abstract: The streams environment includes a plurality of operators coupled with processing elements including a first processing element coupled with a first operator instructed with a first programming instructions, and a second processing element coupled with a second operator instructed with a second programming instructions. A workload of the first processing element and a workload of the second processing element are measured. A first threshold of the workload of the first processing element, and second threshold of the workload of the second processing element are determined. The first programming instructions and the second programming instructions are compared to determine if the first operator and the second operator are susceptible to fusion. The first operator is de-coupled and fused to the second processing element, in response to determining the first threshold and the determination that the first operator and the second operator are susceptible to fusion.

Abstract: A method of analyzing the performance of a query optimizer includes identifying an event trigger. A reproduction object characterizing operational parameters of the customer computer at the time of the event trigger is populated. The reproduction object is transported from the customer computer to a test computer. The reproduction object is analyzed at the test computer to characterize the performance of a query optimizer.

Abstract: A method, system, and computer program product to generate results for a query to an encrypted database stored on a host are described. The method includes generating indexes from the encrypted database, each index identifying records of the encrypted database associated with a range of data for at least one field stored in the records of the encrypted database, and generating index metadata associated with each index, the index metadata indicating the range of data identified by the associated index. The method also includes generating a sub-query from the query for each field associated with the query and determining a subspace of search within the encrypted database based on sub-query results obtained through the index metadata. The method further includes searching the subspace of the encrypted database to generate the results of the query.

Abstract: A computing cluster operated according to a resource allocation policy based on a predictive model of completion time. The predictive model may be applied in a resource control loop that iteratively updates resources assigned to an executing job. At each iteration, the amount of resources allocated to the job may be updated based on of the predictive model so that the job will be scheduled to complete execution at a target completion time. The target completion time may be derived from a utility function determined for the job. The utility function, in turn, may be derived from a service level agreement with service guarantees and penalties for late completion of a job. Allocating resources in this way may maximize utility for an operator of the computing cluster while minimizing disruption to other jobs that may be concurrently executing.

Abstract: A method, system, and computer program product to generate results for a query to an encrypted database stored on a host are described. The system includes a host comprising a storage device to store the encrypted database, and a a secure processor to generate indexes and index metadata from the encrypted database, each index identifying records of the encrypted database associated with a range of data for at least one field stored in the records of the encrypted database and the metadata indicating the range of data identified by the associated index. The system also includes an interface of the host to receive the query, and a host processor to generate a sub-query form the query for each field associated with the query. Based on sub-query results obtained through the index metadata, the secure processor searches a subspace of the encrypted database to generate the results of the query.

Abstract: Reroutable data columns are identified in an ETL process by receiving an ETL process definition describing a set of processing stages and how each processing stage output data column is a result of a function that operates on a set of input data columns, representing the ETL process definition as a directed graph with nodes representing processing stages and links representing data flow between processing stages, traversing at least part of the directed graph and identifying a set of subsequent nodes of the directed graph where at least one data column is involved only as input data in identity functions, the at least one data column being reroutable between outmost nodes of the set of subsequent nodes, and in connection with traversing the at least part of the directed graph, maintaining information about reroutable data columns and the respective outmost nodes.

Abstract: Systems and computer program products for identifying symbolic links in network file systems are provided herein. An absolute path may be determined at a network file server. This may include determining a complete client path from an initial client path and combining the complete client path with a server export path. Once the absolute path is determined, it may be traversed using a file descriptor of each file in the absolute path to identify a symbolic link.

Abstract: An electronic device for media file caching for conserving resources is described. The electronic device includes a processor and instructions stored in memory. The electronic device monitors requests for a media file, stores media file information and determines whether the media file should be stored locally based on the media file information. If it is determined that the media file should be stored, the electronic device saves the media file and presents the media file.

Abstract: The invention identifies and caches query plans in a shared-nothing distributed data store that are unlikely to become invalid because they do not reference objects that are likely to be changed or deleted. Plans that are likely to become invalid and are not cached are those plans that reference data that is partitioned across segment/query execution nodes of the data store, plans that are complex, and plans that reference objects that are not “built-in” (primitive) objects. The effect is that most plans which are generated on a query dispatch (master) node are not cached, whereas most plans generated on an execution (segment) node are cached.

Abstract: In an embodiment, a recursive query is received that comprises a first select statement with a seed select statement and a second select statement with a recursive reference, wherein the recursive query further identifies at least two columns in at least one table, wherein the at least two columns have parent-child relationships represented by nodes in a graph, wherein the graph represents the organization of values in rows in the at least one table. A target recursion depth is calculated for the graph based on a fanout of the graph. In an embodiment, the target recursion depth is calculated by summing the fanout at each recursion depth of the graph multiplied by a number of nodes at each recursion depth of the graph. An estimated number of rows that the recursive query will return is estimated based on the target recursion depth.

Abstract: A query plan predictor receives a query instance according to a query template having a set of parameters. The query plan predictor determines a plan space point for the query instance which is based upon the set of parameters. The query plan predictor predicts a predicted query plan for the plan space point, which is selected from a map of query plans according to a confidence threshold from a plan cache that includes at least one histogram. Each histogram is representative of a cached query plan in the plan cache combined with a previously-selected locality-sensitive hashing function. The histogram stores a frequency distribution of a mapping of sample plan points for the cached query plan by the locality-sensitive hashing function.

Abstract: A method, computer program product, and computer system for shared execution of mixed data flows, performed by one or more computing devices, comprises identifying one or more resource sharing opportunities across a plurality of parallel tasks. The plurality of parallel tasks includes zero or more relational operations and at least one non-relational operation. The plurality of parallel tasks relative to the relational operations and the at least one non-relational operation are executed. In response to executing the plurality of parallel tasks, one or more resources of the identified resource sharing opportunities is shared across the relational operations and the at least one non-relational operation.

Abstract: A method, computer program product, and computer system for shared execution of mixed data flows, performed by one or more computing devices, comprises identifying one or more resource sharing opportunities across a plurality of parallel tasks. The plurality of parallel tasks includes zero or more relational operations and at least one non-relational operation. The plurality of parallel tasks relative to the relational operations and the at least one non-relational operation are executed. In response to executing the plurality of parallel tasks, one or more resources of the identified resource sharing opportunities is shared across the relational operations and the at least one non-relational operation.

Abstract: A method is provided for identifying documents that include a searchable form relevant to a topic. A document is received. If the received document comprises a form is determined. A form includes a field presented to a user requesting information from the user. If the received document is determined to comprise a form, a determination is made concerning whether or not the form is a searchable form. A searchable form returns non-trivial information to a requester in response to a submission of the form. If the form is determined to be a searchable form, a determination is made concerning whether or not the form is relevant to an identified topic. If the form is determined to be relevant to the identified topic, the document is identified as a searchable form relevant to the identified topic.

Abstract: In an embodiment, a first query is received that specifies a first object identifier. If a first query plan exists that implements the first query, except that the first query plan does not comprise the first object identifier and instead comprises a parameter marker in place of the first object identifier, a first query execution plan is created from the first query plan, substituting the first object identifier in the first query execution plan for the parameter marker, and the first query execution plan is executed to read a first object identified by the first object identifier.

Abstract: Both live and offline transcoding of media is managed for storage in a cache. Requests for media are made by a client on a network and media taken from the cache and transcoded for delivery to the client. Media is also transcoded and stored in memory during idle request in anticipation of future requests during idle transcoder time between client requests. A prioritizer manages at least offline transcoding requests. A high priority queue operates at a higher priority than the low priority queue. The high priority queue receives live transcoding requests from a client on the network. A low priority queue receives offline transcoding requests. A media transcoder primarily transcodes media in accordance with requests from the high priority queue for delivery to a client on the network and then secondarily transcodes media in accordance with requests from the low priority queue when the high priority queue is empty.

Abstract: A query governor intelligently sets tailored thresholds for a query accessing a computer database. The query governor preferably generates a tailored threshold for each query sent to the database for execution. The tailored threshold for the query is preferably compared to an estimated query execution time to determine whether to execute the query. The query governor uses one or more factors applied to a standard threshold to generate the tailored threshold. The factors preferably include user factors and query factors. These factors are dynamically adjusted by the query governor in an intelligent way to increase optimal use of the database. Other factors may include factors such as job priority factor, resource factor and an application factor.

Abstract: A method and apparatus for generating search results including searching by subdomain and providing sponsored results by subdomain is provided. A search system according to embodiments of the present invention analyzes search queries to determine if they are to be routed to subdomains and presents results include sponsored hits sponsored on a subdomain by subdomain basis.

Abstract: An elastic parallel database system where metadata is specified out-of-band during database operations via a set of augmentation rules. The rules are used to augment or modify commands received, and indicate whether they are to be used for specific connections on which they are received, for all client connections, or some other scope.

Abstract: A data source, such as a database, is modeled using state variables, state, and actions performed on the data source. State variable queries may be automatically generated by generating queries or functions based on intercepting data queries to a data source and using the intercepted data query language. State variable queries may automatically inspect data objects within the data store and automatically generate state variable values. The state variable queries or functions may be executed automatically after an action is run against a data source such that state information is automatically determined. Further, a graphical user interface used to modify, add and delete state information and automatic state variable queries associated with the verification model is described.

Abstract: A master monitor spawns a local monitor in each node of a server cluster. Each local monitor, responsive to a database event, determines a metric of a performance-related variable. A maximum period between measurements may be imposed to reduce irrelevant results, as may a minimum period. The metric may be stored in a measurement file on the respective node and/or may be reported to the master monitor. The master monitor may make any reported metrics available to a user via a front end.

Abstract: An image processing apparatus stores therein a use charging table that shows a charging condition with respect to use of the image processing apparatus and a cooperative job charging table that shows a charging condition with respect to execution of a cooperative job. In the case where a received job is an apparatus job to be executed only by the image processing apparatus, the use charging table is read. Charging processing is performed based on the read charging table, and a remaining amount is calculated. In the case where a received job is a cooperative job to be executed by the image processing apparatus and a service providing apparatus cooperatively with each other, the cooperative job charging table is read, and charging processing is performed with reference to the read cooperative job charging table as necessary.

Abstract: A method, apparatus and computer program product are provided for performing a query using a decision diagram. In this regard, in response to receiving a query instruction a compression value for a query result decision diagram can be determined. A compression value for a compressed query result set may also be determined. A compression ratio can be determined between the compression value for the query result decision diagram and the compression value for a compressed query result set. A query result decision diagram may be generated based on the query instruction and a relationship between the compression ratio and a threshold ratio. The query result decision diagram may also be transmitted based upon a relationship between the compression ratio and a threshold ratio.

Abstract: A computer-implemented method, apparatus and article of manufacture for optimizing a database query. A query execution plan is generated for the database query, which may be stored in an in-line query capture database. Execution of the query execution plan may be monitored in order to generate performance information, which also may be stored in the in-line query capture database. The data stored in in-line query capture database may be used during the initial generation of query execution plans, as well as for the re-optimization of a query execution plan, during the execution of the query execution plan.

Abstract: Method and apparatus for stress management in a searchable data service. The searchable data service may provide a searchable index to a backend data store, and an interface to build and query the searchable index, that enables client applications to search for and retrieve locators for stored entities in the backend data store. Embodiments of the searchable data service may implement a distributed stress management mechanism that may provide functionality including, but not limited to, the automated monitoring of critical resources, analysis of resource usage, and decisions on and performance of actions to keep resource usage within comfort zones. In one embodiment, in response to usage of a particular resource being detected as out of the comfort zone on a node, an action may be performed to transfer at least part of the resource usage for the local resource to another node that provides a similar resource.

Abstract: There is provided a computer-implemented method of executing a query plan against a database. An exemplary method comprises accessing a first subset of rows of a database table using a direct access method for an index. The query plan may comprise the direct access method. The exemplary method also comprises determining a processing cost of accessing the first subset of rows. The exemplary method further comprises modifying the direct access method for the index in response to determining that the processing cost exceeds a specified threshold. Additionally, the exemplary method comprises accessing a second subset of rows of the database table using the modified direct access method.

Abstract: An optimizer uses comprehensive reasoning regarding partitioning, sorting, and grouping properties for query optimization. When optimizing an input query expression, logical exploration generates alternative logical expressions. Physical optimization explores physical operator alternatives for logical operators. Required partitioning, sorting, and grouping properties of inputs to physical operators are determined. Additionally, delivered partitioning, sorting, and grouping properties of outputs from physical operators are determined. In some embodiments, enforcer rules are employed to modify structural property requirements to introduce alternatives for consideration. Property matching identifies valid execution plans in which the delivered partitioning, sorting, and grouping properties satisfy corresponding required partitioning, sorting, and grouping properties. An execution plan having the lowest cost is selected as the optimized execution plan.

Abstract: A database access model and storage structure that efficiently support concurrent OLTP and OLAP activity independently of the data model or schema used, are described. The storage structure and access model presented avoid the need to design schemas for particular workloads or query patterns and avoid the need to design or implement indexing to support specific queries. Indeed, the access model presented is independent of the database model used and can equally support relational, object and hierarchical models amongst others.