Abstract: Intelligent memory brokering for multiple process instances, such as relational databases (e.g., SQL servers), reclaims memory based on value, thereby minimizing cost across instances. An exemplary solution includes: based at least on a trigger event, determining a memory profile for each of a plurality of process instances at a computing node; determining an aggregate memory profile, the aggregate memory profile indicating a memory unit cost for each of a plurality of memory units; determining a count of memory units to be reclaimed; identifying, based at least on the aggregate memory profile and the count of memory units to be reclaimed, a count of memory units to be reclaimed within each process instance so that a total cost is minimized to reclaim the determined count; and communicating, to each process instance having identified memory units to be reclaimed, a count of memory units to be reclaimed within the process instance.

Abstract: Methods, systems, and computer program products are provided for a compute cluster comprising placement and load balancing (PLB) logic that receives data (e.g., state metadata) relating to a service (e.g., database service) executing on the compute cluster, from a resource manager executing on the compute cluster, via a first API associated with the resource manager. The PLB logic receives second data from the service via a second API and determines whether a PLB action is indicated based on one of the second data or a combination of the first data and the second data. When a PLB action is indicated, the PLB logic sends a command to the resource manager to execute the PLB action. The PLB logic also receives queries from clients external to the compute cluster and may spawn a child PLB logic to offload PLB operations, respond to queries, or perform software validation in the child.

Abstract: Intelligent memory brokering for multiple process instances, such as relational databases (e.g., SQL servers), reclaims memory based on value, thereby minimizing cost across instances. An exemplary solution includes: based at least on a trigger event, determining a memory profile for each of a plurality of process instances at a computing node; determining an aggregate memory profile, the aggregate memory profile indicating a memory unit cost for each of a plurality of memory units; determining a count of memory units to be reclaimed; identifying, based at least on the aggregate memory profile and the count of memory units to be reclaimed, a count of memory units to be reclaimed within each process instance so that a total cost is minimized to reclaim the determined count; and communicating, to each process instance having identified memory units to be reclaimed, a count of memory units to be reclaimed within the process instance.

Abstract: Placement of a tenant database in an oversubscribed, database-as-a-service cluster comprised of a plurality of nodes is described. The placement may be based on per-node estimates of a probability of resource demand violation if the tenant database is placed on the node. Past resource usage of similar tenant databases subscribed to the cluster that are collected and stored as compressed traces may be used to obtain the estimates. In some examples, based on the estimates, a per-node expected number of resource violations is determined and compared across nodes, where the determined placement minimizes the number of resource violations. In other examples, when the tenant database is being placed in parallel with other tenant databases, a score assigned to each valid configuration for the placement may be modified based on the estimates, where the determined placement is the configuration having a lowest score.

Abstract: Methods for distributed histogram computation in a framework utilizing data stream sketches and samples are performed by systems and devices. Distributions of large data sets are scanned once and processed by a computing pool, without sorting, to generate local sketches and value samples of each distribution. The local sketches and samples are utilized to construct local histograms on which cardinality estimates are obtained for query plan generation of distributed queries against distributions. Local statistics of distributions are also merged and consolidated to construct a global histogram representative of the entire data set. The global histogram is utilized to determine a cardinality estimation for query plan generation of incoming queries against the entire data set. The addition of new data to a data set or distribution involves a scan of the new data from which new statistics are generated and then merged with existing statistics for a new global histogram.

Abstract: Methods for determining cache activity and for optimizing cache reclamation are performed by systems and devices. A cache entry access is determined at an access time, and a data object of the cache entry for a current time window is identified that includes a time stamp for a previous access and a counter index. A conditional counter operation is then performed on the counter associated with the index to increment the counter when the time stamp is outside the time window or to maintain the counter when the time stamp is within the time window. A counter index that identifies another counter for a previous time window where the other counter value was incremented for the previous cache entry access causes the other counter to be decremented. A cache configuration command to reclaim, or additionally allocate space to, the cache is generated based on the values of the counters.

Abstract: Methods for determining cache activity and for optimizing cache reclamation are performed by systems and devices. A cache entry access is determined at an access time, and a data object of the cache entry for a current time window is identified that includes a time stamp for a previous access and a counter index. A conditional counter operation is then performed on the counter associated with the index to increment the counter when the time stamp is outside the time window or to maintain the counter when the time stamp is within the time window. A counter index that identifies another counter for a previous time window where the other counter value was incremented for the previous cache entry access causes the other counter to be decremented. A cache configuration command to reclaim, or additionally allocate space to, the cache is generated based on the values of the counters.

Abstract: A solution is disclosed for memory management of serverless databases that includes: based at least on detecting a trigger event, determining whether memory is to be reclaimed; based at least on determining that memory is to be reclaimed, determining an amount of memory to be reclaimed; identifying memory to be reclaimed; and reclaiming the identified memory. Disclosed solutions are flexible, enabling customization of the aggressiveness and manner of memory reclamation. This permits users to specify a tailored balance point between performance and cost, for arrangements that bill users based on resource usage (e.g., memory consumed by a serverless database). In some examples, users specify a ramp-down parameter that is used to determine how much memory can be evicted in a particular reclamation event, time intervals (or another criteria) for triggering a reclamation event, and a definition for whether a cache is active.

Abstract: Methods for distributed histogram computation in a framework utilizing data stream sketches and samples are performed by systems and devices. Distributions of large data sets are scanned once and processed by a computing pool, without sorting, to generate local sketches and value samples of each distribution. The local sketches and samples are utilized to construct local histograms on which cardinality estimates are obtained for query plan generation of distributed queries against distributions. Local statistics of distributions are also merged and consolidated to construct a global histogram representative of the entire data set. The global histogram is utilized to determine a cardinality estimation for query plan generation of incoming queries against the entire data set. The addition of new data to a data set or distribution involves a scan of the new data from which new statistics are generated and then merged with existing statistics for a new global histogram.

Abstract: Architecture that enables a Database-as-a-Service (DaaS) to auto-scale container sizes on behalf of tenants. An abstraction is provided that enables tenants to reason about monetary budget and query latency, rather than resource provisioning. An auto-scaling module automatically determines a container size for a subsequent billing interval based on telemetry that comprises latencies (e.g., waits), resource utilizations, and available budget, for example. A set of robust signals are derived from database engine telemetry and combined to significantly improve accuracy of resource demand estimation for database workloads. In a more specific implementation, resource demands can be estimated for arbitrary SQL (structured query language) workloads in a relational database management system (RDBMS).

Abstract: Intelligent memory brokering for multiple process instances, such as relational databases (e.g., SQL servers), reclaims memory based on value, thereby minimizing cost across instances. An exemplary solution includes: based at least on a trigger event, determining a memory profile for each of a plurality of process instances at a computing node; determining an aggregate memory profile, the aggregate memory profile indicating a memory unit cost for each of a plurality of memory units; determining a count of memory units to be reclaimed; identifying, based at least on the aggregate memory profile and the count of memory units to be reclaimed, a count of memory units to be reclaimed within each process instance so that a total cost is minimized to reclaim the determined count; and communicating, to each process instance having identified memory units to be reclaimed, a count of memory units to be reclaimed within the process instance.

Abstract: Intelligent memory brokering for multiple process instances, such as relational databases (e.g., SQL servers), reclaims memory based on value, thereby minimizing cost across instances. An exemplary solution includes: based at least on a trigger event, determining a memory profile for each of a plurality of process instances at a computing node; determining an aggregate memory profile, the aggregate memory profile indicating a memory unit cost for each of a plurality of memory units; determining a count of memory units to be reclaimed; identifying, based at least on the aggregate memory profile and the count of memory units to be reclaimed, a count of memory units to be reclaimed within each process instance so that a total cost is minimized to reclaim the determined count; and communicating, to each process instance having identified memory units to be reclaimed, a count of memory units to be reclaimed within the process instance.

Abstract: Intelligent memory brokering for multiple process instances, such as relational databases (e.g., SQL servers), reclaims memory based on value, thereby minimizing cost across instances. An exemplary solution includes: based at least on a trigger event, determining a memory profile for each of a plurality of process instances at a computing node; determining an aggregate memory profile, the aggregate memory profile indicating a memory unit cost for each of a plurality of memory units; determining a count of memory units to be reclaimed; identifying, based at least on the aggregate memory profile and the count of memory units to be reclaimed, a count of memory units to be reclaimed within each process instance so that a total cost is minimized to reclaim the determined count; and communicating, to each process instance having identified memory units to be reclaimed, a count of memory units to be reclaimed within the process instance.

Abstract: A solution is disclosed for memory management of serverless databases that includes: based at least on detecting a trigger event, determining whether memory is to be reclaimed; based at least on determining that memory is to be reclaimed, determining an amount of memory to be reclaimed; identifying memory to be reclaimed; and reclaiming the identified memory. Disclosed solutions are flexible, enabling customization of the aggressiveness and manner of memory reclamation. This permits users to specify a tailored balance point between performance and cost, for arrangements that bill users based on resource usage (e.g., memory consumed by a serverless database). In some examples, users specify a ramp-down parameter that is used to determine how much memory can be evicted in a particular reclamation event, time intervals (or another criteria) for triggering a reclamation event, and a definition for whether a cache is active.

Abstract: Architecture that enables a Database-as-a-Service (DaaS) to auto-scale container sizes on behalf of tenants. An abstraction is provided that enables tenants to reason about monetary budget and query latency, rather than resource provisioning. An auto-scaling module automatically determines a container size for a subsequent billing interval based on telemetry that comprises latencies (e.g., waits), resource utilizations, and available budget, for example. A set of robust signals are derived from database engine telemetry and combined to significantly improve accuracy of resource demand estimation for database workloads. In a more specific implementation, resource demands can be estimated for arbitrary SQL (structured query language) workloads in a relational database management system (RDBMS).

Abstract: Architecture that enables a Database-as-a-Service (DaaS) to auto-scale container sizes on behalf of tenants. An abstraction is provided that enables tenants to reason about monetary budget and query latency, rather than resource provisioning. An auto-scaling module automatically determines a container size for a subsequent billing interval based on telemetry that comprises latencies (e.g., waits), resource utilizations, and available budget, for example. A set of robust signals are derived from database engine telemetry and combined to significantly improve accuracy of resource demand estimation for database workloads. In a more specific implementation, resource demands can be estimated for arbitrary SQL (structured query language) workloads in a relational database management system (RDBMS).

Abstract: Architecture that enables a Database-as-a-Service (DaaS) to auto-scale container sizes on behalf of tenants. An abstraction is provided that enables tenants to reason about monetary budget and query latency, rather than resource provisioning. An auto-scaling module automatically determines a container size for a subsequent billing interval based on telemetry that comprises latencies (e.g., waits), resource utilizations, and available budget, for example. A set of robust signals are derived from database engine telemetry and combined to significantly improve accuracy of resource demand estimation for database workloads. In a more specific implementation, resource demands can be estimated for arbitrary SQL (structured query language) workloads in a relational database management system (RDBMS).

Abstract: Incremental query results and confidence interval values associated with respective incremental query results may be obtained. Visualization shape objects indicating uncertainty values may be determined, based on mapping values of respective incremental query results and confidence interval values to points in the associated visualization shape objects, the uncertainty values visualized based on proportional shapes of the visualization shape objects. At least one visualization comparison object representing a comparison of a plurality of distributions associated with the obtained incremental query results and confidence interval values may be determined. Display of the plurality of visualization shape objects and the at least one visualization comparison object may be initiated.

Abstract: Hash values corresponding to a file are processed in windows to determine a minimum hash value for each window. Each window may begin at a minimum hash value determined for a previous window and end after a fixed number of hash values. If a hash value is less than a threshold hash value, it is added to a buffer that is used to store the hash values in sorted order for a current window. If a hash value is greater than the threshold, it is added to another buffer whose hash values are not stored in sorted order. At the end of the current window, the minimum hash value in the first buffer is selected as the landmark for the window. If the first buffer is empty, then the hash values in the other buffer are sorted and the minimum hash value is selected as the landmark for the window.

Abstract: The subject disclosure is directed towards simulating query execution to provide incremental visualization for a global data set. A data store may be configured for searching at least a portion of a global data set being stored at an enterprise-level data store. In response to a user-issued query, partial query results are provided to a front-end interface for display to the user. The front-end interface also provides statistical information corresponding to the partial query results in relation to the global data set, which may be used to determine when a current set of query results becomes acceptable as a true/accurate estimate.