Abstract: Disclosed herein are system, method, and computer program product embodiments for adaptive caching for hybrid columnar databases with heterogeneous page sizes. An embodiment operates by scanning one or more pools comprising one or more pages of the same size in a buffer cache. The embodiment determines an increment of a reuse rate for the pools in the buffer cache within a time interval. The embodiment determines a cumulative reuse rate that is the sum of the increments of the reuse rate over several time intervals. The embodiment determines a gliding average reuse rate of the cumulative reuse rate over several time intervals. The embodiment compares the average reuse rates of the plurality of the pools to a threshold to dynamically determine whether a pool should reuse memory from the existing pages of the same pool or rebalance memory from one or more victim pools.

Abstract: Disclosed herein are system, method, and computer program product embodiments for adaptive caching for hybrid columnar databases with heterogeneous page sizes. An embodiment operates by receiving a request to load a new page of memory from a disk in a buffer cache. The embodiment scans one or more pools comprising one or more pages of the same size in a buffer cache. The embodiment determines an increment of a reuse rate for the pools in the buffer cache within a time interval. The embodiment determines a cumulative reuse rate that is the sum of the increments of the reuse rate over several time intervals. The embodiment determines a gliding average reuse rate of the cumulative reuse rate over several time intervals. The embodiment compares the average reuse rates of the plurality of the pools to a threshold to dynamically determine whether a pool should reuse memory from the existing pages of the same pool or rebalance memory from one or more victim pools.

Abstract: System, method, and various embodiments for providing a metadata based memory management system are described herein. An embodiment operates by determining metadata of a table of a database indicating whether data of the table is to be stored in memory or disk storage, wherein the table comprises a plurality of columns and a plurality of rows. A first column of the table is identified. A hierarchy of the metadata for the table including a column value, partition value, and table value is determined, wherein the column value is prioritized over the partition value, and wherein the partition value is prioritized over the table value. The data of the table stored in the column is loaded from the disk storage into the memory based on the hierarchy. A query is processed based on the data loaded into the memory.

Abstract: System, method, and various embodiments for providing a metadata based memory management system are described herein. An embodiment operates by determining metadata of a table of a database indicating whether data of the table is to be stored in memory or disk storage, wherein the table comprises a plurality of columns and a plurality of rows. A first column of the table is identified. A hierarchy of the metadata for the table including a column value, partition value, and table value is determined, wherein the column value is prioritized over the partition value, and wherein the partition value is prioritized over the table value. The data of the table stored in the column is loaded from the disk storage into the memory based on the hierarchy. A query is processed based on the data loaded into the memory.

Abstract: A computer generates labels for machine learning algorithms by retrieving, from a data storage circuit, multiple label sets that contain labels that each classify data points in a corpus of data. A graph is generated that includes a plurality of edges, each edge between two respective labels from different label sets of the multiple label sets. Weights are determined for the plurality of edges based upon a consistency between data points classified by two labels connected by the edges. An algorithm is applied that groups labels from the multiple label sets based upon the weights for the plurality of edges. Data points are identified from the corpus of data that represent conflicts within the grouped labels. An electronic message is transmitted in order to present the identified data points to entities for further classification. A new label set is generated using the further classification received from the entities.

Abstract: Disclosed herein are system, method, and computer program product embodiments for identifying and loading a relevant page of a dictionary into temporary memory. An embodiment operates by receiving a query to be executed. The query includes a value for executing the query. The server queries a dictionary to retrieve a value ID. The server executes a binary search on a helper vector of the dictionary based on the value. The helper vector includes a last value for each page of a dictionary. The server identifies a page of the dictionary including the value. The server loads the page into temporary memory and retrieves the value ID of the value from the page. The server executes the query on a column using the value ID.

Abstract: A computer generates labels for machine learning algorithms by retrieving, from a data storage circuit, multiple label sets that contain labels that each classify data points in a corpus of data. A graph is generated that includes a plurality of edges, each edge between two respective labels from different label sets of the multiple label sets. Weights are determined for the plurality of edges based upon a consistency between data points classified by two labels connected by the edges. An algorithm is applied that groups labels from the multiple label sets based upon the weights for the plurality of edges. Data points are identified from the corpus of data that represent conflicts within the grouped labels. An electronic message is transmitted in order to present the identified data points to entities for further classification. A new label set is generated using the further classification received from the entities.

Abstract: A computer generates labels for machine learning algorithms by retrieving, from a data storage circuit, multiple label sets that contain labels that each classify data points in a corpus of data. A graph is generated that includes a plurality of edges, each edge between two respective labels from different label sets of the multiple label sets. Weights are determined for the plurality of edges based upon a consistency between data points classified by two labels connected by the edges. An algorithm is applied that groups labels from the multiple label sets based upon the weights for the plurality of edges. Data points are identified from the corpus of data that represent conflicts within the grouped labels. An electronic message is transmitted in order to present the identified data points to entities for further classification. A new label set is generated using the further classification received from the entities.

Abstract: Identification of an intent of a conversation can be useful for real-time or post-processing purposes. According to example embodiments, a method, and corresponding apparatus of identifying at least one intent-bearing utterance in a conversation, comprises determining at least one feature for each utterance among a subset of utterances of the conversation; classifying each utterance among the subset of utterances, using a classifier, as an intent classification or a non-intent classification based at least in part on a subset of the at least one determined feature; and selecting at least one utterance, with intent classification, as an intent-bearing utterance based at least in part on classification results by the classifier. Through identification of an intent bearing utterance, a call center for example, can provide improved service for callers through, for example, more effective directing of a call to a live agent.

Abstract: A computer generates labels for machine learning algorithms by retrieving, from a data storage circuit, multiple label sets that contain labels that each classify data points in a corpus of data. A graph is generated that includes a plurality of edges, each edge between two respective labels from different label sets of the multiple label sets. Weights are determined for the plurality of edges based upon a consistency between data points classified by two labels connected by the edges. An algorithm is applied that groups labels from the multiple label sets based upon the weights for the plurality of edges. Data points are identified from the corpus of data that represent conflicts within the grouped labels. An electronic message is transmitted in order to present the identified data points to entities for further classification. A new label set is generated using the further classification received from the entities.

Abstract: A computer generates labels for machine learning algorithms by retrieving, from a data storage circuit, multiple label sets that contain labels that each classify data points in a corpus of data. A graph is generated that includes a plurality of edges, each edge between two respective labels from different label sets of the multiple label sets. Weights are determined for the plurality of edges based upon a consistency between data points classified by two labels connected by the edges. An algorithm is applied that groups labels from the multiple label sets based upon the weights for the plurality of edges. Data points are identified from the corpus of data that represent conflicts within the grouped labels. An electronic message is transmitted in order to present the identified data points to entities for further classification. A new label set is generated using the further classification received from the entities.

Abstract: A computer generates labels for machine learning algorithms by retrieving, from a data storage circuit, multiple label sets that contain labels that each classify data points in a corpus of data. A graph is generated that includes a plurality of edges, each edge between two respective labels from different label sets of the multiple label sets. Weights are determined for the plurality of edges based upon a consistency between data points classified by two labels connected by the edges. An algorithm is applied that groups labels from the multiple label sets based upon the weights for the plurality of edges. Data points are identified from the corpus of data that represent conflicts within the grouped labels. An electronic message is transmitted in order to present the identified data points to entities for further classification. A new label set is generated using the further classification received from the entities.

Abstract: A computer generates labels for machine learning algorithms by retrieving, from a data storage circuit, multiple label sets that contain labels that each classify data points in a corpus of data. A graph is generated that includes a plurality of edges, each edge between two respective labels from different label sets of the multiple label sets. Weights are determined for the plurality of edges based upon a consistency between data points classified by two labels connected by the edges. An algorithm is applied that groups labels from the multiple label sets based upon the weights for the plurality of edges. Data points are identified from the corpus of data that represent conflicts within the grouped labels. An electronic message is transmitted in order to present the identified data points to entities for further classification. A new label set is generated using the further classification received from the entities.

Abstract: A computer generates labels for machine learning algorithms by retrieving, from a data storage circuit, multiple label sets that contain labels that each classify data points in a corpus of data. A graph is generated that includes a plurality of edges, each edge between two respective labels from different label sets of the multiple label sets. Weights are determined for the plurality of edges based upon a consistency between data points classified by two labels connected by the edges. An algorithm is applied that groups labels from the multiple label sets based upon the weights for the plurality of edges. Data points are identified from the corpus of data that represent conflicts within the grouped labels. An electronic message is transmitted in order to present the identified data points to entities for further classification. A new label set is generated using the further classification received from the entities.

Abstract: Identification of an intent of a conversation can be useful for real-time or post-processing purposes. According to example embodiments, a method, and corresponding apparatus of identifying at least one intent-bearing utterance in a conversation, comprises determining at least one feature for each utterance among a subset of utterances of the conversation; classifying each utterance among the subset of utterances, using a classifier, as an intent classification or a non-intent classification based at least in part on a subset of the at least one determined feature; and selecting at least one utterance, with intent classification, as an intent-bearing utterance based at least in part on classification results by the classifier. Through identification of an intent bearing utterance, a call center for example, can provide improved service for callers through, for example, more effective directing of a call to a live agent.

Abstract: A database system providing self-tuned parallel database recovery is described. In one embodiment, for example, in a database system, a method is described for performing recovery operations using an optimal number of recovery threads, the method comprises steps of: (a) spawning an initial recovery thread to perform recovery operations; (b) measuring I/O (input/output) performance with the initial recovery thread; (c) spawning a subsequent recovery thread to perform recovery operations; (d) measuring I/O performance with the subsequent recovery thread; and (e) as long as I/O performance does not degrade beyond a preselected percentage, repeating steps (c) and (d) for spawning a desired number of additional recovery threads. In another embodiment, the database system auto-tunes the cache during performance of database recovery operations to optimize the performance of recovery operations.

Abstract: A database system providing methodology for extended memory support is described. In one embodiment, for example, a method is described for extended memory support in a database system having a primary cache, the method comprises steps of: creating a secondary cache in memory available to the database system; mapping a virtual address range to at least a portion of the secondary cache; when the primary cache is full, replacing pages from the primary cache using the secondary cache; in response to a request for a particular page, searching for the particular page in the secondary cache if the particular page is not found in the primary cache; if the particular page is found in the secondary cache, determining a virtual address in the secondary cache where the particular page resides based on the mapping; and swapping the particular page found in the secondary cache with a page in the primary cache, so as to replace a page in the primary cache with the particular page from the secondary cache.

Abstract: A database system providing methodology for extended memory support is described. In one embodiment, for example, a method is described for extended memory support in a database system having a primary cache, the method comprises steps of: creating a secondary cache in memory available to the database system; mapping a virtual address range to at least a portion of the secondary cache; when the primary cache is full, replacing pages from the primary cache using the secondary cache; in response to a request for a particular page, searching for the particular page in the secondary cache if the particular page is not found in the primary cache; if the particular page is found in the secondary cache, determining a virtual address in the secondary cache where the particular page resides based on the mapping; and swapping the particular page found in the secondary cache with a page in the primary cache, so as to replace a page in the primary cache with the particular page from the secondary cache.

Abstract: A database system providing self-tuned parallel database recovery is described. In one embodiment, for example, in a database system, a method is described for performing recovery operations using an optimal number of recovery threads, the method comprises steps of: (a) spawning an initial recovery thread to perform recovery operations; (b) measuring I/O (input/output) performance with the initial recovery thread; (c) spawning a subsequent recovery thread to perform recovery operations; (d) measuring I/O performance with the subsequent recovery thread; and (e) as long as I/O performance does not degrade beyond a preselected percentage, repeating steps (c) and (d) for spawning a desired number of additional recovery threads. In another embodiment, the database system auto-tunes the cache during performance of database recovery operations to optimize the performance of recovery operations.

Abstract: A database system providing improved methods for asynchronous logging of transactions is described. Log records are created describing changes to a database made by a transaction. When a command committing changes to the database for the transaction is received, a logging request is placed in a queue. An asynchronous logging service removes requests from the queue and transfers log records from the transaction to a shared cache. The shared cache stores log records before they are written to the transaction log. The logging service writes log pages containing log records for the transaction from the cache to the transaction log. After all log pages in the cache for the transaction have been written to the transaction log, changes to the database made by the transaction can be committed to the database.