Abstract: A technique for optimizing complex endpoint resolution in a hierarchical data structure. The technique includes maintaining an encoding between a path through the hierarchical data structure and the endpoint node of the path. When an input pathname corresponding to the path is received, the pathname is used to resolve the endpoint node from the encoding without traversing the path.

Abstract: The method and system generates a first deduplication map (DDM) level including first data and a second DDM level including second data. The method or apparatus also generates a first index summary (IS) level corresponding to the first DDM level and a second IS level corresponding to the second DDM level. The method or apparatus merges the first data of the first DDM level and the second data of the second DDM level to generate a third DDM level comprising third data. The method or apparatus in response to generating the third DDM level, generates a third IS level to accelerate lookup within the third DDM level.

Abstract: A method described herein includes receiving a query that is configured by an issuer of the query to be utilized to retrieve a portion of local search content, wherein the local search content comprises a first dataset and a second data set, wherein the first dataset comprises a first entity and the second dataset comprises a second entity, and wherein the query comprises a first token and a second token. The method also includes parsing the query such that the first token of the query is mapped to the first entity in the first dataset and the second token of the query is mapped to the second entity in the second dataset. The method also includes returning search results based at least in part upon the parsing of the query.

Abstract: A method, system, and computer program product to manage a database is disclosed. The method, system, and computer program product may include structuring the database to have a first table having an index and a second table. A first key of the first table may be related to a second key of the second table. The method, system, and computer program product may include creating an entry locator in the index. The method, system, and computer program product may include maintaining an association between the second key of the second table and the entry locator of the index.

Abstract: Image search conditions that are past search conditions are displayed in an image search history area, and text search conditions are displayed in a text search history area. When a user selects a desired image search condition out of the search conditions displayed in the image search history area, and selects a desired text search condition out of the search conditions displayed in the text search history area, case data that matches the selected image search condition and the text search condition is extracted from a database and displayed in a search result display area.

Abstract: Smart mapping is provided. A smart mapping system (SMS) and method identify an information source (IS) as an intended recipient of an information request item by matching one or more data elements in a dataset associated with an information request (IR) item with an IS candidate stored in a mapping database. An IS candidate may be an exact match, or a partial match with a highest match agreement score. The SMS provides for notifying an information requestor in real time or near real time when a match is determined or when a plurality of matching IS candidates are identified, such that a user is enabled to select an intended IS, or provide additional input. When a trend associated with matching particular data items to a particular IS candidate is identified, the SMS may add mapping property values to the mapping database for refining and improving the system.

Abstract: A first table has more than one key column, and a hash array is built for the table, which includes computing hash indices for respective rows of the first table based on more than one key column. Building the hash array also includes writing in the hash array for each row at a respective location in the hash array, wherein each location is indicated by each row's respective hash index computed from the more than one key column. The hash array is probed responsive to the second table. In the building of the hash array for the first table, the writing in the hash array for each row at a respective location in the hash array writes only one of the key columns as a single key field in the hash array for each row.

Abstract: Embodiments of the present invention disclose a method, computer program product, and a computer system for suspending a database index. A computer determines if an update to a database includes an update to an index of the database. If so, the computer suspending the index of the database, wherein the suspending the index of the database comprises: the computer deleting the index of the database; the computer retaining a definition for the index of the database, wherein the definition includes a command for rebuilding the index of the database; and the computer indicating in a system catalog of the database that a status of suspension has been applied to the index.

Abstract: In one embodiment, a data recovery system sorts data elements of a data array by representation. The data recovery system allocates a representation array corresponding to a position of a corresponding data array element after sorting. A scan module of the data recovery system assigns index values to the representation elements of the representation array based on the values of the data elements of the data array without moving the data elements of the data array. A sort module of the data recovery system swaps the data elements of the data array based on the index values of the representation elements of the representation array corresponding to the data elements. The sort module swaps representation elements of the representation array corresponding to the swapped data elements of the data array.

Abstract: A method, apparatus and program product that determine a density of a key value referenced in a database query over at least one range of rows, and utilize the determined density to optimize the database query. By doing so, the density of various key values may be determined and compared, and a bit map may be generated over the range of rows of the key value that is denser, resulting in a reduction of resources to build and use the bit map. Moreover, from the bit map over the range of rows to be selected after using the determined density, dense regions may be identified that can be used to optimize block I/O's according to the number of rows to be selected based on the query.

Abstract: A system and method is described for database split generation in a massively parallel or distributed database environment including a plurality of databases and a data warehouse layer providing data summarization and querying functionality. A database table accessor of the system obtains, from an associated client application, a query for data in a table of the data warehouse layer, wherein the query includes a user preference. The system obtains table data representative of properties of the table, and determines a splits generator in accordance with one or more of the user preference or the properties of the table. The system generates, by the selected splits generator, table splits dividing the user query into a plurality of query splits, and outputs the plurality of query splits to an associated plurality of mappers for execution by the associated plurality of mappers of each of the plurality of query splits against the table.

Abstract: One embodiment provides a method for reclaiming free space. The method comprises selecting a first blob for reclamation from a first data center; sending a first message to a second data center indicating the first blob is to be reclaimed; sending a second message to the second data center after reclaiming the first blob; receiving a global reclamation complete message from the second data center; reading at least one data set from the first blob; and storing in a write buffer the at least one data set for encoding into a erasure code group in an alternative blob in the first data center. Further, upon receipt of the global reclamation message from the second data center, indicating the first blob is free in the map in the first data center. In one embodiment, selecting the first blob is based on the map indicating free space in the first data center.

Abstract: Facility for transferring data over a network between two network endpoints by transferring hash signatures over the network instead the actual data. The hash signatures are pre-generated from local static data and stored in a hash database before any data is transferred between source and destination. The hash signatures are created on both sides of a network at the point where data is local, and the hash database consists of hash signatures of blocks of data that are stored locally. The hash signatures are created using different traversal patterns across local data so that the hash database can represent a larger dataset then the actual physical storage of the local data. If no local data is present, then arbitrary data is generated and then remains static.

Abstract: A request to open a file from a plurality of files in a storage is received from an application. The storage is made up of an index partition, containing recordings of file system indexes, and a data partition, containing recordings of the indexes and the file system data body. A file descriptor is created with an update flag that references the file. A determination is made that the file is being updated by the application, and the update flag is set to a value representing that the file is being updated. A request to write an index of the file system is received. A determination is made whether a specific file from the plurality of files is being updated. The index is written to the storage with an extended attribute for the specific file indicating that the specific file was being updated at the time the index was written.

Abstract: Techniques on on-device indexing of hosted content items are described. In one embodiment, for example, a method is performed at a portable electronic device. The method includes the device performing the operations of: detecting an indexing event within a content management application; determining a content item, hosted with a content management service, to which the indexing event pertains; determining if the indexing event indicates that an index entry for the content item should be added to an on-device index; determining if a proxy entry for the content item already exists in a database that is a read surrogate for the on-device index at least with respect to a content management application by which the content item may be accessed; and attempting to add the index entry to the on-device index, if the proxy entry does not exist in the database.

Abstract: An illustrative embodiment for optimizing scans using a Bloom filter synopsis, defines metadata to encode distinct values in a range of values associated with a particular portion of a managed object in a database management system into a probabilistic data structure of a Bloom filter that stores an indicator, encoded in a fixed size bit map with one or more bits, indicating whether an element of the particular portion of the managed object is a member of a set of values summarized in the Bloom filter using a value of 1 or definitely not in the set using a value of 0. The Bloom filter is compressed to create a compressed Bloom filter. The Bloom filter is added to the metadata associated with the managed object and used when testing for values associated with predicates.

Abstract: A system includes a database having one or more tables having multiple partitions. The database includes memory modules to store the multiple partitions, a query processor, a data manipulation language (DML) processor configured to process operations to manipulate data in one or more of the partitions and a statistics module. For each partition, the statistics module maintains a partition profile and stores the partition profile in a statistics cache. The partition profile includes at least a first interval of data range values per column for one or more columns. The statistics module updates at least the first interval of a partition profile for a partition with new data range values responsive to the DML processor processing an operation to manipulate data in the partition. The database includes a partition pruning module that is configured to include partitions responsive to a received query based on the partition profile.

Abstract: Techniques for creating and using partial indexes are provided. A partial index is an index that indexes one or more partitions of a partitioned table and does not index one or more other partitions of the partitioned table. Thus, if a partition of a partitioned table is rarely used, then an index may index all other partitions of the partitioned table or at least only those partitions that are written to or read from relatively frequently. Also, in preparation for loading data into a partition, indexing for the partition may be “turned off”, effectively making a full index a partial index. While the data is loaded into the partition, the partial index is still available for query processing. After the data is loaded into the partition, indexing for the partition is “turned on”, which causes the data in the partition to be indexed by the partial index.

Abstract: A mechanism is provided in a data processing system comprising at least one processor and at least one memory, the at least one memory comprising instructions executed by the at least one processor to cause the at least one processor to implement a document analysis device for performing a statistical analysis of documents with respect to a facet. An acceptance module accepts a natural language sentence. An extraction module extracts a first facet from the natural language sentence. A statistical analysis module performs a first statistical analysis of a set of documents with respect to the first facet and determines a value of the first facet based on a result of the first statistical analysis responsive to information being extracted from the natural language sentence, the information requesting for a second statistical analysis. The statistical analysis module performs the second statistical analysis of the set of documents using the value of the first facet.

Abstract: Aspects of the present disclosure relate to a method for timing index write to a tape medium of a tape system. Index write timing data that specifies index write timing based on file metadata attributes is stored. Metadata attributes of a first file are analyzed to determine whether a first index should be written at a first time based on the index write timing data. In response to determining that an index should be written, the first index is written to the tape medium at the first time.

Abstract: The present disclosure involves systems, software, and computer implemented methods for providing a bucket skiplist. In one example, a method comprises identifying a value to be searched for within a skiplist comprising a data structure for referencing a plurality of ordered nodes, the data structure having a plurality of linked lists, each node representing at least one memory block, wherein each node is associated with a bucket including a set of ordered values, where the first value in each bucket of each node is the relatively lowest value in the bucket, comparing the value to the first values in each bucket to identify a particular node in which the first value may be located, and, in response to determining that the first value of the particular node is not the same as the value to be searched, comparing the value to be searched with the ordered values in the bucket.

Abstract: Performing optimized update operations in in-memory database systems. Update values corresponding to a source table of values stored on an in-memory database are identified by an in-memory database computer system based on information from a computer system implementing a software application. The update values are distributed across a plurality of rows, each row including a plurality of column attributes. At least some column attributes of the plurality of column attributes include values updated relative to corresponding column attributes in the source table of values. A row of the update values is identified. The row includes a plurality of column attributes, each column attribute having a respective value. Each value of each column attribute of the row of the update values is the same as a corresponding value of a corresponding attribute of a corresponding row of the source table is determined.

Abstract: A method for query processing includes receiving a query, and determining a query type by determining if the query is a structured query language (SQL) query including multiple distinct aggregates (MDAs), or a SQL query including cube, rollup or grouping sets operations. Based on the determination, the method includes processing the query to generate an output, and performing a predetermined number of maximum group by operations on the output based on the query type to generate a response to the query.

Abstract: A system and method for executing user-provided code securely on a solid state drive (SSD) to perform data processing on the SSD. In one embodiment, a user uses a security-oriented cross-compiler to compile user-provided source code for a data processing task on a host computer containing, or otherwise connected to, an SSD. The resulting binary is combined with lists of input and output file identifiers and sent to the SSD. A central processing unit (CPU) on the SSD extracts the binary and the lists of file identifiers. The CPU obtains from the host file system the addresses of storage areas in the SSD containing the data in the input files, reads the input data, executes the binary using a container, and writes the results of the data processing task back to the SSD, in areas corresponding to the output file identifiers.

Abstract: Multiple communications that encode data are encrypted for transit from one entity to the other. An entity receiving the communications decrypts at least some of the communications to determine how to process the communications. As part of processing the communications, the entity receiving the communications provides at least some of the encrypted communications to a data storage system without reencrypting those communications.

Abstract: Techniques are provided for selectively refreshing a materialized view. A portion of a materialized view is refreshed (or updated) while other portions of the materialized view are not refreshed. Thus, the other portions of the materialized view may contain “stale” data that does not accurately reflect current data in the base table(s) upon which the materialized view is based. In one technique, a user indicates the portion of the materialized view that is to be refreshed. In another technique, a user indicates a plurality of portions of a materialized view and it is automatically determined which portion should be refreshed first. Change data that indicates information about changes made to different portions of one or more base tables and/or read data that indicates information about reads to different portions of the materialized view may be taken into account to determine which portion of a materialized view to refresh.

Abstract: A method is described for distributing a data set. The method may include dividing a data set into a number of data groupings based on a data set attribute value. The groupings of data may be stored in a data store and may be associated with metadata that describes a grouping of data. A grouping of data may be distributed by generating a reference that may be used to access the grouping of data in the data store. The reference may include information that enables access to the grouping of data. When presented, the information included in the reference may be authenticated whereupon the grouping of data may be provided.

Abstract: There are provided a method for operating multi-node data stores. The method performs storing a data table in a first computing node and stores an index table in a second computing node. The index table provides keys used for accessing data in the first computing node and other multi-node data stores. The method performs operations that update or read the data table accessed from the first computing node and the index table accessed from the second computing node. The operations optimizes between latency in updating or reading the data table and the index table and data consistency maintained between data entries in the data table and data entries pointed by indices in the index table.

Abstract: A processor tracks a frequency of access requests of a first index corresponding to a first data page of a plurality of data pages stored in a database. The processor determines the first index corresponding to the first data page having a frequency of access requests that exceeds a configurable target, and the processor retains, with preference, the first data page that corresponds to the first index, within the cache memory.

Abstract: There are provided a system and a computer program product for operating multi-node data stores. The system stores a data table in a first computing node and stores an index table in a second computing node. The index table provides keys used for accessing data in the first computing node and other multi-node data stores. The system performs operations that update or read the data table accessed from the first computing node and the index table accessed from the second computing node. The operations optimizes between latency in updating or reading the data table and the index table and data consistency maintained between data entries in the data table and data entries pointed by indices in the index table.

Abstract: A processor tracks a frequency of access requests of a first index corresponding to a first data page of a plurality of data pages stored in a database. The processor determines the first index corresponding to the first data page having a frequency of access requests that exceeds a configurable target, and the processor retains, with preference, the first data page that corresponds to the first index, within the cache memory.

Abstract: Methods for Editing Hierarchical Data are provided. Data subsets of hierarchical data are copied. Workstreams are generated and sent to users. An edit to a first workstream from a first user and an edit to a second workstream from a second user are received. The hierarchical data in the general database is edited according to the edits made by the first user and the second user. A full organization view of persons and positions in the organization that includes the edits made by the first user and the second user is displayed.

Abstract: Example systems and methods for integrating a sharded primary data store (e.g., a source-of-truth relational database management system), a secondary data store (e.g., external cache) and an external global index are described. The approach implements a modified read-through/write-through data access pattern that integrates read and write flows in order to support high-concurrency environments while maintaining immediate consistency between all three stores. Writes are handled using a three-phase flow that avoids concurrency-related race conditions and the need to block in the secondary store in order to maintain cross-store consistency. Reads are never dirty and will repair the secondary store as needed, presenting an immediately consistent view of data to application consumers.

Abstract: A method can include receiving a request to execute a database command identifying a target table; identifying a plurality of rows to insert into the target table based in part on the database command; writing rows, from the plurality of rows, into a data page until the data page is full; determining, by an index thread manager, a number of threads to use for updating indexes defined for the target table; and upon determining the data page is full, updating, in parallel, the indexes defined for the target table using the number of threads.

Abstract: Creation of an index for a table of sorted data for use by a data storage application is initiated. Thereafter, N+1 logical partition of rows of the table are defined so that each logical partition has a corresponding worker process. Each worker process then builds a sub-index based on the corresponding logical partition which are later merged to form the index. Related apparatus, systems, techniques and articles are also described.

Abstract: Electronic files are selectively assigned to a plurality of different indexing queues by one or more dynamic throughput threshold gates based on characteristics of the different indexing queues as well as the static file characteristics associated with each of the files. The files are then indexed. Upon detecting a change in a dynamic characteristic of one or more indexed files, the throughput threshold gate(s) are then modified to obtain, maintain or modify a desired throughput for one or more of the indexing queues.

Abstract: Embodiments of the present invention disclose a method, a computer program product, and a computer system for suspending and activating a database index. A computer determines a quantity of changes for a number of rows of a database that are to be made by an update of the database. The computer determines whether to suspend an index of the database based, at least in part, on a comparison between the quantity of changes and a pre-determined threshold value. The computer suspends the index of the database based on a result of the comparison that indicates the quantity of changes at least meet the pre-determined threshold value. A suspension of the index of the database initiates a rebuilding of the index.

Abstract: A query that is submitted to a database is analyzed so as to determine a join order restriction. The join order restriction is associated with a join between two tables of a plurality of tables of the database that participate in the query. The join order restriction constrains its associated join to be executed prior to another join of the query. At least one join plan is generated, the join plan being constrained by the join order restriction. Different join plans include different join orders. A join plan is selected from among the join plans for execution of the query.

Abstract: A technique reduces an amount of metadata stored in a memory of a node in a cluster. An extent store layer of a storage input/output (I/O) stack executing on the node stores key-value pairs in a plurality of data structures, e.g., cuckoo hash tables, resident in the memory. The cuckoo hash table embodies metadata that describes an extent and, as such, may be organized to associate a location on disk with a value that identifies the location on disk. The value may be embodied as a locator that includes a reference count used to support deduplication functionality of the extent store layer with respect to the extent. The reference count is divided into two portions: a delta count portion stored in memory for each slot of the hash table and an overflow count portion stored on disk in a header of each extent. One bit of the delta count portion is reserved as an overflow bit that indicates whether the in-memory reference count has overflowed.

Abstract: According to one embodiment of the present invention, a method for processing a query in a column database is provided. The method for analyzing test results may include a computer, associating a first set of metadata values with a set of records in a table, wherein each record has an associated state indicated by a metadata value. The method may further include the computer receiving a query to return one or more records of the set of records. The method may further include the computer determining that one or more records in the set of records satisfies the query. The method may further include the computer determining whether at least a first record in the one or more records that satisfies the query is in a returnable state based, at least in part, on the metadata value.

Abstract: According to one embodiment of the present invention, a method for processing a query in a column database is provided. The method for analyzing test results may include a computer, associating a first set of metadata values with a set of records in a table, wherein each record has an associated state indicated by a metadata value. The method may further include the computer receiving a query to return one or more records of the set of records. The method may further include the computer determining that one or more records in the set of records satisfies the query. The method may further include the computer determining whether at least a first record in the one or more records that satisfies the query is in a returnable state based, at least in part, on the metadata value.

Abstract: The subject matter disclosed herein provides methods for determining the page on which a desired row position value is located. A table having a plurality of rows can be accessed. The rows can be distributed across one or more pages in an in-memory database. Each row can be associated with a unique row position value. Each page can be associated with a capacity that is representative of an amount of data stored on the page. A capacity index having a plurality of entries can be created to record changes in capacity between pages. Neighboring entries in the capacity index can have a different capacity. Each entry in the capacity index can correspond to a page. A page directory can be created based on the capacity index. The page directory can indicate all possible row position values associated with each page. Related apparatus, systems, techniques, and articles are also described.

Abstract: The present invention relates to remote storage auditing. In another embodiment, a remote storage auditing system may include a first remote storage manager configured to be a data owner, a second remote storage manager configured to be a storage donor, and a remote storage auditor. The first remote storage manager sends a data block and a signed fingerprint for the data block to the second remote storage manager. The second remote storage manager verifies that the signed fingerprint is associated with the data block and stores the data block and signed fingerprint. The second remote storage manager calculates a fingerprint for a sub-block of the data block, and sends the fingerprint for the sub-block and signed fingerprint to the remote storage auditor. The remote storage auditor audits a sub-block of the data block and verifies the fingerprint for the sub-block and signed fingerprint.

Abstract: An information processing system including: a plurality of second information processing apparatuses connected to a first information processing apparatus via a network; and a management apparatus. The plurality of second information processing apparatuses dispersedly manage indexes established for searches using data and metadata of the data. The management apparatus includes a processor configured to monitor a search request transmitted from the first information processing apparatus, and to specify, on the basis of a monitoring result by the processor, a combination of types of the metadata, for which the processor should cause a same second information processing apparatus to manage the indexes, and the same second information processing apparatus and cause the specified same second information processing apparatus to manage the indexes of the type of the metadata of the specified combination.

Abstract: In accordance with embodiments, there are provided mechanisms and methods for performing transparent object migration across storage tiers. In an embodiment and by way of example, a method for appending data to large data volumes is provided. The method embodiment includes a) setting a CustomEntityOption bit that determines (at object creation time) where the object is stored, either in the relational or the non-relational data store portion, b) loading the CustomEntityOption bit in a cached CustomEntityDefinition, c) showing the CustomEntityOption bit as EntityInfo, and d) allowing custom object definition and Metadata API functionality when the bit is shown.

Abstract: Various embodiments for identifying data in a data deduplication system, by a processor device, are provided. In one embodiment, a method comprises efficiently identifying duplicate data in the data deduplication system by identifying fingerprint matches using a direct inter-region fingerprint lookup to search for the fingerprint matches in at least one of a plurality of metadata regions, the direct inter-region fingerprint lookup supplementing a central fingerprint index.

Abstract: Processing a database table comprises receiving a multi-dimensional processing instruction for processing at least two columns of the database table, wherein the two columns comprise a selection column and a sort column, wherein the multi-dimensional processing instruction is for selecting a plurality of candidate rows by sorting based on the selection column and the sort column. An index takes the selection column as master column and the sort column as slave column. Nodes in the index are selected according to the selection column. Each of the selected nodes is associated with a respective slave column value sequence. A merge sort is performed on each of the slave column value sequences to obtain a sort result for the candidate rows.

Abstract: A method and apparatus for an in-database sharded queue that supports JMS session ordering is provided. Messages from a plurality of enqueuers are stored in a plurality of shards of a particular queue. For each enqueuer, all messages are stored in only one of the shards. Each shard includes one or more subshards, each subshard having one or more partitions of a queue table maintained by a relational database system. All messages from a first enqueuer are stored in partitions of the queue table that are assigned to a first shard of the plurality of shards. All messages from a second enqueuer are stored in partitions of the queue table that are assigned to a second shard of the plurality of shards. Subscribers of the particular queue are caused to dequeue messages from the plurality of shards.

Abstract: The subject disclosure is directed towards a data deduplication technology in which a hash index service's index is partitioned into subspace indexes, with less than the entire hash index service's index cached to save memory. The subspace index is accessed to determine whether a data chunk already exists or needs to be indexed and stored. The index may be divided into subspaces based on criteria associated with the data to index, such as file type, data type, time of last usage, and so on. Also described is subspace reconciliation, in which duplicate entries in subspaces are detected so as to remove entries and chunks from the deduplication system. Subspace reconciliation may be performed at off-peak time, when more system resources are available, and may be interrupted if resources are needed. Subspaces to reconcile may be based on similarity, including via similarity of signatures that each compactly represents the subspace's hashes.

Abstract: Systems and methods are provided for matching snippets of search results to clusters of objects. A system adds a data snippet of a search result to a cluster of objects. The system calculates a confidence score for the add based on the recency, a job title, an email address, and/or a phone number associated with the data snippet. The system stores the add in the customer accessible database if the confidence score is sufficiently high for the add to be stored in the customer accessible database. The system generates a notice for review if the confidence score is not sufficiently high for the add to be stored in the customer accessible database.