Abstract: Collective buffering and data pattern solutions are provided for storage, retrieval, and/or analysis of data in a collective parallel processing environment. For example, a method can be provided for data storage in a collective parallel processing environment. The method comprises receiving data to be written for a plurality of collective processes within a collective parallel processing environment, extracting a data pattern for the data to be written for the plurality of collective processes, generating a representation describing the data pattern, and saving the data and the representation.

Abstract: A computer-implemented method for implementing non-native file attributes on file systems may include receiving at least one extended attribute to apply to at least one file within a file system that does not natively implement the extended attribute, identifying an index file that indexes extended attributes of files within the file system by file identifiers of the files, determining a file identifier that uniquely identifies the file within the file system, and applying the extended attribute to the file by indexing the extended attribute by the file identifier within the index file. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A system, method, and computer program product for organizing a visual representation of data into a structured data format are presented. Steps for identifying cell data structures from raw data and creating a structured data format are disclosed.

Abstract: A query system for carrying out a query process corresponding to a received query element on a database system and/or a search-engine system, the query system comprising: a synchronization unit operable, as a non-text-centric synchronization process, to convert a non-text-centric data entry of the database system into a corresponding text-centric data entry representative of that non-text-centric data entry, and to enter the text-centric data entry into an index of the search-engine system so that the non-text-centric data entry of the database system is available to both the search-engine and database systems.

Abstract: A search engine may maintain a list of derived metadata. When an event occurs that requires updating a search index, the search engine can determine which metadata is derived metadata and take appropriate actions with respect to the derived metadata. For example, if a request is received to update the index for a particular object, the search engine may protect the derived metadata from change while updating the other metadata in the index. As another example, if a request is received to update the text content for the object, the search engine may change the text content and the derived metadata. By identifying derived metadata, the search engine can protect the derived metadata from change when a request is received that otherwise causes metadata to change and can change the derived metadata when a request is received that would otherwise not change the metadata portion of the index.

Abstract: In one embodiment, a method includes receiving a request to perform an update on a base data set in a record-oriented file system, locate all alternate index (AIX) records for the base data set, each AIX record including an alternate key field along with one or more relative byte address (RBA) values when the base data set is an entry sequenced data set (ESDS) or one or more primary keys when the base data set is a key sequenced data set (KSDS). Each RBA value/primary key corresponds to one record in the base data set. Individual record-level locks are acquired on each affected AIX record that includes alternate key field values and/or RBA values that are modified by the update. The affected AIX records may be updated after acquiring the individual record-level locks on each of the affected AIX records. Moreover, the affected AIX records may be updated in parallel with updating of unaffected AIX records, thereby greatly increasing performance.

Abstract: Overflow access records (OARs) are managed in a database system. An OAR is created in response to receiving an update command for a data record and to the updated data record generated by the update command not fitting onto the page in the table where the data record was stored. The OAR that is created includes an index counter that indicates a number of indexes associated with the table. When an OAR is accessed in response to a query command, an identifier of the accessed OAR is replaced in the index by an identifier of a data record pointed to by the OAR, and the index counter in the accessed OAR is changed by a predefined amount. When the index counter reaches a predefined value, the accessed OAR is removed from the table.

Abstract: A client application installed on end user computers generates metadata from the content of web pages visited by end users and provides the metadata to a search engine. When an end user visits a web page, the end user's computer downloads and displays the web page to the end user. The client application may simultaneously access the web page content and generate this metadata in the form of a content signature of the web page from the web page content. The client application then provides the content signature to a search engine. The search engine may employ content signatures to identify new web pages to crawl and index. Additionally, the search engine may employ content signatures to identify changes to web pages and determine the crawl frequency of web pages.

Abstract: A monitoring server receives the latest monitored data and corresponding monitoring time from monitoring devices connected to the server at a specified time interval. The monitoring server records monitored data of each monitoring item and the corresponding monitoring time to a pointed node of a device data list, and adds the monitored data and the corresponding monitoring time, which are recorded in the pointed node of the device data list, into a monitored data list. When a web server requests real-time monitored data, corresponding monitored data is read from the device data list and sent to the web server. When the web server requests historical monitored data of an designated period, corresponding monitored data of the designated period is read from the monitored data list and sent to the web server.

Abstract: System and method for efficient maintenance of indexes for XML and other documents comprising semi-structured, hierarchical data are described. In one embodiment, the method comprises providing a first index definition document (“IDD”) for defining a first index for the document, wherein the first IDD is applied to the document to create a first set of index keys for the document stored in the database and wherein the first IDD defines at least one set of relationships among nodes in the document; responsive to a change to the document affecting an update node thereof, performing a limited, localized traversal of the document around the update node to determine whether the change affects the first set of index keys; and updating the first set of index keys as necessitated by the change.

Abstract: Input backup data is deduplicated with data of a synthetic backup previously constructed by a deduplication storage. A synthetic backup is constructed by processing metadata instructions provided by a backup application. Deduplication digests are calculated based on the data of the synthetic backup and the deduplication digests are stored in a digests index. When new backup data is processed, deduplication digests of the new data are calculated and searched in the digests index. Matching digests of previously constructed synthetic backups are located in the digests index. Each of the located matching digest references stored data are included in the synthetic backup, and the stored data is similar to the input backup data. Data matches are found in the input data and data in the synthetic backup.

Abstract: The specification relates to framework for removing non-authored content documents from an authored-content database by recording a sequence of authorship data for at least one authored-content document over a period of time. The at least one authored-content document can be indexed in an authored-content database. The sequence of authorship data is analyzed to determine if the at least one authored-content document changed in a meaningful way beyond a set threshold. If the at least one authored-content document is changed beyond the set threshold, the at least one authored-content document is removed from the authored-content database.

Abstract: The present invention provides a method for performing transactions on data entities in a database and a transactional database. The database comprises an ordered set of data stores with at least one static data store, wherein said static data store uses an index structure based on a non-updatable representation of an ordered set of integers according to the principle of compressed inverted indices. The method allows to generate a modifiable data store when the performed transaction comprises an insert, update or delete operation, to execute operations of the transaction on the ordered set being present at the time when the transaction has been started and, if present, on the modifiable data store and to convert data stores to a new static data store, The insert, update or delete operation are executed on the modifiable data store which is the only data store modifiable for the transaction.

Abstract: A plurality of indicators representing a plurality of respective candidate database configurations may be obtained, each of the candidate database configurations including a plurality of database queries and a plurality of candidate database indexes associated with a database table. A portion of the candidate database indexes included in the plurality of database indexes may be selected based on skyline selection. An enumeration of the portion of the plurality of the candidate database indexes may be determined based on a greedy algorithm.

Abstract: Disclosed herein are system, method, and computer program product embodiments for constructing an index for a database table. An index that comprises a data structure may be created. The index can then be populated with data from the database table. When a request to modify the database table is received, the method may determine that the request to modify the database table relates to a portion of the database table corresponding to a portion of the index that has yet to be populated. An entry indicating the requested modification can be inserted into the portion of the index that has yet to be populated.

Abstract: Processing and analyzing hyper-spectral image data and information via dynamic database updating. (a) processing/analyzing representations of objects within a sub set of the hyper spectral image data and information, using a first reference database of hyper spectral image data, information, and parameters, and, a second reference database of biological, chemical, or/and physical data, information, and parameters. Identifying objects of non-interest, and objects of potential interest, from the data/information sub-set. (b) processing/analyzing identified objects of potential interest, by further using first and second reference databases. Determining absence or presence of objects of interest, additional objects of non-interest, and non-classifiable objects of potential interest, from the data/information sub set. (c) updating first and second reference databases, using results of (a) and (b), for forming updated first and second reference databases.

Abstract: An index update unit analyses the information stored in a document repository to create an index for search and stores the index in a time-series divisional index storage unit and creates, from an ACL repository, an access control entry ACE in association with the index for search, which is correlation of information to be searched with access right of at least a group to which the user belongs. The ACL cache generation unit creates ACL cache data that correlates the user with access right to the information to be searched, from the ACE, and registers the ACL cache data created in an ACL cache. A search processing unit searches for an index for search in response to a request for search from said user. In case the ACL cache data correlating the user with the index for search is registered in the ACL cache, the search processing unit takes, from among the information searched, the information, reference to which is allowed for the user as a search result, based on information in the ACL cache.

Abstract: A technique for storing binary large objects (BLObs) in a distributed key-value storage system is disclosed. In an embodiment, the storage of BLObs involves dividing the BLObs into smaller “BLOb chunks” and storing the BLOb chunks in storage nodes of the distributed key-value storage system. A BLOb is first associated with a key, referred to as a “BLOb key,” and each subsequent BLOb chunk is associated with a different unique key, referred to as a “BLOb chunk key.” A list of the BLOb chunks keys is maintained and all of the BLOb chunks of the BLOb can be retrieved from the distributed key-value storage system using the BLOb chunks key list.

Abstract: Methods and systems for maintaining the internal consistency of a fact repository are described. Accessed objects are checked for attribute-value pairs that have links to other objects. For any link to an object, the name of the linked-to object is inserted into the attribute-value pair having the link. The accessed objects are filtered to remove attribute-value pairs meeting predefined criteria, possibly resulting in null objects. Links to null objects are identified and removed.

Abstract: Concepts and technologies are described herein for managing a search index. In accordance with the concepts and technologies disclosed herein, a search index can be hosted by a number of search nodes. The search engine can be configured to determine if a search node is unavailable, and if so, if data stored at the unavailable search node is to be replaced. If data stored at the unavailable search node is to be replaced, the search engine can identify sources for data to be used to replace the data at the unavailable search node. The search engine can issue instructions to the sources and/or the unavailable search node to copy the data from the sources to the unavailable node and/or to replace the data at the unavailable node with the data from the sources. Portions of the data can be copied from each of the sources to the unavailable node.

Abstract: A computer-implemented method for use in maintaining currency of a projection index of a plurality of database objects. The computer-implemented method includes creating the projection index representative of a connection between a first database object and at least a second database object, determining an entity dependency between the first database object and at least the second database object, determining a path dependency between the first database object and at least the second database object, and updating the projection index in response to a modification of one or both of the entity dependency and the path dependency.

Abstract: Systems, methods, and computer program products for enabling assessment of the quality of a search index. In one embodiment, objects are processed to produce corresponding text that is stored and indexed. The objects are also processed to identify and store corresponding metadata values for indexing. Error conditions that are detected during the processing of objects to generate corresponding text are tracked and compared to determine the most severe of the error conditions. An indication of the most severe error condition is stored in a first consolidated error field. Errors that are encountered in the identification and storage of metadata values are counted and this count is stored in a second consolidated error field. Both of the consolidated error fields are indexed in the same manner as the text and metadata for the objects, so that the stored error information can be used in queries of the search index.

Abstract: Provided is a member or content search method in a virtual private community (VPC) network including at least one of a first VPC including communication devices owned by a predetermined user, a second VPC that may be positioned in an upper layer of the first VPC, and a third VPC that may be positioned in an upper layer of the second VPC, the method including receiving, by one of the communication devices, a search request comprising one of VPC identifiers of a user, verifying a VPC corresponding to the VPC identifiers that may be included in the search request, in response to the search request, and searching for members included in the verified VPC, a VPC positioned in a lower layer of the verified VPC, or contents owned by the members included in the verified VPC.

Abstract: Systems and methods for mapping and propagating a source dataset to a plurality of target tables in a target database are described herein. Embodiments include a physical database design with a template-based loader, and a method to propagate changes in the source dataset to a target database. Other embodiments include a database physical design with a plurality of small fact and summary tables. Still other embodiments include a method of extracting and loading data automatically into a target database, while simultaneously maintaining current summary tables.

Abstract: Systems, methods, and computer program product embodiments are provided for path-based query evaluation. The embodiments include representing a markup language document as a descriptive catalog, and capturing relationships in the descriptive catalog as an index summary. The index summary is utilized for evaluating a path-based query.

Abstract: A disappearing index allows faster processing of a database query without the startup time of a prior art sparse index. The disappearing index starts as a full index but is trimmed of index values that point to a row in the table that is not selected by the query. Thus the traditional index disappears as it becomes a sparse index tailored to the query. The query is able to execute faster using the now sparse index since the target database table is not accessed for duplicate values in the base table of the query. The query optimizer may determine to use a disappearing index based on estimates of the number of duplicate values in the base table. When the query is complete, the created disappearing table may be discarded or used for other queries that match the selection of the query that created the disappearing index.

Abstract: A storage system having a plurality of storages. The each of the storages include a memory and a processor coupled to the memory. The processor executes a process including transmitting an update request for data which is commonly stored in the plurality of storages according to a predetermined transmission order indicating a path to transfer the update request. The process includes updating data when receiving an update request from another storage. The process includes changing the predetermined transmission order to a transmission order in which one or more storages included in the path are excluded according to the number of times the update request for the data is received.

Abstract: Embodiments of the present disclosure are directed to, among other things, managing inventory indexing of one or more data storage devices. In some examples, a storage service may store an index associated with archived data. Additionally, the storage service may receive information associated with an operation performed on the archived data. The storage service may also partition the received information into subsets corresponding to an identifier. In some cases, the identifier may be received with or otherwise be part of the received information. The storage service may also retrieve at least a portion of the index that corresponds to the subset. Further, the storage service may update the retrieved portion of the index with at least part of the received information. The updating may be based at least in part on the subsets.

Abstract: A method for writing and reading data into or out of an indexed dataset includes a data structure and an associated index structure, a processing unit that receives data to be written in plain text and writes the data to the data structure by means of write access and updates index data in the index structure. The processing unit detects data to be read out or the memory location thereof by means of access to the index data and reads out a data from the data structure by means of read access and makes the same available in plain text. The data in the data structure and the index data in the index structure are stored in an encrypted manner. Write/read access of the processing unit to the index structure and to the data structure take place via at least one en- and decryption unit.

Abstract: An internet based method and system for communicating terms and conditions of an issued export license to all parties (e.g., exporter, distributor, consignee) in a transaction. The system directs prescribed export related messages from the exporter to the parties. The messages include instructions to the recipients to take certain actions required by the export license terms, and directs those parties to report the results back to the exporter.

Abstract: Methods and Apparatus, including computer program products, for maintaining indexes in a database management system. The database management system includes a table having table entries and an index data structure having index data structure entries, the index data structure relating to the table. The index data structure is updated synchronously with an insertion of table entries. The index data structure is maintained unchanged when the table entries are deleted. It is verified by an index-scan on the index data structure that a valid criterion is fulfilled for index data structure entries, and those index data structure entries that pass the valid criterion from the index-scan are returned.

Abstract: Provided are a computer program product, system, and method for managing multiple locks for data set members in a data set index. The data set index has leaf nodes and internal nodes. In response to determining that a current lock is set at a locked node that prevents a grant to a requested lock, a determination is made as to whether there are different first and second descendant nodes from the locked node at a same level of the tree data structure linked to the requested data set member and the locked data set member, respectively. If so, the requested lock for the requested data set member is set at the first descendant node, the current lock is moved to the second descendant node for the locked data set member and the requested lock is granted.

Abstract: A method for decoding encoded structured data from a bit-stream comprising a plurality of encoded data units having the steps of, obtaining unit information comprising positions of the encoded data units within the bit-stream, retrieving the encoded data units from the bit-stream based on the unit information, creating decoding tasks for decoding the retrieved encoded data units, assigning the created decoding tasks to cores of a multi-core decoder, based on estimated decoding costs of the encoded data units, running the tasks on their assigned cores to decode the encoded data units in parallel is disclosed. It is applied to the decoding of XML documents in the EXI format.

Abstract: A method of determining and optimizing the location of a new insurance agency is disclosed to increase market penetration of underrepresented markets. The method comprises the use of a scoring algorithm to rank various geographic regions or related zip codes. The scoring algorithm may be implemented by a location modeling system based on variables selected by a user.

Abstract: A computer-implemented method for dynamically selecting a logical location for an index is described. Information relating to a computing device and a location within the computing device is collected. The information is uploaded to a database. The database is queried for stored information relating to at least one computing device and at least one location within the at least one computing device. A performance score is calculated from the stored information. A logical location for the index is selected based on the calculated scores.

Abstract: A search system can include a server, a token assignment unit for assigning types of tokens based on different kinds of character string analysis methods, an index generating unit for generating an index list that associates the tokens assigned with the token assignment unit, a type identification value for identifying a type of the character string analysis, and information, a search unit that receives a search word for referencing the information to combine types of search tokens generated from the search word to generate a single search command for parallel inquiry of the information to search for the information, and a search result generating unit for displaying information extracted in relation to the search word through parallel inquiry with the search unit and search tokens so as to identify the tokens.

Abstract: Methods and systems for partitioning search indexes are described. In one embodiment, such a method includes identifying, via a partition manager, a search index to be partitioned, where the search index includes a plurality of entity types, and wherein each of the plurality of entity types associates records stored in a data store with the respective entity type within the search index. The method further includes partitioning, via the partition manager, the search index into a first partitioned search index and a second partitioned search index, correlating, via a partitioning scheme, a first subset of the plurality of entity types with the first partitioned search index, and correlating, via the partitioning scheme, a second subset of the plurality of entity types with the second partitioned search index.

Abstract: This innovation provides a method for a networked and replicated database management system (DBMS) using only one-sided remote direct memory access (RDMA). Replicated databases retain some access to the stored data in the face of server failure. In the prior state of the art, after the software in the DBMS on one of the servers acted on a client's request to update the database, it would contact the other replicas of the database and ensure that they had recorded the change, before responding to the client that the transaction was complete. This innovation describes a method whereby the database client directly interacts with each DBMS replica over the network using only RDMA to directly modify the stored data while maintaining the properties of database atomicity and consistency. This method reduces transactional latency by removing any need for the server DBMS software to respond to or forward requests for service.

Abstract: Collisions in hash tables are reduced by removing each empty bucket from a hash table and compacting the non-empty buckets, generating a map of the hash table indicating a status of the buckets of the hash table, and accessing data in the hash table by applying a hash key to the generated map to determine a corresponding bucket containing the data.

Abstract: To determine a row with modified data, a dataset is retrieved from a data store and displayed on a user interface in rows and columns. A first database index corresponding to the rows is generated and stored in a technical column in the data store. An input is received on the UI to modify the displayed dataset and the modified dataset is stored in the data store. A second database index corresponding to the rows including modified dataset is generated and the associated field of the technical column in the data store is updated with the second database index. The row including the modified dataset is determined based on the second database index stored in the technical column.

Abstract: Distributed database management systems may maintain collections of items spanning multiple partitions. Index structures may correspond to items on one partition or to items on multiple partitions. Item collections and indexes may be replicated. Changes to the data maintained by the distributed database management system may result in updates to multiple index structures. The changes may be compiled into an instruction set applicable to the index structures. In-memory buffers may contain the instructions prior to transmission to affected partitions. Replication logs may be combined with an acknowledgment mechanism for reliable transmission of the instructions to the affected partitions.

Abstract: Technologies are generally described for a multi-layer file system. In the multi-layer file system, a file is arranged into two or more data blocks respectively associated with corresponding multi-entry file indices. Each multi-entry file index can point to a location index at the beginning of a linked list (an initial entry in the linked list). The linked list may have at least one location index as an entry to identify a location in the storage device where the associated data block is stored.

Abstract: Identifying significant media granulars may comprise partitioning a media document into one or more media granulars. A score may be assigned to each of the one or more media granulars. The one or more media granulars may be monitored to detect one or more user actions on the one or more media granular. The score associated with a respective one of the media granulars may be updated based on a number of user actions detected on the respective one of the media granulars. A media granular may be identified as being significant based on the score meeting a threshold.

Abstract: Embodiments of the present invention relate to systems, methods and computer storage media for facilitating the structured storage of binary large objects (Blobs) to be accessed by an application program being executed by a computing device. Generally, the manipulation of Blobs in a structured storage system includes receiving a request for a Blob, which may be located by way of a Blob pointer. The Blob pointer allows for the data, such as properties, of the Blob to be identified and located. Expired properties are garbage collected as a manipulation of the Blob data within a structured storage system. In an embodiment, the Blob is identified by a key that is utilized within a primary structured index to located the requested Blob. In another embodiment, the requested Blob is located utilizing a secondary hash index. In an additional embodiment, the Blob is locate utilizing a file table.

Abstract: The invention concerns a method for assigning a value to at least one data item of a test case instance. The invention is characterized in that the method comprises steps of selecting an input data item from a test design library, the library comprising a plurality of input data items and information about semantics and/or data type of the input data items and assigning the selected input data item as a value of data item of the test case instance. Also an arrangement and a computer readable media comprising a computer executable program are disclosed.

Abstract: Documents and search terms may be received for which an index that supports both incremental document indexing and incremental term indexing is to be generated. From this information, an index may be generated, partitioned in a first dimension by documents to create master segments and in a second dimension by search terms to create slave segments. A request to update the index to include a new document or a new search term may be received. The new document or new search term may be added to the index without modifying the entire index. Further, document identifiers may be synchronized across all segments. Synchronization refers to maintaining consistency of document identifiers across segments, despite renumbering of document identifiers during certain operations such as merging segments.

Abstract: A method for difference analysis in file sub-regions includes determining a start location in each of a first file and a second file. The start location is defined from a start indicator. The method includes determining a stop location in each of the first file and the second file. The stop location is defined from a stop indicator. The method includes defining a first difference region in the first file and a second difference region in the second file. The first and second difference regions include portions of the first and second files respectively. The portions are between the start location and the stop location in the first and second files. The method includes performing a difference analysis between the first difference region and the second difference region, including identifying differences between the first difference region of the first file and the second difference region of the second file.

Abstract: A method comprising receiving data, where the data includes one or more elements, the data is associated with a first set of indices, and the first set of indices references the one or more elements. The method may further include folding a plurality of indices into a single index, where the single index references the one or more elements of the received data that were referenced by the plurality of indices, the folding generates a second set of indices, and the folding is performed when concurrent process threads are not generated based on the first set of indices. The method may further include determining whether concurrent process threads should be generated based on the second set of indices and generating the concurrent process threads for the second set of indices when the determining determines that the concurrent process threads should be generated based on the second set of indices.

Abstract: A method is provided within a storage processor of a data storage system having a fixed storage capacity shared between a plurality of data storage units, The method includes (a) performing a Compression-related operation on a data storage unit of the plurality of data storage units using a free pool of storage space, associated with the fixed storage capacity, as a swap space for the Compression-related operation, (b) detecting a size of the free pool, the size of the free pool representing unallocated storage space of the data storage system, (c) comparing the size of the free pool with a restricted threshold, and (d) if the size of the free pool falls below the restricted threshold, then pausing the Compression-related operation on the data storage unit, otherwise, continuing to perform the Compression-related operation on the data storage unit. Apparatus and computer program product for practicing the method are also provided.

Abstract: An appropriate index value for guaranteeing the anonymity of data is specified, even when the data number of data included in a predetermined group increases and decreases with time.