Abstract: A method and system for a location based full text search is disclosed. The system generates interwoven bit vectors for documents that are stored in a database. The system may determine a location value based on a portion of a map that a user is viewing. The system may also receive at least one search term inputted by a user. Based on the location value and the search term, the system may generate one or more interwoven bit vectors, which may be used to query an index that identifies responsive documents stored in the database.

Abstract: Described herein are techniques for the dynamic allocation of storage capacity between multiple namespaces associated with a storage system. Data from multiple namespaces is stored in a common pool of addresses of the storage system and data from each namespace is capable of being written to any free address of the storage system at any time. Metadata associated with each of the files stored in the storage system is maintained that tracks the particular storage address or addresses and namespace associated with the file. Storage capacity can be dynamically allocated between multiple namespaces by writing data to free blocks and updating the metadata accordingly.

Abstract: A method and a system are provided for exploring a large textual data set via interactive aggregation. In one example, the method includes receiving the large textual data set and an original query template, building an index for the query template, wherein the building the index comprises ordering the index a particular way to optimize query time, receiving one or more bindings for the original query template, computing an answer to the original query template using the index and the one or more bindings, and anticipating one or more future queries that a user may submit and that are related to the original query template.

Abstract: A method and apparatus for storing and searching for handwritten memo data in a database in which the handwritten memo data are stored in the form of bit streams representing continuous and discontinuous elements constituting the handwritten memo data are provided. The memo data search method includes receiving an input of memo data, storing the memo data in the form of a bit stream, searching, when a search for the memo data is requested, a memo index database for the memo data, and displaying a search result. The handwritten memo data management method and apparatus stores and searches for the memo data regardless of orientation and size.

Abstract: A method for providing a compressed index for a stream of binary data records comprises steps of indexing a field from each record in a bitmap index, compressing stored bits in each column of the bitmap index by replacing a group of successive bits with a code and outputting the code. There is provided at least one of a first code for replacing a sequence of a first filling, a literal and a second filling, and a second code for replacing a sequence of a first literal, a filling and a second literal. In this context, a filling is a sequence of bits with the same value and a literal is a sequence of bits with different values.

Abstract: One embodiment of the present invention provides a system that implements a security policy in a database. During operation, the system receives a request associated with a set of objects in the database. Next, the system obtains a set of access control lists (ACLs) associated with the database, wherein a respective ACL specifies one or more access privileges associated with a user or user group, and wherein a respective ACLs is not specific to a particular object in the database. The system then evaluates the ACLs to obtain a set of ACL results associated with the request and processes the request by applying the set of ACL results to the objects without evaluating the ACLs repeatedly for each of the objects.

Abstract: A database query optimization method is provided for database queries that include predicates, the queries being directed to at least one database table having data in rows and columns. The method includes associating a lineage bit set with each row of the database table, the lineage bit set providing a binary predicate state indication whether each predicate is met or are not met by data in the row. A dependency bit set is established to provide a binary indication of the predicates that are included in each query. Queries are applied against the database table by a logical operation between the lineage bit set and the dependency bit set. A database query optimization data structure includes a lineage bit set associated with each row in the database table. The lineage bit set provides a binary predicate state indication whether each predicate is met or are not met by data in the row.

Abstract: A system may include determination of a plurality of data structures associated with an entity, each of the plurality of data structures associated with a respective validity period, determination of a plurality of non-overlapping time periods based on the validity periods, the plurality of non-overlapping time periods collectively spanning all of the validity periods, determination, for each of the plurality of non-overlapping time periods, of a composite data structure based on each of the data structures associated with a validity period including the non-overlapping time period, assignment of a respective document identifier to each composite data structure, each document identifier indicating the entity, and indexing of the composite data structures within an index.

Abstract: A database query of point data among two or more axes of a database is received. The database stores point data in distinct integer vectors with a shared dictionary. Thereafter, the dictionary is scanned to determine boundaries for each axis specified by the query. In response, results characterizing data responsive to the query within the determined boundaries for each axis are returned. Related apparatus, systems, techniques and articles are also described.

Abstract: Embodiments of the present invention are directed to a method and system for managing resources. The method includes receiving a request, within an electronic system, which corresponds to an object of a file system and accessing a local data structure. The data structure comprises information corresponding to a plurality of inode numbers. The method further includes performing the request and updating the local data structure based on the request. The updating of the local data structure is independent of a plurality of data structures of a cluster of servers.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training scoring models. One method includes storing data identifying a plurality of positive and a plurality of negative training images for a query. The method further includes selecting a first image from either the positive group of images or the negative group of images, and applying a scoring model to the first image. The method further includes selecting a plurality of candidate images from the other group of images, applying the scoring model to each of the candidate images, and then selecting a second image from the candidate images according to scores for the images. The method further includes determining that the scores for the first image and the second image fail to satisfy a criterion, updating the scoring model, and storing the updated scoring model.

Abstract: A content search system includes multiple pipelined search engines that implement different portions of a regular expression search operation. For some embodiments, the search pipeline includes a DFA engine, an NFA engine, and a token stitcher that combines partial match results generated by the DFA and NFA engines. The token stitcher can be configured to implement unbounded sub-expressions without utilizing resources of the DFA or NFA engines. A token stitcher may comprise an input line for receiving tokens that indicate a partial match between an input string and a regular expression, a flag bank that stores flags which, when activated, identify one or more of the sub-expressions that match the input string, a program memory that stores programs that each comprises instructions for processing tokens, and an engine configured to identify programs that are associated with a newly received token.

Abstract: Each of a plurality of Worker processes are allowed to perform any and all of the following tasks involving logged work items: (1) reading a subset of the work items from a log; (2) sequentially ordering work items for corresponding data objects; (3) applying a sequentially ordered set of work items to a corresponding data object; and (4) transmitting a subset of work items to a Worker process running on another database server in a cluster, if necessary. These tasks can be performed concurrently, at will, and as available, by the Worker processes. An improved checkpointing technique eliminates the need for the Worker processes to get to a synchronization point and stop. Instead, a Coordinator process examines the current state of progress of the Worker processes and computes a past point in the sequence of work items at which all work items before that point have been completely processed, and records this point as the checkpoint.

Abstract: A method of providing an index structure in a database comprising a plurality of types of objects, each type of objects comprising at least one object. It comprises the steps setting one position of an identity index bit vector of each object in said database to one and all other positions to zero, said position being different for each object, and forming, for each object in said database, a relation index bit vector by combining all identity index bit vectors of objects that said each object is related to, wherein each bit vector has a length equal to the number of objects.

Abstract: A Searchable Symmetric Encryption (SSE) mechanism is described which allows efficient dynamic updating of encrypted index information. The encrypted index information includes pointer information that is encrypted using a malleable encryption scheme. The SSE mechanism updates the encrypted index information by modifying at least one instance of the pointer information without decrypting the pointer information, and thereby without revealing the nature of the changes being made. In one implementation, the SSE mechanism includes a main indexing structure and a deletion indexing structure. An updating operation involves patching applied to both the main indexing structure and deletion indexing structure.

Abstract: A method performs a database query in a relational database, the query being carried out by a database engine and being based on user-defined search criteria. The method includes retrieving a number N of properties of a record within a main database table, the number N being higher than zero, creating a search criteria option for each of the N properties, creating a search criteria table for every search criteria option, creating an index for every search criteria table, and performing the database query based on a user-defined combination of a plurality of the search criteria options.

Abstract: The present invention relates to a system and method of data restoration, for instance, after the occurrence of a user error. In particular, snapshot database can be maintained that stores a copy of database data. The snapshot database does not have to store a complete copy of all data on a source database, but rather shares data that is common to both but not necessarily identical. If an error occurs on the primary database then the database can be reverted back to a point in time prior to the error by replacing source database files with snapshot files. Additionally, an undo component can be employed in conjunction with the snapshot to approach an error to a finer grained point in time. In brief, the present invention can restore a database much faster and simpler, while utilizing less space and resources than conventional data restoration technologies.

Abstract: The present invention is directed towards systems, methods and computer program products for providing query-based advertising content. According to one embodiment, a method for providing query-based advertising content comprises receiving a web query and generating an ad query associated with the web query, wherein the ad query is generated on the basis of one of a query to advertisement relationship, a query to feature vector relationship, or a rewritten query vector. An advertisement is selected on the basis of the generated ad query.

Abstract: Methods and apparatuses that search tree representations of a bitmap for available blocks to allocate in storage devices are described. An allocation request for a file may be received to initiate the search. In one embodiment, the bitmap may include an array of bits corresponding to blocks in the storage devices. Each bit may indicate whether one of the blocks is available. The tree representations may include at least one red-black tree having nodes corresponding to one or more consecutive bits in the bitmap indicating an extent of available blocks. One of the tree representations may be selected according to a file associated with an allocation request to identify an extent of available block matching the allocation request. The tree representations may be synchronized as the bitmap is updated with changes of block allocations in the storage devices.

Abstract: Inverted indexes for terms and for term separators are separately provided to minimize data redundancy. Search queries are parsed to identify terms and term separators, if any, and the corresponding inverted indexes are searched for responsive documents. Related apparatus, systems, techniques and articles are also described.

Abstract: The present disclosure includes techniques for storing indexed values that may increase computing performance. In one example, a method includes receiving a request to store an input value in a data structure. The method also includes determining the input value is not represented by an index value of the first group of index values, generating an index value that represents the input value, and determining the generated index value is not included in the first group of representations. The method also includes generating a second group of index values wherein a first size of index values of the first group is different than a second size of index values of the second group. The method also includes modifying a storage capacity of the data structure to store the index values of the second group of the second size and storing the second group of index values in the data structure.

Abstract: A method and system for a location based full text search is disclosed. The system generates interwoven bit vectors for documents that are stored in a database. The system may determine a location value based on a portion of a map that a user is viewing. The system may also receive at least one search term inputted by a user. Based on the location value and the search term, the system may generate one or more interwoven bit vectors, which may be used to query an index that identifies responsive documents stored in the database.

Abstract: A minimized journaling mechanism stores minimized journal data in a format that allows for display and outputting the journal data in human-readable form. When a change to a record occurs, instead of writing only the changed bytes, all of the bytes in each field that changed are written to the journal, along with all of the bytes in each field selected to be always journaled. A default object is created with default data in all of the fields. When the journal entry needs to be output in human-readable form, the default object is read, and the minimized journal entry is then overlaid on the default object. The result is an object that contains default data in all non-selected fields that were not changed, with the journal data in all fields and that did change and in all fields that were selected to always be journaled.

Abstract: Each of a plurality of Worker processes are allowed to perform any and all of the following tasks involving logged work items: (1) reading a subset of the work items from a log; (2) sequentially ordering work items for corresponding data objects; (3) applying a sequentially ordered set of work items to a corresponding data object; and (4) transmitting a subset of work items to a Worker process running on another database server in a cluster, if necessary. These tasks can be performed concurrently, at will, and as available, by the Worker processes. An improved checkpointing technique eliminates the need for the Worker processes to get to a synchronization point and stop. Instead, a Coordinator process examines the current state of progress of the Worker processes and computes a past point in the sequence of work items at which all work items before that point have been completely processed, and records this point as the checkpoint.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training scoring models. One method includes storing data identifying a plurality of positive and a plurality of negative training images for a query. The method further includes selecting a first image from either the positive group of images or the negative group of images, and applying a scoring model to the first image. The method further includes selecting a plurality of candidate images from the other group of images, applying the scoring model to each of the candidate images, and then selecting a second image from the candidate images according to scores for the images. The method further includes determining that the scores for the first image and the second image fail to satisfy a criterion, updating the scoring model, and storing the updated scoring model.

Abstract: Disclosed is an improved approach for managing, tracking, and querying hierarchical data in layouts. According to some aspects, hierarchical grids are employed utilizing a scheme that organizes physical objects into a set of gradually refined grids that avoids the need to maintain duplicates while enhancing the desirable characteristics of existing schemes, including fast query times, fast data structure initialization and reduced memory footprint. Each grid-cell may be further partitioned into sub-containers to more efficiently provide space pruning during query operations. According to one approach, structures maintained to track existence of objects in a descendent hierarchy.

Abstract: In general, the subject matter described in this specification can be embodied in methods, systems, and programs for providing location-based information. A geographical location of a mobile computing device is received at a server system. A request for content is received at the server system and from the mobile computing device. A search area that corresponds to the geographical location is selected by the sever system. The search area is selected from among a collection of search areas that correspond to the geographical location. At least one of the search areas in the collection is geographically nested within at least one other of the search areas. A search result that is most-relevantly ranked for the selected search area is determined by the server system. Content that represents the determined search result is provided by the server system and for receipt by the mobile computing device.

Abstract: Keys of an index are associated with corresponding data chunks in a storage system. In response to a request for a particular key, a particular location is accessed in a first data structure which is maintained in a first storage using a first portion of the particular key. In response to the particular location containing a first value, indicating that the particular key is not present in the index. In response to the particular location containing a second value, a map is accessed using a second portion of the particular key to retrieve an entry in a second data structure maintained in a second storage. The second data structure provides a collision chain to link entries of the second data structure that share a common value of the second portion of the particular key. The first data structure, map, and second data structure are part of the index.

Abstract: Techniques are provided for (2) extending SQL to support direct invocation of frequent itemset operations, (3) improving the performance of frequent itemset operations by clustering itemset combinations to more efficiently use previously produced results, and (4) making on-the-fly selection of the occurrence counting technique to use during each phase of a multiple phase frequent itemset operation. When directly invoked in an SQL statement, a frequent itemset operation may receive input from results of operations specified in the SQL statement, and provide its results directly to other operations specified in the SQL statement. By clustering itemset combinations, resources may be used more efficiently by retaining intermediate information as long as it is useful, and then discarding it to free up volatile memory.

Abstract: A system and method for enhancing bitmap indexing representation of a dataset, which comprises a plurality of cases and features, each case characterized by one or more values of each feature. Currently, the bins vector for each case in the dataset, is a binary array, which is a bitmap indexing representation of each respective feature of the case. The system and method enhance the bitmap indexing by padding each bins vector. The padding is carried out by identifying all target bit locations with a ‘1’ value and replacing at least one ‘0’ bit adjacent to a target bit location with a non-zero numerical value, thereby creating a padded bitmap index. The padding factor may be based on any mathematical or statistical factor concerning population or subpopulation relevant to each of the features of the dataset.

Abstract: Systems and methods for accessing data stored in a data array, mapping the data using a bitmap index, and processing data queries by determining positions of query attributes in the bitmap index and locating values corresponding to the positions in the data array are described herein.

Abstract: To realize a high speed merge sort method by applying a coupled node tree, which method extracts a smallest or largest key from a plurality of sorted key storage areas in each of which is stored keys including bit strings that are sorted, and generates a coupled node tree for merge while adding a processing source identifier that identifies the sorted storage area wherefrom the key has been extracted, and repeats the actions of writing out into the merged key storage area a key being obtained by a minimum or maximum value search on the coupled node tree and deleting the key, and inserting into the coupled node tree a key by extracting the key from one of the plurality of sorted key storage areas.

Abstract: Systems and methods for accessing data stored in a data array, mapping the data using a bitmap index, and processing data queries by determining positions of query attributes in the bitmap index and locating values corresponding to the positions in the data array are described herein.

Abstract: A system and method for enhancing bitmap indexing representation of a dataset, which comprises a plurality of cases and features, each case characterized by one or more values of each feature. Currently, the bins vector for each case in the dataset, is a binary array, which is a bitmap indexing representation of each respective feature of the case. The system and method enhance the bitmap indexing by padding each bins vector. The padding is carried out by identifying all target bit locations with a ‘1’ value and replacing at least one ‘0’ bit adjacent to a target bit location with a non-zero numerical value, thereby creating a padded bitmap index. The padding factor may be based on any mathematical or statistical factor concerning population or subpopulation relevant to each of the features of the dataset.

Abstract: A method and apparatus for allocating storage addresses are disclosed. The method includes: receiving a storage address allocation request; searching a level-2 bitmap in a hierarchical bitmap in bidirectional mode; outputting an idle bit according to the result of searching in the level-2 bitmap; obtaining a storage address according to the output idle bit, and allocating the storage address. The apparatus includes: a first receiving module, configured to receive a storage address allocation request; a first searching module, configured to search a level-2 bitmap in a hierarchical bitmap in bidirectional mode for an idle bit, wherein the hierarchical bitmap includes N level-1 bitmaps and the level-2 bitmap; and an allocating module, configured to: obtain a storage address according to the output idle bit in the level-2 bitmap, and allocate the obtained storage address.

Abstract: To increase space efficiency of a coupled node tree, a branch node does not have an area that holds an array element number of an array element wherein is stored the primary node of the node pair that is the link target, and the root node is disposed in an array element with a node location number 1, and the primary node is disposed in an array element whose node location number is twice the node location number of the branch node. The node location number of an array element wherein is disposed a link target node is obtained by adding the bit value in the search key at the discrimination bit position for the link source branch node to twice the node location number of the branch node.

Abstract: A coupled node tree comprises a root node and a node pair, the node pair being a branch node and leaf node, or a pair of branch nodes, or a pair of leaf nodes arranged in adjacent storage areas. The branch node includes a discrimination bit position of the search key and the first position information of a primary node, one node of a node pair of a link target. The leaf node includes the second position information of the storage area holding an index key that is the target of a search. According to a bit value of the search key of a discrimination bit position in the branch node, repeated linking to a primary node of a node pair of the link target or a node at a position in a memory area adjacent thereto until a leaf node is reached.

Abstract: When keys are to be classified into a plurality of blocks, to provide a classification method wherein the range of key values does not overlap and a method for distributing the classified keys by applying the art of a coupled-node tree. Keys are successively selected as classification keys from a key storage means holding the keys to be classified, and a classification tree, which is an application of a coupled-node tree, is generated by means of the classification keys, and the classification is done by making a correspondence between its leaf nodes and the keys to be classified into each of N blocks. The number of levels in the classification tree is constrained as a function of the block number N. A leaf node is extracted from the classification tree and the corresponding key is extracted as a classified key and distributed.

Abstract: A method, system and program product for data evolution on column oriented databases is disclosed. For an input evolution operation, reusable and non-reusable attributes are identified. For attributes in a target schema that cannot be reused from the source schema, data and bitmap indexes of those attributes are generated from source data and bitmap indexes. A decompose operation is disclosed for decomposing a table into two tables. A merge operation is disclosed in which only one input table can be reused for mergence. A second merge operation is disclosed in which both input tables cannot be reused for mergence.

Abstract: Mechanisms are provided for optimizing multiple files in an efficient format that allows maintenance of the original namespace. Multiple files and associated metadata are written to a suitcase file. The suitcase file includes index information for accessing compressed data associated with compacted files. A hardlink to the suitcase file includes an index number used to access the appropriate index information. A simulated link to a particular file maintains the name of the particular file prior to compaction.

Abstract: A system, method, and computer program product are provided for improving performance of positional text conditions in concert with bitmap-based conditions. This is accomplished, for example, by focusing positional text condition evaluation to candidate rows based on a bitmap resulting from application of a prior condition. Additionally, the result of a positional text condition may itself be provided in the form of a bitmap to accelerate further bitmap-based condition processing.

Abstract: Data de-duplication schemes reduce the amount of storage necessary to store a data set by dividing the data into segments and storing a segment identifier on a storage medium in place of each data segment. Each unique data segment is stored in a repository, and duplicate data segments are not stored. Methods and apparatus are provided for distributing data segments across multiple repositories in a data storage system, thereby reducing the quantity of data stored at a particular repository. Segments are assigned to repositories based upon a characteristic of the segments. The characteristic may be the length of the segment or some other value produced by a repeatable, uniformly-distributed function of the segment. The characteristic may be stored on the storage medium along with the segment identifier. The original data may be regenerated by retrieving the segment identifiers and characteristics from the storage medium and retrieving each segment from the repository identified by the characteristic.

Abstract: An embodiment of providing a computer-executed method of subsequence matching is provided. The method comprises receiving a search string. A plurality of subsequences for the search string are stored in a tree structure. The tree structure comprise a plurality of nodes. Each of the plurality of nodes comprises a presence bit map, a sequence bit map, and a list of address pointers. The method further includes traversing the tree structure using the search string, the presence bit map, the sequence bit map, and the list of address pointers. Additionally, the method includes identifying, in linear time, the plurality of subsequences based on the search string, the presence bit map, the sequence bit map, and the list of address pointers.

Abstract: An integrated circuit includes a search unit configured to access an input vector including a number of bits, and to find a first and a second instance of a predetermined bit value such as a logic zero or a logic one, for example. The search unit may be further configured to generate an output that includes an indication of a bit position of the first instance of the predetermined bit value within the input vector, and an indication of a bit position of the second instance of the predetermined bit value within the input vector.

Abstract: An index key is a key string formed of three or more keys whose tail-end key is unique, and a search key string for a bit string search has a unique key as its tail-end key. A branch node includes a search key sequence number expressing the position information of a key in the search key string to be used in the bit string comparison during the bit string search and the discrimination bit position for that key. Search for index keys is performed by extracting from the search key string the key with the search key sequence number in the branch node and repeatedly linking to one of the nodes of a node pair of the link target in response to the bit value at the discrimination bit position of that key until the leaf node corresponding to that index key is reached.

Abstract: Systems and methods for compressing an index are described. In one exemplary method, the results of a search are annotated and then encoded into one or more chunks of compressed data in accordance with the annotations of the results. The annotations include an indication of a best encoding method selected from a set of available encoding methods, and an indication of whether to switch to a new chunk during encoding or to continue encoding in the current chunk. Other methods are described and data processing systems and machine readable media are also described.

Abstract: Methods and apparatuses for managing file extensions in a processing system. An exemplary method of managing file extensions in a digital processing system involves a user interface and a plurality of files, each file having a name that comprises a filename and an extension. The method includes associating a file with an indicator which is user selectable for a single file in a plurality of files in said digital processing system and which indicates how to display an extension of the file, and assigning a value to the indicator, and displaying a displayed name of the file in the user interface in a style determined by the indicator.

Abstract: Using a tree configuration wherein node groups of four or more nodes composed of combinations of branch nodes, leaf nodes or empty nodes are linked into a tree form, a bit string search by a search key string is enabled by repeatedly linking to one of the nodes of a node group to which a primary node belongs in response to the bit values of keys of the search key string at the discrimination bit position included in the branch node.

Abstract: Bit string searching apparatus using a coupled node tree with a root node and a node pair stored in adjacent areas that is formed by a branch node and a leaf node, branch nodes, or leaf nodes; the branch node including a discrimination bit position in the search key and information indicating a position of a primary node that is one node of a node pair; the leaf node including an index key formed by a bit string; from the root node of an arbitrary subtree of the coupled node tree, linking is repeated based on the search key's bit value at the discrimination bit position and information indicating a position of a primary node until a leaf node is reached; an index key stored in the leaf node is obtained as a search result key of the subtree by means of the search key.

Abstract: Techniques are described for combining pieces of information from two sources. The techniques may be used to improve the performance, for example, of hash join operations that are parallelized using slaves distributed across multiple nodes. According to one technique, bitmap filtering operations are performed by the probe-phase producer slaves, rather than the probe-phase consumer slaves. To avoid having to merge separately built bitmap filter chunks, the left-hand rows may be sent to every probe-phase consumer slave. Alternatively, the merge operation may be avoided by distributing the rows of one source based on how the other source has been statically partitioned.