Abstract: Embodiments for data tagging in a computing environment are provided. A write operation in an operating system (OS) file system level storage layer is intercepted. A set of signatures in a sub-chunk level is calculated. The set of signatures are aligned to the beginning of an OS file system-level object and stored in a memory location, such as a cache, along with file system information relating to the write operation and to the file system-level object that the data is written into. Following file system processing, and as the data is written into storage in blocks, the write operation is intercepted in the block level storage layer. A secondary set of signatures in a sub-block level is calculated using a common algorithm used to create the original set of signatures. The sets of signatures are compared against each other, and blocks of the data having matching signatures are tagged with the file system information stored in the memory location.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training scoring models. One method includes storing data identifying a plurality of positive and a plurality of negative training images for a query. The method further includes selecting a first image from either the positive group of images or the negative group of images, and applying a scoring model to the first image. The method further includes selecting a plurality of candidate images from the other group of images, applying the scoring model to each of the candidate images, and then selecting a second image from the candidate images according to scores for the images. The method further includes determining that the scores for the first image and the second image fail to satisfy a criterion, updating the scoring model, and storing the updated scoring model.

Abstract: An embodiment of providing a computer-executed method of subsequence matching is provided. The method comprises receiving a search string. A plurality of subsequences for the search string are stored in a tree structure. The tree structure comprise a plurality of nodes. Each of the plurality of nodes comprises a presence bit map, a sequence bit map, and a list of address pointers. The method further includes traversing the tree structure using the search string, the presence bit map, the sequence bit map, and the list of address pointers. Additionally, the method includes identifying, in linear time, the plurality of subsequences based on the search string, the presence bit map, the sequence bit map, and the list of address pointers.

Abstract: A personalized firewall or other network gateway is provided by a method of matching a data packet to a rule in a network gateway having a rule base. One or more identification values are determined based on the data packet and property value(s) associated with said one or more identification values are queried and received from a property server. The property value(s) describe for example allowed connections and services for an entity associated with the identification value(s). The property value(s) are compared to at least one rule in the rule base, said at least one rule comprising property value(s) and an action, and the action defined in said at least one rule is taken, if said property value(s) of the rule match corresponding property value(s) associated with said one or more identification values.

Abstract: Systems, methods, and other embodiments associated with logically expanding a document and determining the relevance of the logically expanded document to a query are described. One method embodiment includes searching an index to locate a document identifier for a document in which a query term appears. The method includes determining whether the index entry includes an expansion identifier, and, if so, producing a logically expanded document. The logically expanded document may include both a document associated with the document identifier and a document associated with the expansion identifier. The method may then determine a relevance value of the logically expanded document with respect to the query and may provide a signal corresponding to the relevance value.

Abstract: An indexing system uses a graph-like data structure that clusters features indexes together. The minimum atomic value in the data structure is represented as a leaf node which is either a single feature index or a sequence of two or more feature indexes when a minimum sequence length is imposed. Root nodes are formed as clustered collections of leaf nodes and/or other root nodes. Context nodes are formed from root nodes that are associated with content that is being indexed. Links between a root node and other nodes each include a sequence order value that is used to maintain the sequencing order for feature indexes relative to the root node. The collection of nodes forms a graph-like data structure, where each context node is indexed according to the sequenced pattern of feature indexes. Clusters can be split, merged, and promoted to increase the efficiency in searching the data structure.

Abstract: A minimized journaling mechanism stores minimized journal data in a format that allows for display and outputting the journal data in human-readable form. When a change to a record occurs, instead of writing only the changed bytes, all of the bytes in each field that changed are written to the journal, along with all of the bytes in each field selected to be always journaled. A default object is created with default data in all of the fields. When the journal entry needs to be output in human-readable form, the default object is read, and the minimized journal entry is then overlaid on the default object. The result is an object that contains default data in all non-selected fields that were not changed, with the journal data in all fields and that did change and in all fields that were selected to always be journaled.

Abstract: A method, system and program product for data evolution on column oriented databases is disclosed. For an input evolution operation, reusable and non-reusable attributes are identified. For attributes in a target schema that cannot be reused from the source schema, data and bitmap indexes of those attributes are generated from source data and bitmap indexes. A decompose operation is disclosed for decomposing a table into two tables. A merge operation is disclosed in which only one input table can be reused for mergence. A second merge operation is disclosed in which both input tables cannot be reused for mergence.

Abstract: An improved system and method for classifying tags of content using a hyperlinked corpus of classified web pages is provided. An anchor text index may be searched to find anchor texts that may match text of the tag, documents referenced by the matching anchor texts may be found, and the documents referenced by the matching anchor texts may be grouped to disambiguate multiple classifications that result from matching the anchor texts with the categories of the reference documents. To resolve ambiguity between multiple classifications, weighted classifications may be used where each document may be assigned a positive weight for a mapping to a category to indicate the confidence of the classification of the document to the category. The classification for the grouping of the documents referenced by the matching anchor texts with greatest frequency may be selected and output as the classification for the tag.

Abstract: Embodiments of the present invention address deficiencies of the art in respect to structured content storage and provide a novel and non-obvious method, system and computer program product for managing structured content stored in a BLOB. In an embodiment of the invention, a performance optimized structured content management system can include a content repository, a content manager configured to provide access to structured content in the content repository and multiple different performance optimized containers disposed in the content repository. Each of the containers can store a portion of the structured content, and each of the containers can include a flattened form of original structured content in a primary binary large object (BLOB) and a parsed form of the original structured content in a secondary BLOB, the parsed form of the original structured content in the secondary BLOB indexing the flattened form of the original structured content in the primary BLOB.

Abstract: System and method for visually tracking a target object silhouette in a plurality of video frames under very general conditions. The tracker does not make any assumption about the object or the scene. The tracker works by approximating, in each frame, a PDF (probability distribution function) of the target's bitmap and then estimating the maximum a posteriori bitmap. The PDF is marginalized over all possible motions per pixel, thus avoiding the stage in which optical flow is determined. This is an advantage over other general-context trackers that do not use the motion cue at all or rely on the error-prone calculation of optical flow. Using a Gibbs distribution with a first order neighborhood system yields a bitmap PDF whose maximization may be transformed into that of a quadratic pseudo-Boolean function, the maximum of which is approximated via a reduction to a maximum-flow problem.

Abstract: Techniques are provided for (2) extending SQL to support direct invocation of frequent itemset operations, (3) improving the performance of frequent itemset operations by clustering itemset combinations to more efficiently use previously produced results, and (4) making on-the-fly selection of the occurrence counting technique to use during each phase of a multiple phase frequent itemset operation. When directly invoked in an SQL statement, a frequent itemset operation may receive input from results of operations specified in the SQL statement, and provide its results directly to other operations specified in the SQL statement. By clustering itemset combinations, resources may be used more efficiently by retaining intermediate information as long as it is useful, and then discarding it to free up volatile memory.

Abstract: The cost of running a query (having a query range) on a multidimensional database may be estimated using a process factors criteria beyond merely the number of affected records. First, a materialized view of the database may be represented as a container of tuples, sorted by key. Then keys may be stepped through, each key representing a mapping of a combination of tuples from the container. At each step, the process may request the next smallest key in the query range greater than or equal to the key of the current step, which results in the tuple in the database whose key is the smallest, greater than or equal to the requested key, and determine if the resulting is in the query range. The cost of the query may then be estimated as the number of tuples upon which the range check was performed.

Abstract: Dynamic processor allocation is implemented based upon bitmap data density. A bitmap index is used to process the query. A bitmap is created for the query. The bitmap is partitioned into single I/O operations. A variable partition size is provided based upon data density. Data density for each partition is calculated. Processors are assigned based upon data density of each partition. Then the partitions are processed and query results are returned.

Abstract: Database systems using BitSets to model and manage relationships between and/or among entities describing the corresponding fast user defined scalar and column functions used for BitSets, including a technique to model categorization and steps to perform rule evaluation from retrieving the entities associated with ancestors and descendent categories, as well as a technique to model Boolean Expression using Bitsets for relationships and the steps to perform fast Boolean evaluation.

Abstract: A disclosed embodiment relates to a system for populating an index table. The system comprises a component that decomposes a spatial object into a set of object pairs, a component that employs a value associated with the object pairs to sort the set of object pairs into a sequence of object pairs, a component that uses a sliding border analysis to create a subset of the sequence of object pairs depending upon a position of the object pair in the sequence of object pairs, and a component that populates the index table with the subset of object pairs.

Abstract: Methods and apparatus provide for a local search indexer to allow for an optimized search within a web server that returns accurate search results while maintaining independent control as to defining search patterns, search prioritization, and updated content available for search. Specifically, the local search indexer organizes content according to a hierarchical directory structure at a web server. The hierarchical directory structure includes at least one directory level that provides at least one directory for storing the content. The local search indexer builds a search index associated with the directory and stores the search index at the web server. The search index is populated with indexed content based on an update of the content stored in the directory. The local search indexer employs a search engine, at the web server, to process search queries against the indexed content to provide a search result that includes the update of the content.

Abstract: Techniques enable locating a position within a file that corresponds to a desired seek time without having access to an index specifying the desired seek time's position. An iterative process may be used to estimate the position that corresponds to the desired seek time. The process may iterate through multiple estimations until a difference between a time corresponding to an estimated position and the desired seek time is within an acceptable amount or until the process reaches an iteration threshold. The file may then be played beginning at or near the desired seek time. The techniques may therefore allow a user to seek within a file while the user progressively downloads or streams the file.

Abstract: A database query optimization method is provided for database queries that include predicates, the queries being directed to at least one database table having data in rows and columns. The method includes associating a lineage bit set with each row of the database table, the lineage bit set providing a binary predicate state indication whether each predicate is met or are not met by data in the row. A dependency bit set is established to provide a binary indication of the predicates that are included in each query. Queries are applied against the database table by a logical operation between the lineage bit set and the dependency bit set. A database query optimization data structure includes a lineage bit set associated with each row in the database table. The lineage bit set provides a binary predicate state indication whether each predicate is met or are not met by data in the row.

Abstract: A computing device executing a file system maintains a search tree that includes extents for managing first regions of unallocated storage space and bitmaps for managing second regions of unallocated storage space. For each region of unallocated storage space, the file system determines whether to manage that region using an extent or a bitmap based on one or more space management criteria.

Abstract: A search engine searches a database for key candidates having a longest matching prefix with a search key. The search engine includes first stage decoders each having a matrix of interconnected cells for identifying preliminary candidate keys in the database. The search engine also includes a second stage decoder having a matrix of interconnected cells for identifying secondary candidate keys from the preliminary candidate keys. Additionally, the search engine includes a longest candidate prefix module to determine whether one of the secondary candidate keys matches the search key. In some embodiments, the search engine includes a longest prefix match module for identifying the secondary candidate key having a longest matching prefix with the search key.

Abstract: A statistical tree representing an eXtensible Markup Language (XML) Schema document (XSD) is generated. The statistical tree captures information defined by the XSD by representing elements, attributes, and enumerations of the XSD as branches, nodes, and leaves of the statistical tree. The statistical tree has bits corresponding to nodes of the statistical tree. An XML document defined by the XSD is adaptively encoded, or compressed, as a number of bits based on the statistical tree that has been generated. The number of bits encoding the XML document are decoded, or decompressed, to yield the XML document also based on the statistical tree that has been generated.

Abstract: A method and system for parsing of input addresses for further automated processing. A relevant locale for an input address is determined. Based on the locale, an applicable parsing tree is provided so that different address formats can be tested against the input address. The parsing tree is generated from a local address format specification that defines permissible formats for the locale. The local address format specification and the local address component rules are provided to a parsing engine to determine one or more potential parsed addresses based on compliance with specifications. The local address component rules specification is applied to the input address to determine one or more branches of the parsing tree for which the input address matches criteria of the component rules specification. Penalties may be assigned to branches of the tree when disfavored matches occur.

Abstract: Systems and methods for compressing indices are described. In one aspect, a plurality of items are selected where each item has an entry in an inverted index and each item entry comprises a listing of articles that the item appears in. At least a first item entry and a second item entry are determined for compression and the second item entry is compressed into the first item entry resulting in a compressed first item entry.

Abstract: Methods and apparatuses for managing file extensions in a processing system. An exemplary method of managing file extensions in a digital processing system involves a user interface and a plurality of files, each file having a name that comprises a filename and an extension. The method includes associating a file with an indicator which is user selectable for a single file in a plurality of files in said digital processing system and which indicates how to display an extension of the file, and assigning a value to the indicator, and displaying a displayed name of the file in the user interface in a style determined by the indicator.

Abstract: A database management system that has bitmap values in which set bits in a representation of a bitstring specify a set of objects whose definitions are built into the database management system. The database management system further includes user-accessible operations on the bitmap values. The bitmap values are represented by means of a mapping specifier that specifies a range of the set of objects and a representation of a string of bits that has been mapped onto the set of object specified by the range specifier. Objects containing bitmap values may be indexed by means of an index tree that includes summary bitmap values.

Abstract: An element of a file system is virtually deleted by creating a deletion marker for the element. Two or more separate physical file system directories are presented as one merged (virtual) file system directory to a process running in a silo. The operating system provides the merged view of the file system directories by monitoring file system requests made by processes in silos on a computer or computer system and filtering out those elements associated with deletion markers. Special processing is invoked in response to detecting certain types of file system access requests, including: enumeration, open, create, rename or delete.

Abstract: Systems and methods for compressing an index are described. In one exemplary method, the results of a search are annotated and then encoded into one or more chunks of compressed data in accordance with the annotations of the results. The annotations include an indication of a best encoding method selected from a set of available encoding methods, and an indication of whether to switch to a new chunk during encoding or to continue encoding in the current chunk. Other methods are described and data processing systems and machine readable media are also described.

Abstract: A system and method for locating an unallocated bit in a bitmap array includes traversing the bitmap array using a plurality of pointers to locate a unit. The unit includes a plurality of entities and at least one of the plurality of entities is unallocated. The method further includes traversing the at least one of the plurality of unallocated entities in the unit to obtain an unallocated entity. The unit is associated with at least one pointer, and the at least one pointer is associated with a plurality of threshold values and a fill count, the fill count being less than a maximum fill count of the bitmap array.

Abstract: One embodiment is a method that uses a hierarchical bitmap tree in a storage system to store where available and unavailable memory space exists. Bitmap blocks are retrieved from the hierarchical bitmap tree to locate the available memory space.

Abstract: Described is a method comprising initializing a first set of bits corresponding to a first plurality of predicate indices to a first value, and initializing a second set of bits corresponding to a second plurality of predicate indices to a second value. A subscription is processed. The subscription includes a plurality of subscription predicates, each of the subscription predicates associated with corresponding predicate indices in one of the first and second pluralities. An event is processed. The event includes an event predicate. When the event predicate matches a first predicate index in the first plurality, a first bit associated with the first predicate index is toggled to the second value. When the event predicate matches a second predicate index in the second plurality, a second bit associated with the second predicate index is toggled to the first value. When each bit associated with the corresponding predicate indices is the second value, the event is outputted to a source of the subscription.

Abstract: A method, article of manufacture, and apparatus for tracking a plurality of objects being stored are disclosed. In an embodiment, this comprises computing the hash value of the name of each object being stored, transforming the hash value into a plurality of bytes such that none of the bytes has the value of a flag used by the system, concatenating the transformed hashed values into a hint, and storing the hint in an index. In an embodiment, bytes having the flag value are mapped to an unused value during the transformation. In an embodiment, the hint is retrieved from the index and hashed values are transformed back. Mapped values are restored to the flag values. This allows use of the hint with a system that uses a flag in the index as an indicator; for example, to indicate that an endpoint has been reached.