Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling access to APIs. One of the methods includes receiving a request from a client for a computer authorization challenge to access an application programming interface; determining a computer authorization challenge with a difficulty of completion that satisfies a target computational cost for the application programming interface; and providing the computer authorization challenge to the client for access to the application programming interface.

Abstract: Automatically extracting profile feature attribute data from event data is disclosed, including: receiving a set of event data; receiving a feature associated with a profiling technique; determining that a plurality of events associated with a user in the set of event data corresponds to a first attribute corresponding to the feature, wherein the first attribute corresponds to a first bin having a first defined value; determining that the plurality of events associated with the user in the set of event data corresponds to a second attribute corresponding to the feature, wherein the second attribute corresponds to a second bin having a second defined value; and creating a user record corresponding to the user indicating presence in the first bin and the second bin.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling access to APIs. One of the methods includes receiving a request from a client for a computer authorization challenge to access an application programming interface; determining a computer authorization challenge with a difficulty of completion that satisfies a target computational cost for the application programming interface; and providing the computer authorization challenge to the client for access to the application programming interface.

Abstract: Systems, methods and computer program products are disclosed for managing availability of a slave components executing in a distributed system. A master server and a heartbeat protocol may be provided. The heartbeat protocol may define state machines for the master server and/or the slave components. The master server and/or the slave components may act in accordance with the state machines to manage availability of the slave components in the distributed system.

Abstract: Techniques are disclosed for synchronizing a primary data system with an auxiliary data system that processes data for the primary data system. In one embodiment, how current the primary data system and the auxiliary data system are is determined. Requests sent from the primary data system that were not processed by the auxiliary data system are determined. The requests are resent to the auxiliary data system for processing.

Abstract: Provided are techniques for partitioning a physical index into one or more physical partitions; assigning each of the one or more physical partitions to a node in a cluster of nodes; for each received document, assigning an assigned-doc-ID comprising an integer document identifier; and, in response to assigning the assigned-doc-ID to a document, determining a cut-off of assignment of new documents to a current virtual-index-epoch comprising a first set of physical partitions and placing the new documents into a new virtual-index-epoch comprising a second set of physical partitions by inserting each new document to a specific one of the physical partitions in the second set using one or more functions that direct the placement based on one of the assigned-doc-id, a field value derived from a set of fields obtained from the document, and a combination of the assigned-doc-id and the field value.

Abstract: Systems, methods and computer program products are disclosed for managing availability of a slave components executing in a distributed system. A master server and a heartbeat protocol may be provided. The heartbeat protocol may define state machines for the master server and/or the slave components. The master server and/or the slave components may act in accordance with the state machines to manage availability of the slave components in the distributed system.

Abstract: Systems, methods and computer program products are disclosed for managing availability of a slave components executing in a distributed system. A master server and a heartbeat protocol may be provided. The heartbeat protocol may define state machines for the master server and/or the slave components. The master server and/or the slave components may act in accordance with the state machines to manage availability of the slave components in the distributed system.

Abstract: In a data mining system, data is gathered into a data store using, e.g., a Web crawler. The data is classified into entities. Data miners use rules to process the entities and append respective keys to the entities representing characteristics of the entities as derived from rules embodied in the miners. With these keys, characteristics of entities as defined by disparate expert authors of the data miners are identified for use in responding to complex data requests from customers.

Abstract: In a data mining system, data is gathered into a data store using, e.g., a Web crawler. The data is classified into entities. Data miners use rules to process the entities and append respective keys to the entities representing characteristics of the entities as derived from rules embodied in the miners. With these keys, characteristics of entities as defined by disparate expert authors of the data miners are identified for use in responding to complex data requests from customers.

Abstract: A dynamic Bloom filter comprises a cascaded set of Bloom filters. The system estimates or guesses a cardinality of input items, selects a number of hash functions based on the desired false positive rate, and allocates memory for an initial Bloom filter based on the estimated cardinality and desired false positive rate. The system inserts items into the initial Bloom filter and counts the bits set as they are inserted. If the number of bits set in the current Bloom filter reaches a predetermined target, the system declares the current Bloom filter full. The system recursively generates additional Bloom filters as needed for items remaining after the initial Bloom filter is filled; items are checked to eliminate duplicates. Each of the set of Bloom filters is individually queried to identify a positive or negative in response to a query.

Abstract: Provided are techniques for partitioning a physical index into one or more physical partitions; assigning each of the one or more physical partitions to a node in a cluster of nodes; for each received document, assigning an assigned-doc-ID comprising an integer document identifier; and, in response to assigning the assigned-doc-ID to a document, determining a cut-off of assignment of new documents to a current virtual-index-epoch comprising a first set of physical partitions and placing the new documents into a new virtual-index-epoch comprising a second set of physical partitions by inserting each new document to a specific one of the physical partitions in the second set using one or more functions that direct the placement based on one of the assigned-doc-id, a field value derived from a set of fields obtained from the document, and a combination of the assigned-doc-id and the field value.

Abstract: Systems, methods and computer program products are disclosed for managing availability of a slave components executing in a distributed system. A master server and a heartbeat protocol may be provided. The heartbeat protocol may define state machines for the master server and/or the slave components. The master server and/or the slave components may act in accordance with the state machines to manage availability of the slave components in the distributed system.

Abstract: Systems, methods and articles of manufacture are disclosed for managing availability of a component executing in a distributed system. The component may have an address space closed to the distributed system. In one embodiment, the component may be initiated. A state of the component may be analyzed to determine the availability of the component. The determined availability may be transmitted to the distributed system. The component may also be restarted responsive to a request from the distributed system to restart the component.

Abstract: A cache-aware Bloom filter system segments a bit vector of a cache-aware Bloom filter into fixed-size blocks. The system hashes an item to be inserted into the cache-aware Bloom filter to identify one of the fixed-size blocks as a selected block for receiving the item and hashes the item k times to generate k hashed values for encoding the item for insertion in the in the selected block. The system sets bits within the selected block with addresses corresponding to the k hashed values such that accessing the item in the cache-aware Bloom filter requires accessing only the selected block to check the k hashed values. The size of the fixed-size block corresponds to a cache-line size of an associated computer architecture on which the cache-aware Bloom filter is installed.

Abstract: Disclosed is an evaluation technique for text search with black-box scoring functions, where it is unnecessary for the evaluation engine to maintain details of the scoring function. Included is a description of a system for dealing with blackbox searching, proofs of correctness, as well experimental evidence showing that the performance of the technique is comparable in efficiency to those techniques used in custom-built engines.

Abstract: Systems, methods and articles of manufacture are disclosed for synchronizing a primary data system with an auxiliary data system that processes data for the primary data system. In one embodiment, how current the primary data system and the auxiliary data system are may be determined. Requests sent from the primary data system that were not processed by the auxiliary data system may be determined. The requests may be resent to the auxiliary data system for processing.

Abstract: A dynamic Bloom filter comprises a cascaded set of Bloom filters. The system estimates or guesses a cardinality of input items, selects a number of hash functions based on the desired false positive rate, and allocates memory for an initial Bloom filter based on the estimated cardinality and desired false positive rate. The system inserts items into the initial Bloom filter and counts the bits set as they are inserted. If the number of bits set in the current Bloom filter reaches a predetermined target, the system declares the current Bloom filter full. The system recursively generates additional Bloom filters as needed for items remaining after the initial Bloom filter is filled; items are checked to eliminate duplicates. Each of the set of Bloom filters is individually queried to identify a positive or negative in response to a query.

Abstract: Systems, methods and articles of manufacture are disclosed for managing availability of a component executing in a distributed system. The component may have an address space closed to the distributed system. In one embodiment, the component may be initiated. A state of the component may be analyzed to determine the availability of the component. The determined availability may be transmitted to the distributed system. The component may also be restarted responsive to a request from the distributed system to restart the component.

Abstract: Provided is a technique for building an index. A new indexi+1 is built and an anchor text tablei+1 and a duplicates tablei+1 are output using a storei, a delta store, and previously generated global analysis computationsi, wherein the previously generated global analysis computationsi include an anchor text tablei, a rank tablei, and a duplicates tablei. New global analysis computationsi+1 are generated using the anchor text tablei+1, the duplicates tablei+1, and the previously generated global analysis computationsi.