Abstract: The disclosed embodiments provide a system for managing a data store. During operation, the system stores a set of pending updates to a data store in a registry. Next, the system executes an asynchronous process that applies a first subset of updates from the registry as writes to records in the data store without blocking processing of read queries of the data store. Upon completing a write by the asynchronous process at a second portion of the data store, the system updates the registry with an indication of the completed write at the second portion of the data store. During processing of a read query of the data store, the system applies a second subset of updates from the registry to a result of the read query. Finally, the system returns the result in a response to the read query.

Abstract: In order to ingest data from an arbitrary source in a set of sources, a computer system accesses predefined configuration instructions. Then, the computer system generates a dynamic data-ingestion pipeline that is compatible with a Hadoop file system based on the predefined configuration instructions. This dynamic data-ingestion pipeline includes a modular arrangement of operators from a set of operators that includes: an extraction operator for extracting the data of interest from the source, a converter operator for transforming the data, and a quality-checker operator for checking the transformed data. Moreover, the computer system receives the data from the source. Next, the computer system processes the data using the dynamic data-ingestion pipeline as the data is received without storing the data in memory for the purpose of subsequent ingestion processing.

Abstract: The disclosed embodiments provide a system for performing data management. During operation, the system obtains a first schema with a first syntax for describing a first data set and a second schema with a second syntax for describing a second data set. Next, the system converts the first schema into a first standardized form with a standardized syntax and the second schema into a second standardized form with the standardized syntax. The system then outputs the first and second standardized forms for use in accessing the first and second data sets.

Abstract: The disclosed embodiments provide a system for managing execution of a data flow. During operation, the system obtains a data dependency description for a data flow, wherein the data dependency description includes data sources to be consumed by the data flow, data targets to be produced by the data flow, and one or more data ranges associated with the data sources and the data targets. Next, the system uses the data dependency description to determine an availability of the data sources in an execution environment. After the availability of the data sources in the execution environment is confirmed, the system generates output for initiating execution of the data flow in the execution environment.

Abstract: In order to ingest data from an arbitrary source in a set of sources, a computer system accesses predefined configuration instructions. Then, the computer system generates a dynamic data-ingestion pipeline that is compatible with a Hadoop file system based on the predefined configuration instructions. This dynamic data-ingestion pipeline includes a modular arrangement of operators from a set of operators that includes: an extraction operator for extracting the data of interest from the source, a converter operator for transforming the data, and a quality-checker operator for checking the transformed data. Moreover, the computer system receives the data from the source. Next, the computer system processes the data using the dynamic data-ingestion pipeline as the data is received without storing the data in memory for the purpose of subsequent ingestion processing.

Abstract: A method of maintaining a data store is disclosed. The data store is distributed across a plurality of storage provider instances, including a master storage provider instance and a plurality of additional storage provider instances. A master data log is updated to reflect a change to the data store. The master data log is synchronized with a plurality of replicas of the master data log. The change is propagated across the plurality of additional storage provider instances based on an analysis of the replicas.

Abstract: During a storage technique, multiple messages (such as emails) associated with a user of a communication application are received. Then, the multiple messages are stored in a message table associated with the user and the multiple messages are indexed in an index associated with the user. This index may be divided into multiple divisions if a total number of messages stored in the message table exceeds a threshold value, where each division corresponds to messages received during a different time interval.

Abstract: As part of a query-processing technique, in response to receiving queries for information stored in a distributed storage system and associated query-processing information identifying sets of predefined operations to perform on one or more databases in the distributed storage system, the distributed storage system accesses sets of predefined operations. Then, for the sets of predefined operations on the one or more databases, the distributed storage system uses one or more adaptors for multiple storage subsystems in the distributed storage system and one or more indexes corresponding to collections of information in the one or more databases. Moreover, the distributed storage system executes the sets of predefined operations on the multiple storage subsystems to obtain results for the queries. The sets of predefined operations may be defined by different users, so that the distributed storage system can be adapted to these users' needs.

Abstract: As part of a query-processing technique, in response to receiving queries for information stored in a distributed storage system and associated query-processing information identifying sets of predefined operations to perform on one or more databases in the distributed storage system, the distributed storage system accesses sets of predefined operations. Then, for the sets of predefined operations on the one or more databases, the distributed storage system uses one or more adaptors for multiple storage subsystems in the distributed storage system and one or more indexes corresponding to collections of information in the one or more databases. Moreover, the distributed storage system executes the sets of predefined operations on the multiple storage subsystems to obtain results for the queries. The sets of predefined operations may be defined by different users, so that the distributed storage system can be adapted to these users' needs.

Abstract: As part of a query-processing technique, in response to receiving queries for information stored in a distributed storage system and associated query-processing information identifying sets of predefined operations to perform on one or more databases in the distributed storage system, the distributed storage system accesses sets of predefined operations. Then, for the sets of predefined operations on the one or more databases, the distributed storage system uses one or more adaptors for multiple storage subsystems in the distributed storage system and one or more indexes corresponding to collections of information in the one or more databases. Moreover, the distributed storage system executes the sets of predefined operations on the multiple storage subsystems to obtain results for the queries. The sets of predefined operations may be defined by different users, so that the distributed storage system can be adapted to these users' needs.

Abstract: As part of a query-processing technique, in response to receiving queries for information stored in a distributed storage system and associated query-processing information identifying sets of predefined operations to perform on one or more databases in the distributed storage system, the distributed storage system accesses sets of predefined operations. Then, for the sets of predefined operations on the one or more databases, the distributed storage system uses one or more adaptors for multiple storage subsystems in the distributed storage system and one or more indexes corresponding to collections of information in the one or more databases. Moreover, the distributed storage system executes the sets of predefined operations on the multiple storage subsystems to obtain results for the queries. The sets of predefined operations may be defined by different users, so that the distributed storage system can be adapted to these users' needs.

Abstract: During a search technique, indexes associated with user accounts of users that are using the communication application are opened in memory from a transactional key-value database. These indexes encompass messages (such as emails) communicated using the communication application, and each of the users has at least one separate, associated index. When a search query associated with a target user account is received from the communication application, a search based on the search query is performed by reading the associated index in the memory from the transactional key-value database without managing the index using a file system. Then, a result for the search query is returned.

Abstract: During a storage technique, multiple messages (such as emails) associated with a user of a communication application are received. Then, the multiple messages are stored in a message table associated with the user and the multiple messages are indexed in an index associated with the user. This index may be divided into multiple divisions if a total number of messages stored in the message table exceeds a threshold value, where each division corresponds to messages received during a different time interval.

Abstract: A method of maintaining a data store is disclosed. The data store is distributed across a plurality of storage provider instances, including a master storage provider instance and a plurality of additional storage provider instances. A master data log is updated to reflect a change to the data store. The master data log is synchronized with a plurality of replicas of the master data log. The change is propagated across the plurality of additional storage provider instances based on an analysis of the replicas.

Abstract: An online advertising system integrates third party agents to permit the third party agents to participate in auctions to bid on a per opportunity basis. An advertising exchange module receives requests for opportunities to serve online advertisements to users. In response, an advertising exchange module applies one or more business rules to determine third party agents that qualify to serve the online advertisement. A bid gateway module generates and transmits requests for bids to the third party agents. The bid gateway module then receives bids from the third party agents in response to the requests for bids. The advertising exchange module then selects an advertisement based on the bid. The online advertisement exchange system provides a unified marketplace to permit integrator networks to bid on both ads pursuant to guaranteed contracts and ads not subject to guaranteed contracts (e.g., non-guaranteed ads).

Abstract: A method for conducting demand-side, real-time bidding includes: constructing an exchange graph (G) of nodes representing publishers and third-party advertisers that provide third-party ads, the graph including directed edges connected between the nodes that represent bilateral business agreements; receiving an opportunity for displaying an ad to a user that is associated with a publisher node; exploring the graph to identify third-party ads reachable from the publisher node through a valid path of the exchange graph with which corresponding third-party advertisers are thereby eligible to bid on the opportunity; retrieving statistics from the memory associated with historical selectivity of demand predicates for the third-party ads; and initiating, before beginning graph exploration on at least some paths to the third-party ads, a call out for bids from at least some of the third-party advertisers for the corresponding third-party ads that are unlikely to be discarded during the graph exploration based on the

Abstract: Automatic management of networked publisher-subscriber relationships in an advertising server network. The method comprises steps for constructing a directed graph representation comprising at least one publisher node (e.g. an Internet property), at least one subscriber node (e.g. an Internet advertiser), at least one intermediary node (e.g. an Internet advertising agent), and at least one edge (e.g. an advertising target predicate) wherein any one of the edges is directly associated with at least one target predicate. The directed graph representation is used in conjunction with an inverted index for retrieving a valid node list comprising only nodes having at least one target predicate that matches at least one event predicate. The event predicate (as well as any target predicate) is any arbitrarily complex Boolean expression, and is used in producing a result node list comprising only nodes that concurrently match the event predicate with an advertising target predicate and are reachable.

Abstract: A modeling system to evaluate cost-based viability of a real-time, auction-based advertising system with third-party integration includes an exchange server configured to receive advertising bids, create bid requests to third-party entities based thereon, and select a winning bid from responses to the requests. A computer, coupled with the exchange server: computes a plurality of valid paths from publishers to and from the third-party entities through the exchange server; estimates server and network costs, including fixed hardware costs and variable operational costs, amortized over a predetermined period of time, based on a number of average queries per second (QPS) transmitted at different portions of the valid paths; compares current periodic fees paid by the third-party entities to the amortized costs, to determine cost-based system viability; and determines updates, if needed, to the periodic fees based on the plurality of costs to maintain cost-based system viability.

Abstract: An online advertising system integrates third party agents to permit the third party agents to participate in auctions to bid on a per opportunity basis. An advertising exchange module receives requests for opportunities to serve online advertisements to users. In response, an advertising exchange module applies one or more business rules to determine third party agents that qualify to serve the online advertisement. A bid gateway module generates and transmits requests for bids to the third party agents. The bid gateway module then receives bids from the third party agents in response to the requests for bids. The advertising exchange module then selects an advertisement based on the bid. The online advertisement exchange system provides a unified marketplace to permit integrator networks to bid on both ads pursuant to guaranteed contracts and ads not subject to guaranteed contracts (e.g., non-guaranteed ads).

Abstract: An online advertising system integrates third party agents to permit the third party agents to participate in auctions to bid on a per opportunity basis. An advertising exchange module receives requests for opportunities to serve online advertisements to users. In response, an advertising exchange module applies one or more business rules to determine third party agents that qualify to serve the online advertisement. A bid gateway module generates and transmits requests for bids to the third party agents. The bid gateway module then receives bids from the third party agents in response to the requests for bids. The advertising exchange module then selects an advertisement based on the bid. The online advertisement exchange system provides a unified marketplace to permit integrator networks to bid on both ads pursuant to guaranteed contracts and ads not subject to guaranteed contracts (e.g., non-guaranteed ads).