Abstract: Application object materialization is described. A system inputs data objects from tables in a source database, and then outputs the data objects to tables in a target database. A materializer constructs an application object based on applying relationships between the tables in the source database to the data objects in the tables in the target database. The system receives an application object request from an application associated with the target database, and outputs the application object.

Abstract: Object materialization and replication policies are described. A system identifies data objects in tables in a source database by applying a data object replication policy to a superset of the data objects in the tables in the source database. The system inputs the data objects from the tables in the source database, and outputs the data objects to tables in a target database. A materializer constructs an application object based on applying an application object materialization policy associated with the tables in the source database to the data objects in the tables in the target database. The system receives an application object request from an application associated with the target database, and outputs the application object.

Abstract: Disclosed are a method and/or a system to improve data synchronization and integration of heterogeneous databases distributed across enterprise and/or cloud using bi-directional transactional bus of asynchronous change data system. In one embodiment, a method of snapshot materialization and application consistency includes running a change capture system to capture all changes by collecting a change capture data, running an initial bulk load of all data in a source system, and applying all change transactions to a particular transaction id. The method includes removing a reappearance of a record using keys that handle de-duplication of entries and deeming a snapshot of a target system. The change capture data concerns in the source system in an order of its occurrence. A logical clock value determines the order in which the changes have occurred. The changes are a transactional and/or a non-transactional. The transaction boundaries are preserved as part of the change capture data.

Abstract: In order to ingest data from an arbitrary source in a set of sources, a computer system accesses predefined configuration instructions. Then, the computer system generates a dynamic data-ingestion pipeline that is compatible with a Hadoop file system based on the predefined configuration instructions. This dynamic data-ingestion pipeline includes a modular arrangement of operators from a set of operators that includes: an extraction operator for extracting the data of interest from the source, a converter operator for transforming the data, and a quality-checker operator for checking the transformed data. Moreover, the computer system receives the data from the source. Next, the computer system processes the data using the dynamic data-ingestion pipeline as the data is received without storing the data in memory for the purpose of subsequent ingestion processing.

Abstract: In order to ingest data from an arbitrary source in a set of sources, a computer system accesses predefined configuration instructions. Then, the computer system generates a dynamic data-ingestion pipeline that is compatible with a Hadoop file system based on the predefined configuration instructions. This dynamic data-ingestion pipeline includes a modular arrangement of operators from a set of operators that includes: an extraction operator for extracting the data of interest from the source, a converter operator for transforming the data, and a quality-checker operator for checking the transformed data. Moreover, the computer system receives the data from the source. Next, the computer system processes the data using the dynamic data-ingestion pipeline as the data is received without storing the data in memory for the purpose of subsequent ingestion processing.

Abstract: A method of maintaining a data store is disclosed. The data store is distributed across a plurality of storage provider instances, including a master storage provider instance and a plurality of additional storage provider instances. A master data log is updated to reflect a change to the data store. The master data log is synchronized with a plurality of replicas of the master data log. The change is propagated across the plurality of additional storage provider instances based on an analysis of the replicas.

Abstract: A system, methods, and apparatus are provided for performing capacity planning within a system that experiences high volumes of data having high velocity and high variability. Based on historical traffic, a forecast is generated for one or more relatively coarse time periods (e.g., weeks, days), and is decomposed to yield finer-grained forecasts (e.g., for hours, minutes) by applying a distribution index also generated from historical traffic. Estimated replication latency for the forecast period can be calculated from the traffic forecast and an expected level of replication capacity. Further, a required amount of replication capacity can be determined based on a traffic forecast and a maximum replication latency permitted by a service level agreement (SLA) of an event consumer. In addition, replication headroom can be computed, to identify a maximum level of traffic that can be sustained without violating an SLA and/or a date/time at which a violation may occur.

Abstract: A method of maintaining a data store is disclosed. The data store is distributed across a plurality of storage provider instances, including a master storage provider instance and a plurality of additional storage provider instances. A master data log is updated to reflect a change to the data store. The master data log is synchronized with a plurality of replicas of the master data log. The change is propagated across the plurality of additional storage provider instances based on an analysis of the replicas.

Abstract: An online advertising system integrates third party agents to permit the third party agents to participate in auctions to bid on a per opportunity basis. An advertising exchange module receives requests for opportunities to serve online advertisements to users. In response, an advertising exchange module applies one or more business rules to determine third party agents that qualify to serve the online advertisement. A bid gateway module generates and transmits requests for bids to the third party agents. The bid gateway module then receives bids from the third party agents in response to the requests for bids. The advertising exchange module then selects an advertisement based on the bid. The online advertisement exchange system provides a unified marketplace to permit integrator networks to bid on both ads pursuant to guaranteed contracts and ads not subject to guaranteed contracts (e.g., non-guaranteed ads).

Abstract: A method is disclosed for limiting latency in filling a display opportunity in an ad exchange including: constructing an exchange graph comprising nodes representing a plurality of publishers and advertisers, the exchange graph also including a plurality of directed edges that represent bilateral business agreements connecting the nodes; receiving an opportunity for displaying an ad to a user, wherein the opportunity is associated with a publisher node; receiving ads from the advertisers from which to choose to fill the opportunity; determining whether a threshold total number of ads (T) is surpassed by the received ads; and randomly downsampling the number of ads from each of at least some of the advertisers when the threshold total number of ads (T) is surpassed by the received ads to reduce the total number of ads to a target number of ads (S) that reduces overall latency in determining which of sampled ads will fill the opportunity.

Abstract: An online advertising system integrates third party agents to permit the third party agents to participate in auctions to bid on a per opportunity basis. An advertising exchange module receives requests for opportunities to serve online advertisements to users. In response, an advertising exchange module applies one or more business rules to determine third party agents that qualify to serve the online advertisement. A bid gateway module generates and transmits requests for bids to the third party agents. The bid gateway module then receives bids from the third party agents in response to the requests for bids. The advertising exchange module then selects an advertisement based on the bid. The online advertisement exchange system provides a unified marketplace to permit integrator networks to bid on both ads pursuant to guaranteed contracts and ads not subject to guaranteed contracts (e.g., non-guaranteed ads).