Abstract: A data preprocessing system builds transformation scripts for preprocessing datasets for processing by a data analysis system. The data preprocessing system presents various representations of data of a dataset including visual representations, textual representations, or structural representations. The data preprocessing system receives selections of attributes or values based on these representations. The data preprocessing system generates recommendations of transformations based on the attributes or values selected. The data preprocessing system builds a transformation script based on the recommendations of the transformations. The transformation script can be used for preprocessing the dataset for analysis by a data analysis system.

Abstract: A data preprocessing system builds transformation scripts for preprocessing datasets for processing by a data analysis system. The data preprocessing system presents various representations of data of a dataset including visual representations, textual representations, or structural representations. The data preprocessing system receives selections of attributes or values based on these representations. The data preprocessing system generates recommendations of transformations based on the attributes or values selected. The data preprocessing system builds a transformation script based on the recommendations of the transformations. The transformation script can be used for preprocessing the dataset for analysis by a data analysis system.

Abstract: Software for an online social service receives a request for a newsfeed from a user. The newsfeed depends at least in part on a memory-cached list of social friends of the user and on a memory-cached list of events associated with each social friend. The software refreshes a memory-cached copy of the list of social friends from a corresponding list in a persistent data store, based upon a determination of a utility associated with the user. And the software refreshes a memory-cached copy of a list of events associated with a social friend from a corresponding list of events in a persistent data store, based upon a determination of a utility associated with the social friend. Then the software causes the newsfeed to be displayed to the user.

Abstract: Software for an online social service receives a request for a newsfeed from a user. The newsfeed depends at least in part on a memory-cached list of social friends of the user and on a memory-cached list of events associated with each social friend. The software refreshes a memory-cached copy of the list of social friends from a corresponding list in a persistent data store, based upon a determination of a utility associated with the user. And the software refreshes a memory-cached copy of a list of events associated with a social friend from a corresponding list of events in a persistent data store, based upon a determination of a utility associated with the social friend. Then the software causes the newsfeed to be displayed to the user.

Abstract: Methods and system for providing social feeds from a plurality of third party sites to a user at a host site includes retrieving one or more access logs capturing online behavior of the user. The access logs are analyzed to determine the user's interactive behavioral pattern related to social feeds from each of the plurality of third party sites. A refresh schedule for the user is computed to refresh cache entries of social feeds at the host site based on the analysis of the user's online behavior at the social feeds. Cache entries of social feeds for the user are refreshed at the host site from the one or more of the plurality of third party sites at an allotted time specified by the refresh schedule.

Abstract: In a large-scale transaction such as the bulk loading of new records into an ordered, distributed database, a transaction limit such as an insert limit may be chosen, partitions on overfull storage servers may be designated to be moved to underfull storage servers, and the move assignments may be based, at least in part on the degree to which a storage server is underfull and the move and insertion costs of the partitions to be moved.

Abstract: Method, system, and programs for balancing work load in a distributed system. A plurality of multi-dimensional load metrics are received from a plurality of resource units in the distributed system. Based on the received plurality of multi-dimensional load metrics and a global statistical load model, a load deviance for each resource unit is computed. The plurality of resource units in the distributed system are then ranked based on the load deviance of each resource unit. At least one load balancing action is further determined based on the ranked resource units and at least one load balancing policy.

Abstract: Methods and system for providing social feeds from a plurality of third party sites to a user at a host site includes retrieving one or more access logs capturing online behavior of the user. The access logs are analyzed to determine the user's interactive behavioral pattern related to social feeds from each of the plurality of third party sites. A refresh schedule for the user is computed to refresh cache entries of social feeds at the host site based on the analysis of the user's online behavior at the social feeds. Cache entries of social feeds for the user are refreshed at the host site from the one or more of the plurality of third party sites at an allotted time specified by the refresh schedule.

Abstract: Method, system, and programs for balancing work load in a distributed system. A plurality of multi-dimensional load metrics are received from a plurality of resource units in the distributed system. Based on the received plurality of multi-dimensional load metrics and a global statistical load model, a load deviance for each resource unit is computed. The plurality of resource units in the distributed system are then ranked based on the load deviance of each resource unit. At least one load balancing action is further determined based on the ranked resource units and at least one load balancing policy.

Abstract: Various methods and apparatuses are provided which may be implemented using one or more computing devices within a networked computing environment to support a computing grid having selective storage of shared data files within certain distributed the systems provided by dusters of computing devices. The selective storage may represent limited duplicative storage of a shared file.

Abstract: An improved system and method for loading records into a partitioned database table is provided. A translation of records may be generated from a set of source partitions to a set of target partitions by generating a bipartite graph, determining a maximal matching using dynamic programming for a chain of nodes remaining in the bipartite graph after removing singleton edges, and generating a maximal matching after adding back the singleton edges for translation of records from the set of source partitions to the set of target partitions. The partition translation may be executed by traversing from top to bottom the set of source partitions and the set of target partitions in record key order to generate an optimal sequence of operations to transfer the records from the set of source partitions to the set of target partitions.

Abstract: A method of allocating servers for range requests includes receiving a range request for items in a database that is distributed across storage devices that are accessible through corresponding servers in a network that includes the storage devices and the servers; and initializing a server-allocation value for the range request, where the server-allocation value specifies a number of servers to allocate for executing the range request. The method further includes executing the range request by allocating the servers and using the allocated servers to provide values from the range request to a client that accesses the network; and updating the server-allocation value while executing the range request to improve a consumption rate for the client by comparing changes in the consumption rate with changes in the number of allocated servers.

Abstract: A system is described for providing scalable in-memory caching for a distributed database. The system may include a cache, an interface, a non-volatile memory and a processor. The cache may store a cached copy of data items stored in the non-volatile memory. The interface may communicate with devices and a replication server. The non-volatile memory may store the data items. The processor may receive an update to a data item from a device to be applied to the non-volatile memory. The processor may apply the update to the cache. The processor may generate an acknowledgement indicating that the update was applied to the non-volatile memory and may communicate the acknowledgment to the device. The processor may then communicate the update to a replication server. The processor may apply the update to the non-volatile memory upon receiving an indication that the update was stored by the replication server.

Abstract: A method comprises receiving a range query from a requestor. The range query requests a range of sequential items in a database that is distributed among a plurality of storage devices or partitions. The range query is divided into R sub-range queries, where R is an integer. Each sub-range query corresponds to a respective portion of the range of sequential items stored in a respective storage device or partition. The sub-range queries are issued to respective ones of up to K storage servers, where K is an integer less than or equal to R. Each of the K storage servers is configured with read access to the respective storage device or partition storing the respective portion of the range of sequential items in the respective sub-range query issued to that storage server.

Abstract: An improved system and method for loading records into a partitioned database table is provided. A translation of records may be generated from a set of source partitions to a set of target partitions by generating a bipartite graph, determining a maximal matching using dynamic programming for a chain of nodes remaining in the bipartite graph after removing singleton edges, and generating a maximal matching after adding back the singleton edges for translation of records from the set of source partitions to the set of target partitions. The partition translation may be executed by traversing from top to bottom the set of source partitions and the set of target partitions in record key order to generate an optimal sequence of operations to transfer the records from the set of source partitions to the set of target partitions.

Abstract: An improved system and method for applying once a transaction delivered in a message published asynchronously in a distributed database is provided. In various embodiments, apply once messaging may be achieved for asynchronous publication by having a persistent log stored on a messaging server. A messaging server may receive an update message for a transaction to be published asynchronously in a distributed database, may generate a sequence number for the transaction in a message, and may log the update message with the sequence number in a log file persistently stored on the messaging server. The messaging server may then send an acknowledgement that the update message is published and may asynchronously publish the update message with the sequence number to subscribers. The publication may only succeed if there may not be any message tagged with a sequence number that has been previously published by the messaging server.

Abstract: In a distributed system that includes multiple machines, a scheduler attempts to schedule a task on a machine that is not currently overloaded with work. If a task is scheduled on a machine that does not yet have copies of the portions of the data set on which the task needs to operate, then that machine obtains copies of those portions from other machines that already have them. Whenever a “source” machine ships a copy of a portion to another “destination” machine in the distributed system, the destination machine persistently stores that copy on the destination machine's persistent storage mechanism. The copy also remains on the source machine. Thus, portions of the data set are automatically replicated whenever those portions are shipped between machines of the distributed system. Each machine in the distributed system has access to “global” information that indicates which machines have which portions of the data set.

Abstract: In a large-scale transaction such as the bulk loading of new records into an ordered, distributed database, a transaction limit such as an insert limit may be chosen, partitions on overfull storage servers may be designated to be moved to underfull storage servers, and the move assignments may be based, at least in part on the degree to which a storage server is underfull and the move and insertion costs of the partitions to be moved.

Abstract: The subject matter disclosed herein relates to bulk loading of data into a database comprising a plurality of database partitions. In one particular example, the database partitioning may be revised before addition of the new data to the partitions.