Abstract: Graph partitioning for massive scale graphs is described, such as for graphs having vertices representing people and edges representing connections between people in a social networking system; or for graphs where the vertices represent other items and the edges represent relationships between the items. In various embodiments a graph data allocator receives a graph vertex and its edges and allocates the vertex to one of a plurality of clusters each associated with one or more computing devices. In various embodiments the allocation is made by optimizing an objective function which takes into account both a cost of edges between clusters and a cost related to sizes of the clusters. In some examples the cost related to sizes of the clusters comprises a convex function applied to each of the cluster sizes. In examples, computations on the graph data are carried out with reduced runtimes and communications cost.

Abstract: Apparatuses and methods for determining if a computer program is malware and to which malware class it belongs to. In the method the behaviour of a computer program is traced by observing the activity of the program. Behaviour sequences comprising API-calls or similar activity of a computer program are then provided into a classifier for classifying the computer program. From the outcome of the classifier a classification result and the portions relevant to decision can be provided to a person for further confirmation.

Abstract: Embodiments for efficient queue management for cluster scheduling and managing task queues for tasks which are to be executed in a distributed computing environment. Both centralized and distributed scheduling is provided. Task queues may be bound by length-based bounding or delay-based bounding. Tasks may be prioritized and task queues may be dynamically reordered based on task priorities. Job completion times and cluster resource utilization may both be improved.

Abstract: Graph partitioning for massive scale graphs is described, such as for graphs having vertices representing people and edges representing connections between people in a social networking system; or for graphs where the vertices represent other items and the edges represent relationships between the items. In various embodiments a graph data allocator receives a graph vertex and its edges and allocates the vertex to one of a plurality of clusters each associated with one or more computing devices. In various embodiments the allocation is made by optimizing an objective function which takes into account both a cost of edges between clusters and a cost related to sizes of the clusters. In some examples the cost related to sizes of the clusters comprises a convex function applied to each of the cluster sizes. In examples, computations on the graph data are carried out with reduced runtimes and communications cost.

Abstract: Graph partitioning for massive scale graphs is described, such as for graphs having vertices representing people and edges representing connections between people in a social networking system; or for graphs where the vertices represent other items and the edges represent relationships between the items. In various embodiments a graph data allocator receives a graph vertex and its edges and allocates the vertex to one of a plurality of clusters each associated with one or more computing devices. In various embodiments the allocation is made by optimizing an objective function which takes into account both a cost of edges between clusters and a cost related to sizes of the clusters. In some examples the cost related to sizes of the clusters comprises a convex function applied to each of the cluster sizes. In examples, computations on the graph data are carried out with reduced runtimes and communications cost.

Abstract: Embodiments for efficient queue management for cluster scheduling and managing task queues for tasks which are to be executed in a distributed computing environment. Both centralized and distributed scheduling is provided. Task queues may be bound by length-based bounding or delay-based bounding. Tasks may be prioritized and task queues may be dynamically reordered based on task priorities. Job completion times and cluster resource utilization may both be improved.

Abstract: Count tracking in distributed environments is described, for example, as in data centers where many sites receive data and a coordinator node estimates a sum of the data received across the sites. Count tracking may be used in database applications, search engines, social networking applications and others. In various embodiments sites and a coordinator node work together to implement a process for summing data received at sites, where the sum takes into account both increments and decrements. In examples, a site decides whether to notify the coordinator node of a new data item according to a sampling probability that is related to an estimate of the current global sum of the data input across sites. In some examples a multi-mode algorithm is implemented which increases or decreases communication between the sites and the coordinator node according to behavior of the estimated global sum such that communications costs are optimized.

Abstract: A computation node of a neural network training system is described. The node has a memory storing a plurality of gradients of a loss function of the neural network and an encoder. The encoder encodes the plurality of gradients by setting individual ones of the gradients either to zero or to a quantization level according to a probability related to at least the magnitude of the individual gradient. The node has a processor which sends the encoded plurality of gradients to one or more other computation nodes of the neural network training system over a communications network.

Abstract: Streaming query resource control is described, for example, to allocate streaming queries to servers in a data center providing a streaming query platform. In various embodiments streaming queries are allocated to servers in a manner seeking to balance load between the servers and also to reduce network traffic costs between data stream sources and the servers. In various examples, query types are taken into account, where a query type is the identity of one or more data stream sources used by the query, and optionally also traffic rates of the data stream sources. In some examples, processes for allocating incoming queries in an online fashion are described and in some examples, processes for allocating queries in an offline fashion are described. In examples, a network traffic cost metric is used which takes into account an incremental network traffic cost of adding a given query at a server.

Abstract: Streaming query resource control is described, for example, to allocate streaming queries to servers in a data center providing a streaming query platform. In various embodiments streaming queries are allocated to servers in a manner seeking to balance load between the servers and also to reduce network traffic costs between data stream sources and the servers. In various examples, query types are taken into account, where a query type is the identity of one or more data stream sources used by the query, and optionally also traffic rates of the data stream sources. In some examples, processes for allocating incoming queries in an online fashion are described and in some examples, processes for allocating queries in an offline fashion are described. In examples, a network traffic cost metric is used which takes into account an incremental network traffic cost of adding a given query at a server.

Abstract: A decentralized relaying algorithm for mobile devices is described. In an embodiment, a mobile device acts as a relay within a network of mobile devices and on contact with a source device downloads messages from the source according to a locally stored relaying probability for each channel of information within the network. These messages are subsequently downloaded to another device which is the end user of the message. Where the relay does not download the message from the source as a result of the decision made based on the relaying probability, a virtual message is downloaded which comprises metadata only and not the payload of the message. The relay updates the stored relaying probabilities for each channel based on locally observable information which includes feedback received from mobile devices to which the relay has downloaded messages. The feedback identifies unique paths for the payload of messages through the network.

Abstract: Graph partitioning for massive scale graphs is described, such as for graphs having vertices representing people and edges representing connections between people in a social networking system; or for graphs where the vertices represent other items and the edges represent relationships between the items. In various embodiments a graph data allocator receives a graph vertex and its edges and allocates the vertex to one of a plurality of clusters each associated with one or more computing devices. In various embodiments the allocation is made by optimizing an objective function which takes into account both a cost of edges between clusters and a cost related to sizes of the clusters. In some examples the cost related to sizes of the clusters comprises a convex function applied to each of the cluster sizes. In examples, computations on the graph data are carried out with reduced runtimes and communications cost.

Abstract: Allocating tasks to machines in computing clusters is described. In an embodiment a set of tasks associated with a job are received at a scheduler. In an embodiment an index can be computed for each combination of tasks and processors and stored in a lookup table. In an example the index may be include an indication of the preference for the task to be processed on a particular processor, an indication of a waiting time for the task to be processed and an indication of how other tasks being processed in the computing cluster may be penalized by assigning a task to a particular processor. In an embodiment tasks are assigned to a processor by accessing the lookup table, selecting a task for processing using the index and scheduling the selected task for allocation to a processor.

Abstract: Count tracking in distributed environments is described, for example, as in data centers where many sites receive data and a coordinator node estimates a sum of the data received across the sites. Count tracking may be used in database applications, search engines, social networking applications and others. In various embodiments sites and a coordinator node work together to implement a process for summing data received at sites, where the sum takes into account both increments and decrements. In examples, a site decides whether to notify the coordinator node of a new data item according to a sampling probability that is related to an estimate of the current global sum of the data input across sites. In some examples a multi-mode algorithm is implemented which increases or decreases communication between the sites and the coordinator node according to behavior of the estimated global sum such that communications costs are optimized.

Abstract: Crowdsourcing is described, for example, where solutions to tasks such as designing a logo, writing a piece of code or answering a question are solicited by making open calls to large scale communities. In an example, a crowdsourcing node models a plurality of contests as all-pay auctions, each contest having a task and a reward. In examples, the crowdsourcing node is arranged to set rewards for the contests so that the optimal results are received for each contest owner, those owners having provided a budget and using a utility function for each of the contests.

Abstract: A decentralized relaying algorithm for mobile devices is described. In an embodiment, a mobile device acts as a relay within a network of mobile devices and on contact with a source device downloads messages from the source according to a locally stored relaying probability for each channel of information within the network. These messages are subsequently downloaded to another device which is the end user of the message. Where the relay does not download the message from the source as a result of the decision made based on the relaying probability, a virtual message is downloaded which comprises metadata only and not the payload of the message. The relay updates the stored relaying probabilities for each channel based on locally observable information which includes feedback received from mobile devices to which the relay has downloaded messages. The feedback identifies unique paths for the payload of messages through the network.

Abstract: Allocating tasks to machines in computing clusters is described. In an embodiment a set of tasks associated with a job are received at a scheduler. In an embodiment an index can be computed for each combination of tasks and processors and stored in a lookup table. In an example the index may be include an indication of the preference for the task to be processed on a particular processor, an indication of a waiting time for the task to be processed and an indication of how other tasks being processed in the computing cluster may be penalized by assigning a task to a particular processor. In an embodiment tasks are assigned to a processor by accessing the lookup table, selecting a task for processing using the index and scheduling the selected task for allocation to a processor.

Abstract: Crowdsourcing is described, for example, where solutions to tasks such as designing a logo, writing a piece of code or answering a question are solicited by making open calls to large scale communities. In an example, a crowdsourcing node models a plurality of contests as all-pay auctions, each contest having a task and a reward. In examples, the crowdsourcing node is arranged to set rewards for the contests so that the optimal results are received for each contest owner, those owners having provided a budget and using a utility function for each of the contests.

Abstract: Crowdsourcing is described, for example, where solutions to tasks such as designing a logo, writing a piece of code or answering a question are solicited by making open calls to large scale communities. In an example, a crowdsourcing node models a plurality of contests as all-pay auctions, each contest having a task and a reward. In examples, the crowdsourcing node is arranged to set rewards for the contests so that the optimal results are received for each contest owner, those owners having provided a budget and using a utility function for each of the contests. In examples, the crowdsourcing node is arranged to recommend contests to potential participants so that those participants can more easily decide which tasks to take on next.

Abstract: Sampling rules for information dissemination are described which may be applied in a system containing a number of nodes arranged into groups. A target address is selected using one of two methods: selection of an address from the entire address space of the system and selection of an address from a part of the address space which corresponds to set of groups of nodes. The set of groups of nodes is updated when information is successfully disseminated to a node at a target address selected using the first of the two methods. Rules to determine which of the two methods are used for any particular selection operation are also described.