Abstract: Parallel processing of data may include a set of map processes and a set of reduce processes. Each map process may include at least one map thread. Map threads may access distinct input data blocks assigned to the map process, and may apply an application specific map operation to the input data blocks to produce key-value pairs. Each map process may include a multiblock combiner configured to apply a combining operation to values associated with common keys in the key-value pairs to produce combined values, and to output intermediate data including pairs of keys and combined values. Each reduce process may be configured to access the intermediate data output by the multiblock combiners. For each key, an application specific reduce operation may be applied to the combined values associated with the key to produce output data.

Abstract: Implementations of the disclosed subject matter provide methods and systems for using a multistage learner for efficiently boosting large datasets in a machine learning system. A method may include obtaining a first plurality of examples for a machine learning system and selecting a first point in time. Next, a second point in time occurring subsequent to the first point in time may be selected. The machine learning system may be trained using m of the first plurality of examples. Each of the m examples may include a feature initially occurring after the second point in time. In addition, the machine learning system may be trained using n of the first plurality of examples, and each of the n examples may include a feature initially occurring after the first point in time.

Abstract: Systems and techniques are disclosed for training a machine learning model based on one or more regularization penalties associated with one or more features. A template having a lower regularization penalty may be given preference over a template having a higher regularization penalty. A regularization penalty may be determined based on domain knowledge. A restrictive regularization penalty may be assigned to a template based on determining that a template occurrence is below a stability threshold and may be modified if the template occurrence meets or exceeds the stability threshold.

Abstract: Systems and techniques are disclosed for generating weighted machine learned models using multi-shard combiners. A learner in a machine learning system may receive labeled positive and negative examples and workers within the learner may be configured to receive either positive or negative examples. A positive and negative statistic may be calculated for a given feature and may either be applied separately in a model or may be combined to generate an overall statistic.

Abstract: Systems and techniques are disclosed for generating entries for a searchable index based on rules generated by one or more machine-learned models. The index entries can include one or more tokens correlated with an outcome and an outcome probability. A subset of tokens can be identified based on the characteristics of an event. The index may be searched for outcomes and their respective probabilities that correspond to tokens that are similar to or match the subset of tokens based on the event.

Abstract: Implementations of the disclosed subject matter provide methods and systems for using a multistage learner for efficiently boosting large datasets in a machine learning system. A method may include obtaining a first plurality of examples for a machine learning system and selecting a first point in time. Next, a second point in time occurring subsequent to the first point in time may be selected. The machine learning system may be trained using m of the first plurality of examples. Each of the m examples may include a feature initially occurring after the second point in time. In addition, the machine learning system may be trained using n of the first plurality of examples, and each of the n examples may include a feature initially occurring after the first point in time.

Abstract: Systems and techniques are disclosed for training a machine learning model based on one or more regularization penalties associated with one or more features. A template having a lower regularization penalty may be given preference over a template having a higher regularization penalty. A regularization penalty may be determined based on domain knowledge. A restrictive regularization penalty may be assigned to a template based on determining that a template occurrence is below a stability threshold and may be modified if the template occurrence meets or exceeds the stability threshold.

Abstract: A technique for reliably multicasting a message within a router network is provided. At least one special router in the network has associated persistent storage for logging a message being routed to one or more clients. When a message is received at this logger node, the logger places the message in persistent storage and sends a logging acknowledgment back to those routers to which the message was originally routed, as well as back towards the source of the message. The logger acknowledgment includes the message id, the logger id, and a logging number. When the logger acknowledgment is received by a router, it looks up the routing information from the original message, and sends the acknowledgment to those neighboring routers to which the original message was sent, excluding the link from which the logger acknowledgment was received. If the original message corresponding to the logger acknowledgment was buffered at this router waiting to be delivered, the message is then delivered to its client nodes.

Abstract: Program product for an application programming interface that unifies a plurality of mechanisms into a single framework. The interface includes a mechanism for communicating between members of a process group of related processes, and a mechanism for synchronizing the related processes of the process group. Additionally, the application programming interface may include mechanisms for managing membership of the process group or a processor group of processors, and/or a mechanism for controlling a group state value for the process group.

Abstract: A scalable merge technique for information retrieval across a distributed network is provided, as well as a fault tolerant technique for communicating information across the distributed network. The merge technique includes multicasting a query request across a spanning tree of the distributed network from a root node to multiple clients of the tree, and receiving back at the root node, a merged response produced from responses from at least some of the clients of the spanning tree. The fault tolerant technique for communicating information across the distributed network includes assigning a unique identifier to each query request, automatically reconfiguring the network upon detection of a failure of a node of the network, and retransmitting the query request and/or any responses thereto within the reconfigured network.

Abstract: The number of instances of an application running in a computing environment are controlled by monitoring the current load on the application, and altering the current number of instances of the application based on results of the monitoring. The load may be monitored by monitoring the current number of instances of the application running in the computing environment. Where a maximum, minimum and/or initial number on startup, of instances of the application are specified, the altering is done based on comparing the current number to one or more of the specified numbers of instances.

Abstract: A technique for reliably multicasting a message within a router network in a sequenced order with other messages is provided. At least one special router in the network has associated persistent storage for sequencing a message being routed to one or more clients. When a message is received at this sequencer node, the sequencer places a sequence number for the message in persistent storage and sends a sequencing acknowledgment back to those routers to which the message was originally routed, as well as back towards the source of the message. The sequencer acknowledgment includes the message id, and the sequencing number. When the sequencer acknowledgment is received by a router, it looks up the routing information from the original message, and sends the acknowledgment to those neighboring routers to which the original message was sent, excluding the link from which the sequencer acknowledgment was received.

Abstract: Resources are checkpointed in order to save the state of the resources. The resources can then be brought back to the same running state, during a restart procedure, by making use of the saved state. The determination of when to take a checkpoint or when to restart a resource is made by an entity, such as a cluster manager, external to the entity initiating or taking the checkpoint or performing the restart. The decision to checkpoint/restart a resource is provided by the cluster manager to a resource manager associated with the resource. This communication is facilitated by interfaces to the cluster manager provided by the resource manager.

Abstract: Within a computer cluster usage reference counts are maintained for replicated databases within a computer cluster using cluster membership and cluster voting services. Such a method includes the maintaining of a local reference count for all open distributed data resources within a given node, tracking by a group services client of those nodes that have the open distributed data resources, and using cluster membership services to update the local reference counts for node failures.

Abstract: A generalized search data structure is used to match consumers to events in event computing systems. The search data structure includes one or more paths from a root of the structure to one or more leaves of the structure. Each path has at least one level and each level corresponds to a filter attribute. The value of at least one filter attribute in at least one path is a don't care value indicating traversal of that path is guaranteed to proceed. In addition to following the path with the don't care value, one or more additional paths may also be followed. Thus, traversal of the search data structure may yield zero or more results, indicating that zero or more consumers match the specified event. Various optimizations of the search data structure are possible.

Abstract: Program product for an application programming interface that unifies a plurality of mechanisms into a single framework. The interface includes a mechanism for communicating between members of a process group of related processes, and a mechanism for synchronizing the related processes of the process group. Additionally, the application programming interface may include mechanisms for managing membership of the process group or a processor group of processors, and/or a mechanism for controlling a group state value for the process group.

Abstract: A method and program product for managing membership of process groups of a distributed computing environment. Requests to alter membership of a first process group are received and presented to the first process group as a batched request. However, requests to change the membership of a second process group are received and presented to the second process group serially. Thus, the manner in which requests are presented can be different for various process groups. Additionally, different types of requests can be presented to the same process group in a differing manner. For instance, requests to join a process group can be presented as a batched request and requests to leave the process group can be presented serially, or vice versa.

Abstract: A router within a network uses the data of a message received at the router to determine on which links out of the router the message needs to be forwarded. The router does not need any destination information from the message, and thus, the message need not include any destination information. Instead, the router uses an annotated search data structure to determine which links correspond to consumers interested in receiving the message. The message is sent only over those links.

Abstract: A barrier synchronization system in a distributed computing environment. A group of members, including a plurality of related processes executing on one or more processors of the distributed computing environment, is provided a protocol having a dynamic number of synchronization phases. The protocol is voted on by one or more members of the group to determine how the protocol should proceed for the group of members. The protocol may complete successfully, continue to another synchronization phase or be rejected. The group of members determines how many synchronization phases the protocol is to include.

Abstract: An application programming interface that unifies a plurality of mechanisms into a single framework. The interface includes a mechanism for communicating between members of a process group of related processes, and a mechanism for synchronizing the related processes of the process group. Additionally, the application programming interface may include mechanisms for managing membership of the process group or a processor group of processors, and/or a mechanism for controlling a group state value for the process group.