Abstract: Described is automated learning of failure recovery policies based upon existing information regarding previous policies and actions. A learning mechanism automatically constructs a new policy for controlling a recovery process, based upon collected observable interactions of an existing policy with the process. In one aspect, the learning mechanism builds a partially observable Markov decision process (POMDP) model, and computes the new policy base upon the learned model. The new policy may perform automatic fault recovery, e.g., on a machine in a datacenter corresponding to the controlled process.

Abstract: Described is automated learning of failure recovery policies based upon existing information regarding previous policies and actions. A learning mechanism automatically constructs a new policy for controlling a recovery process, based upon collected observable interactions of an existing policy with the process. In one aspect, the learning mechanism builds a partially observable Markov decision process (POMDP) model, and computes the new policy base upon the learned model. The new policy may perform automatic fault recovery, e.g., on a machine in a datacenter corresponding to the controlled process.

Abstract: Functionality is described for probabilistically determining the location of an agent within an environment. The functionality performs this task using a topological representation of the environment provided by a directed graph. Nodes in the directed graph represent locations in the environment, while edges represent transition paths between the locations. The functionality also provides a mechanism by which the agent can navigate in the environment based on its probabilistic assessment of location. Such a mechanism can use a high-level control module and a low-level control module. The high-level control module determines an action for the agent to take by considering a plurality of votes associated with different locations in the directed graph. The low-level control module allows the agent to navigate along a selected edge when the high-level control module votes for a navigation action.

Abstract: Advertisement response probabilities are utilized to alter advertisement scores. A plurality of possible advertisements is accessed from, for example, an advertisement database or advertisement pipeline. A response probability for each advertisement is determined. A response probability may be a probability that a user will “click,” or otherwise select an advertisement. Advertisements may be associated with probabilistic prediction models that take advertisement recipient attribute values as inputs and provide a probability distribution as output. A score associated with each of the possible advertisements is altered based on the response probability for each of the advertisements. Statistical prediction is used to determine how scores are to be altered. Advertisements with response probabilities less than a mean probability may have associated scores decreased. Conversely, advertisements with response probabilities greater than a mean probability may have associated scores increased.

Abstract: Advertisement response probabilities are utilized to alter advertisement scores. A plurality of possible advertisements is accessed from, for example, an advertisement database or advertisement pipeline. A response probability for each advertisement is determined. A response probability may be a probability that a user will “click,” or otherwise select an advertisement. Advertisements may be associated with probabilistic prediction models that take advertisement recipient attribute values as inputs and provide a probability distribution as output. A score associated with each of the possible advertisements is altered based on the response probability for each of the advertisements. Statistical prediction is used to determine how scores are to be altered. Advertisements with response probabilities less than a mean probability may have associated scores decreased. Conversely, advertisements with response probabilities greater than a mean probability may have associated scores increased.