Abstract: The claimed subject matter includes techniques for providing an application testing service with a user interface that enables a user to evaluate performance data for computer implemented decision policies. An example method includes displaying a first reward statistic comprising an actual performance result for a policy implemented by an application. The method also includes obtaining experimental data corresponding to previously implemented policies, computing a second reward statistic for a hypothetical policy using a reward function applied to the experimental data. The method also includes displaying the second reward statistic together with the first reward statistic to enable a user to compare the first reward statistic and the second first reward statistic.

Abstract: The claimed subject matter includes techniques for providing an application testing service. An example method includes receiving context information from a client system, the context information comprising parameters that describe details of a user interaction with an application under test (AUT). The method also includes receiving a set of potential actions from the client system. The method also includes identifying a selected action from the set of potential actions and sending the selected action to the client system, wherein the AUT is customized based on the selected action. The method also includes receiving reward data from the client system based on a user's interaction with the AUT. The method also includes storing the context information, the selected action, and the reward data to a log of application test data.

Abstract: The claimed subject matter includes techniques for providing an application testing service with a user interface that enables a user to evaluate performance data for computer implemented decision policies. An example method includes displaying a first reward statistic comprising an actual performance result for a policy implemented by an application. The method also includes obtaining experimental data corresponding to previously implemented policies, computing a second reward statistic for a hypothetical policy using a reward function applied to the experimental data. The method also includes displaying the second reward statistic together with the first reward statistic to enable a user to compare the first reward statistic and the second first reward statistic.

Abstract: Entities that answer questions are organized into entity groups. An interface determines an answer to a question from one of the entity groups with a confidence that is greater than a second confidence threshold. To determine the answer, the interface performs an exploration phase where the interface asks the question to each of the entity groups. Entity groups that provide answers with a confidence that is greater than a first confidence threshold are selected for a subset. The interface enters an exploitation phase where the question is asked of the entity groups in the subset. If an answer is determined with a confidence that is greater than the second confidence threshold, then the answer is provided. If no answer is determined before a duration of time expires, then the interface enters a rollback phase where all of the entity groups are considered.

Abstract: A graph of a social network is received. The graph may include a node for each user account and an edge between nodes that represent social networking relationships such as messages between the user accounts or a friend relationship. The graph is transformed into a transformed graph where nodes have direct edges depending on a local test among its neighbors in the original graph. Small subsets of the transformed graph are categorized. The categories are used to identify subgraphs in the transformed graph. Each subgraph is grown by adding an edge from the transformed graph to the subgraph depending on local tests among nodes associated with the edge that have at least one edge that is already in the subgraph. The categorized subgraphs are used to provide targeted advertising, suggest new connections, identify different personalities and interests of users, or to provide other services.

Abstract: The claimed subject matter includes techniques for providing an application testing service. An example method includes receiving context information from a client system, the context information comprising parameters that describe details of a user interaction with an application under test (AUT). The method also includes receiving a set of potential actions from the client system. The method also includes identifying a selected action from the set of potential actions and sending the selected action to the client system, wherein the AUT is customized based on the selected action. The method also includes receiving reward data from the client system based on a user's interaction with the AUT. The method also includes storing the context information, the selected action, and the reward data to a log of application test data.

Abstract: A graph of a social network is received. The graph may include a node for each user account and an edge between nodes that represent social networking relationships such as messages between the user accounts or a friend relationship. The graph is transformed into a transformed graph where nodes have direct edges depending on a local test among its neighbors in the original graph. Small subsets of the transformed graph are categorized. The categories are used to identify subgraphs in the transformed graph. Each subgraph is grown by adding an edge from the transformed graph to the subgraph depending on local tests among nodes associated with the edge that have at least one edge that is already in the subgraph. The categorized subgraphs are used to provide targeted advertising, suggest new connections, identify different personalities and interests of users, or to provide other services.

Abstract: Entities that answer questions are organized into entity groups. An interface determines an answer to a question from one of the entity groups with a confidence that is greater than a second confidence threshold. To determine the answer, the interface performs an exploration phase where the interface asks the question to each of the entity groups. Entity groups that provide answers with a confidence that is greater than a first confidence threshold are selected for a subset. The interface enters an exploitation phase where the question is asked of the entity groups in the subset. If an answer is determined with a confidence that is greater than the second confidence threshold, then the answer is provided. If no answer is determined before a duration of time expires, then the interface enters a rollback phase where all of the entity groups are considered.

Abstract: Techniques and systems are disclosed for returning temporally-aware results from an Internet-based search query. To determine if a query is temporally-based one or more query features are collected and input into a trained classifier, yielding a temporal classification for the query. Further, if a query is classified as temporal, the query results are shifted by determining an alternate set of results for the query, and returning one or more alternate results to one or more users. Based on user interactions with the one or more alternate results, the classifier can be updated, for example, by changing the query to a non-temporal query if the user interactions identify it as such.

Abstract: An auction may include a general schema to create incentives for those agents or bidders to bid on items at auction truthfully. The schema may implement any allocation rule, including a monotone allocation rule. The schema may provide for accepting bids from bidders, transforming the bids in a manner appropriate to the item being auctioned, and applying the allocation rule on the transformed bid. The schema may further provide for providing a rebate or payment back to the bidders.

Abstract: A document selector selects and ranks documents that are relevant to a query. The document selector executes an instance of a multi-armed bandits algorithm to select a document for each slot of a results page according to one or more strategies. The documents are selected in an order defined by the results page and documents selected for previous slots are used to guide the selection of a document for a current slot. If a document in a slot is subsequently selected, the strategy used to select the document is rewarded with positive feedback. When the uncertainty in an estimate of the utility of a strategy is less than the variation between documents associated with the strategy, the strategy is subdivided into multiple strategies. The document selector is able to “zoom in” on effective strategies and provide more relevant search results.

Abstract: Techniques and systems are disclosed for returning temporally-aware results from an Internet-based search query. To determine if a query is temporally-based one or more query features are collected and input into a trained classifier, yielding a temporal classification for the query. Further, if a query is classified as temporal, the query results are shifted by determining an alternate set of results for the query, and returning one or more alternate results to one or more users. Based on user interactions with the one or more alternate results, the classifier can be updated, for example, by changing the query to a non-temporal query if the user interactions identify it as such.