Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for click or skip evaluation of optionalization rules. In one aspect, a method includes accessing query log data that identifies, for an initial search query, one or more revised search queries that were identified by designating a query term of the initial search query as optional according to one or more optionalization rules, and a particular search result selected by the user. A determination is made, using the query log data, that the particular search result corresponds to the one or more revised search queries that were identified by designating the query term of the initial search query as optional according to one or more optionalization rules. In response to this determination, a click count is incremented for the one or more optionalization rules that corresponds to the particular search result.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for removing substitution rules. According to one implementation, a method includes identifying a revised search query that was revised to include a substitute term of a query term; identifying search results that were generated using the revised search query, wherein each search result references a resource; determining, by one or more computers, that none of the resources referenced by a subset of the search results include the substitute term of the query term; and in response to determining that none of the resources referenced by the subset of search results include the substitute term of the query term, incrementing a no-match score for the substitute term.