Patents by Inventor Thomas J. Duerig
Thomas J. Duerig has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).
-
Publication number: 20240078258Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for jointly training an image embedding model and a text embedding model. In one aspect, a method comprises: processing data from a historical query log of a search system to generate a candidate set of training examples, wherein each training example comprises: (i) a search query comprising a sequence of one or more words, (ii) an image, and (iii) selection data characterizing how often users selected the image in response to the image being identified by a search result for the search query; selecting a plurality of training examples from the candidate set of training examples; and using the training data to jointly train the image embedding model and the text embedding model.Type: ApplicationFiled: November 9, 2023Publication date: March 7, 2024Inventors: Zhen Li, Yi-ting Chen, Ning Ye, Yaxi Gao, Zijian Guo, Aleksei Timofeev, Futang Peng, Thomas J. Duerig
-
Patent number: 11790264Abstract: The present disclosure is directed to methods and systems for knowledge distillation.Type: GrantFiled: June 19, 2019Date of Patent: October 17, 2023Assignee: GOOGLE LLCInventors: Thomas J. Duerig, Hongsheng Wang, Scott Alexander Rudkin
-
Publication number: 20230205813Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.Type: ApplicationFiled: February 20, 2023Publication date: June 29, 2023Inventors: Zhen Li, Yi-Ting Chen, Yaxi Gao, Da-Cheng Juan, Aleksei Timofeev, Chun-Ta Lu, Futang Peng, Sujith Ravi, Andrew Tomkins, Thomas J. Duerig
-
Patent number: 11586927Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.Type: GrantFiled: February 1, 2019Date of Patent: February 21, 2023Assignee: GOOGLE LLCInventors: Zhen Li, Yi-ting Chen, Yaxi Gao, Da-Cheng Juan, Aleksei Timofeev, Chun-Ta Lu, Futang Peng, Sujith Ravi, Andrew Tomkins, Thomas J. Duerig
-
Publication number: 20200401929Abstract: The present disclosure is directed to methods and systems for knowledge distillation.Type: ApplicationFiled: June 19, 2019Publication date: December 24, 2020Inventors: Thomas J. Duerig, Hongsheng Wang, Scott Alexander Rudkin
-
Publication number: 20200250538Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for jointly training an image embedding model and a text embedding model. In one aspect, a method comprises: processing data from a historical query log of a search system to generate a candidate set of training examples, wherein each training example comprises: (i) a search query comprising a sequence of one or more words, (ii) an image, and (iii) selection data characterizing how often users selected the image in response to the image being identified by a search result for the search query; selecting a plurality of training examples from the candidate set of training examples; and using the training data to jointly train the image embedding model and the text embedding model.Type: ApplicationFiled: February 1, 2019Publication date: August 6, 2020Inventors: Zhen Li, Yi-ting Chen, Ning Ye, Yaxi Gao, Zijian Guo, Aleksei Timofeev, Futang Peng, Thomas J. Duerig
-
Publication number: 20200250537Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image embedding model. In one aspect, a method comprises: obtaining training data comprising a plurality of training examples, wherein each training example comprises: an image pair comprising a first image and a second image; and selection data indicating one or more of: (i) a co-click rate of the image pair, and (ii) a similar-image click rate of the image pair; and using the training data to train an image embedding model having a plurality of image embedding model parameters.Type: ApplicationFiled: February 1, 2019Publication date: August 6, 2020Inventors: Zhen Li, Yi-ting Chen, Yaxi Gao, Da-Cheng Juan, Aleksei Timofeev, Chun-Ta Lu, Futang Peng, Sujith Ravi, Andrew Tomkins, Thomas J. Duerig
-
Patent number: 10311096Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for analyzing image search result relevance. In one aspect, a method includes receiving result data specifying a search query and responsive image search results that reference images that are responsive to the search query. A determination is made that the search query matches an indexed query. An image relevance model is identified for the indexed query. The image relevance model can output a relevance score adjustment factor for an image search result based on image feature values of the image that is referenced by the search result. A relevance score adjustment factor is determined for each image search result using the identified image relevance model. A relevance score for each image search result is adjusted using the image's image relevance score adjustment factor. The images are ranked based on the adjusted relevance scores.Type: GrantFiled: February 28, 2013Date of Patent: June 4, 2019Assignee: Google LLCInventors: Kunlong Gu, Sean Arietta, Charles J. Rosenberg, Thomas J. Duerig, Erik Murphy-Chutorian
-
Patent number: 10185725Abstract: Implementations include actions of receiving an initial data set including a plurality of images, each image being associated with a set of labels, wherein each label in the set of labels is assigned to a respective image of the plurality of images by an initial model, the initial model being specific to a ground-truth label; for each image in the plurality of images: providing a list of categories associated with a respective image based on a respective set of labels, and determining a primary category of the respective image based on the list of categories; determining a category of the ground-truth label; and providing a revised data set based on the initial data set by comparing the category to primary categories of respective images in the plurality of images, the initial model being trained based on the revised data set to provide a revised model.Type: GrantFiled: May 31, 2018Date of Patent: January 22, 2019Assignee: Google LLCInventors: David Cai, Zhen Hao Zhou, Neil G. Alldrin, Thomas J. Duerig
-
Patent number: 10013436Abstract: Implementations include actions of receiving an initial data set including a plurality of images, each image being associated with a set of labels, wherein each label in the set of labels is assigned to a respective image of the plurality of images by an initial model, the initial model being specific to a ground-truth label; for each image in the plurality of images: providing a list of categories associated with a respective image based on a respective set of labels, and determining a primary category of the respective image based on the list of categories; determining a category of the ground-truth label; and providing a revised data set based on the initial data set by comparing the category to primary categories of respective images in the plurality of images, the initial model being trained based on the revised data set to provide a revised model.Type: GrantFiled: June 17, 2015Date of Patent: July 3, 2018Assignee: Google LLCInventors: David Cai, Zhen Hao Zhou, Neil G. Alldrin, Thomas J. Duerig
-
Patent number: 9454600Abstract: Methods, systems and apparatus for refining image relevance models. In general, one aspect includes receiving a trained image relevance model that generates relevance measures of content feature values of images to a query, identifying a first threshold number of common content feature values for the set of training images, the common content feature values being identified as a set of content feature values that are each shared by at least a portion of the training images, identifying a subset of the set of training images having a quantity of the common content feature values greater than a second threshold number of content features values, and generating a re-trained image relevance model based on content feature values of the set of training images, wherein content feature values of the subset of training images are weighted higher than content feature values of the training images not in the subset.Type: GrantFiled: February 1, 2012Date of Patent: September 27, 2016Assignee: Google Inc.Inventors: Thomas J. Duerig, Jason E. Weston, Charles J. Rosenberg, Kunlong Gu, Samy Bengio
-
Patent number: 9177046Abstract: Methods, systems and apparatus for refining image relevance models. In general, one aspect of the subject matter described in this specification can be implemented in methods that include re-training an image relevance model by generating a first re-trained model based on content feature values of first images of a first portion of training images in a set of training images, receiving, from the first re-trained model, image relevance scores for second images of a second portion of the set of training images, removing, from the set of training images, some of the second images identified as outlier images for which the image relevance score received from the first re-trained model is below a threshold score, and generating a second re-trained model based on content feature values of the first images of the first portion and the second images of the second portion that remain following removal of the outlier images.Type: GrantFiled: November 17, 2014Date of Patent: November 3, 2015Assignee: Google Inc.Inventors: Arcot J. Preetham, Thomas J. Duerig, Charles J. Rosenberg, Yangli Hector Yee, Samy Bengio
-
Patent number: 9152700Abstract: A method includes receiving a search query comprising one or more query terms, receiving image relevance models that each generate relevance measures of content feature values of images to a query, each image relevance model being a predictive model that has been trained based on content feature values of a set of training images, and each of the queries being a unique set of one or more query terms received by a search system as a query input, identifying an image relevance model for a different query that has been identified as similar to the received search query, and calculating a fractional adjustment multiplier for search results responsive to the received search query, the fractional adjustment multiplier being based on a relevance measure generated by the identified image relevance model for the different query and based on a degree of similarity between the different query and the received search query.Type: GrantFiled: January 13, 2012Date of Patent: October 6, 2015Assignee: Google Inc.Inventors: Thomas J. Duerig, Charles J. Rosenberg, Kunlong Gu, Samy Bengio, Yun Zhou
-
Patent number: 9152652Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying images responsive to a search phrase are disclosed. In one aspect, a method includes identifying a set of responsive images for a search phrase that includes two or more terms. Interaction rankings are determined for images in the set of responsive images. Two or more sub-queries are created based on the search phrase. Sub-query model rankings are determined for images in the set of responsive images. A search phrase score is determined for the image relevance model. Based on the search phrase scores for the sub-queries, one of the sub-query models is selected as a model for the search phrase.Type: GrantFiled: March 14, 2013Date of Patent: October 6, 2015Assignee: Google Inc.Inventors: Kunlong Gu, Charles J. Rosenberg, Mingchen Gao, Thomas J. Duerig
-
Publication number: 20150169999Abstract: Methods, systems and apparatus for refining image relevance models. In general, one aspect includes receiving a trained image relevance model that generates relevance measures of content feature values of images to a query, identifying a first threshold number of common content feature values for the set of training images, the common content feature values being identified as a set of content feature values that are each shared by at least a portion of the training images, identifying a subset of the set of training images having a quantity of the common content feature values greater than a second threshold number of content features values, and generating a re-trained image relevance model based on content feature values of the set of training images, wherein content feature values of the subset of training images are weighted higher than content feature values of the training images not in the subset.Type: ApplicationFiled: February 1, 2012Publication date: June 18, 2015Applicant: GOOGLE INC.Inventors: Thomas J. Duerig, Jason E. Weston, Charles J. Rosenberg, Kunlong Gu, Samy Bengio
-
Publication number: 20150169738Abstract: A method includes receiving a search query comprising one or more query terms, receiving image relevance models that each generate relevance measures of content feature values of images to a query, each image relevance model being a predictive model that has been trained based on content feature values of a set of training images, and each of the queries being a unique set of one or more query terms received by a search system as a query input, identifying an image relevance model for a different query that has been identified as similar to the received search query, and calculating a fractional adjustment multiplier for search results responsive to the received search query, the fractional adjustment multiplier being based on a relevance measure generated by the identified image relevance model for the different query and based on a degree of similarity between the different query and the received search query.Type: ApplicationFiled: January 13, 2012Publication date: June 18, 2015Inventors: Thomas J. Duerig, Charles J. Rosenberg, Kunlong Gu, Samy Bengio, Yun Zhou
-
Publication number: 20150169631Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying images responsive to a search phrase are disclosed. In one aspect, a method includes identifying a set of responsive images for a search phrase that includes two or more terms. Interaction rankings are determined for images in the set of responsive images. Two or more sub-queries are created based on the search phrase. Sub-query model rankings are determined for images in the set of responsive images. A search phrase score is determined for the image relevance model. Based on the search phrase scores for the sub-queries, one of the sub-query models is selected as a model for the search phrase.Type: ApplicationFiled: March 14, 2013Publication date: June 18, 2015Inventors: Kunlong Gu, Charles J. Rosenberg, Mingchen Gao, Thomas J. Duerig
-
Publication number: 20150169754Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for analyzing image search result relevance. In one aspect, a method includes receiving result data specifying a search query and responsive image search results that reference images that are responsive to the search query. A determination is made that the search query matches an indexed query. An image relevance model is identified for the indexed query. The image relevance model can output a relevance score adjustment factor for an image search result based on image feature values of the image that is referenced by the search result. A relevance score adjustment factor is determined for each image search result using the identified image relevance model. A relevance score for each image search result is adjusted using the image's image relevance score adjustment factor. The images are ranked based on the adjusted relevance scores.Type: ApplicationFiled: February 28, 2013Publication date: June 18, 2015Inventors: Kunlong Gu, Sean Arietta, Charles J. Rosenberg, Thomas J. Duerig, Erik Murphy-Chutorian
-
Publication number: 20150169708Abstract: A computer implemented technique for presenting selected image search results is presented. The technique includes obtaining a first query at a first time and obtaining a first set of image search results responsive to the first query. The technique also includes providing the first set of image search results in response to the first query and obtaining input data reflecting a selection of at least one of the first set of image search results. The technique further includes obtaining a second query at a second time subsequent to the first time and obtaining a second set of image search results responsive to the second query. The technique further includes providing the second set of image search results together with the selected at least one of the first set of image search results.Type: ApplicationFiled: April 10, 2014Publication date: June 18, 2015Applicant: Google Inc.Inventors: Yang Song, Thomas J. Duerig
-
Publication number: 20150161482Abstract: Methods, systems and apparatus for refining image relevance models. In general, one aspect of the subject matter described in this specification can be implemented in methods that include re-training an image relevance model by generating a first re-trained model based on content feature values of first images of a first portion of training images in a set of training images, receiving, from the first re-trained model, image relevance scores for second images of a second portion of the set of training images, removing, from the set of training images, some of the second images identified as outlier images for which the image relevance score received from the first re-trained model is below a threshold score, and generating a second re-trained model based on content feature values of the first images of the first portion and the second images of the second portion that remain following removal of the outlier images.Type: ApplicationFiled: November 17, 2014Publication date: June 11, 2015Inventors: Arcot J. Preetham, Thomas J. Duerig, Charles J. Rosenberg, Yangli Hector Yee, Samy Bengio