Patents by Inventor Jonathon Shlens

Jonathon Shlens has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10521729
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining neural network architectures. One of the methods includes generating, using a controller neural network having controller parameters and in accordance with current values of the controller parameters, a batch of output sequences. The method includes, for each output sequence in the batch: generating an instance of a child convolutional neural network (CNN) that includes multiple instances of a first convolutional cell having an architecture defined by the output sequence; training the instance of the child CNN to perform an image processing task; and evaluating a performance of the trained instance of the child CNN on the task to determine a performance metric for the trained instance of the child CNN; and using the performance metrics for the trained instances of the child CNN to adjust current values of the controller parameters of the controller neural network.
    Type: Grant
    Filed: July 19, 2018
    Date of Patent: December 31, 2019
    Assignee: Google LLC
    Inventors: Vijay Vasudevan, Barret Zoph, Jonathon Shlens, Quoc V. Le
  • Publication number: 20190370648
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining neural network architectures. One of the methods includes obtaining training data for a dense image prediction task; and determining an architecture for a neural network configured to perform the dense image prediction task, comprising: searching a space of candidate architectures to identify one or more best performing architectures using the training data, wherein each candidate architecture in the space of candidate architectures comprises (i) the same first neural network backbone that is configured to receive an input image and to process the input image to generate a plurality of feature maps and (ii) a different dense prediction cell configured to process the plurality of feature maps and to generate an output for the dense image prediction task; and determining the architecture for the neural network based on the best performing candidate architectures.
    Type: Application
    Filed: May 29, 2019
    Publication date: December 5, 2019
    Inventors: Barret Zoph, Jonathon Shlens, Yukun Zhu, Maxwell Donald Emmet Collins, Liang-Chieh Chen, Gerhard Florian Schroff, Hartwig Adam, Georgios Papandreou
  • Patent number: 10445623
    Abstract: Systems and techniques are disclosed for labeling objects within an image. The objects may be labeled by selecting an option from a plurality of options such that each option is a potential label for the object. An option may have an option score associated with. Additionally, a relation score may be calculated for a first option and a second option corresponding to a second object in an image. The relation score may be based on a frequency, probability, or observance corresponding to the co-occurrence of text associated with the first option and the second option in a text corpus such as the World Wide Web. An option may be selected as a label for an object based on a global score calculated based at least on an option score and relation score associated with the option.
    Type: Grant
    Filed: April 14, 2017
    Date of Patent: October 15, 2019
    Assignee: Google LLC
    Inventors: Samy Bengio, Jeffrey Adgate Dean, Quoc V. Le, Jonathon Shlens, Yoram Singer
  • Publication number: 20190236814
    Abstract: A method for applying a style to an input image to generate a stylized image. The method includes maintaining data specifying respective parameter values for each image style in a set of image styles, receiving an input including an input image and data identifying an input style to be applied to the input image to generate a stylized image that is in the input style, determining, from the maintained data, parameter values for the input style, and generating the stylized image by processing the input image using a style transfer neural network that is configured to process the input image to generate the stylized image.
    Type: Application
    Filed: April 10, 2019
    Publication date: August 1, 2019
    Inventors: Jonathon Shlens, Vincent Dumoulin, Manjunath Kudlur Venkatakrishna
  • Publication number: 20190026639
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining neural network architectures. One of the methods includes generating, using a controller neural network having controller parameters and in accordance with current values of the controller parameters, a batch of output sequences. The method includes, for each output sequence in the batch: generating an instance of a child convolutional neural network (CNN) that includes multiple instances of a first convolutional cell having an architecture defined by the output sequence; training the instance of the child CNN to perform an image processing task; and evaluating a performance of the trained instance of the child CNN on the task to determine a performance metric for the trained instance of the child CNN; and using the performance metrics for the trained instances of the child CNN to adjust current values of the controller parameters of the controller neural network.
    Type: Application
    Filed: July 19, 2018
    Publication date: January 24, 2019
    Inventors: Vijay Vasudevan, Barret Zoph, Jonathon Shlens, Quoc V. Le
  • Patent number: 10127475
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classifying images.
    Type: Grant
    Filed: September 22, 2016
    Date of Patent: November 13, 2018
    Assignee: Google LLC
    Inventors: Gregory S. Corrado, Jeffrey A. Dean, Samy Bengio, Andrea L. Frome, Jonathon Shlens
  • Patent number: 9852363
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating labeled images. One of the methods includes selecting a plurality of candidate videos from videos identified in a response to a search query derived from a label for an object category; selecting one or more initial frames from each of the candidate videos; detecting one or more initial images of objects in the object category in the initial frames; for each initial frame including an initial image of an object in the object category, tracking the object through surrounding frames to identify additional images of the object; and selecting one or more images from the one or more initial images and one or more additional images as database images of objects belonging to the object category.
    Type: Grant
    Filed: January 5, 2016
    Date of Patent: December 26, 2017
    Assignee: Google Inc.
    Inventors: Jonathon Shlens, Quoc V. Le, Gregory Sean Corrado, Marc'Aurelio Ranzato
  • Publication number: 20170220906
    Abstract: Systems and techniques are disclosed for labeling objects within an image. The objects may be labeled by selecting an option from a plurality of options such that each option is a potential label for the object. An option may have an option score associated with. Additionally, a relation score may be calculated for a first option and a second option corresponding to a second object in an image. The relation score may be based on a frequency, probability, or observance corresponding to the co-occurrence of text associated with the first option and the second option in a text corpus such as the World Wide Web. An option may be selected as a label for an object based on a global score calculated based at least on an option score and relation score associated with the option.
    Type: Application
    Filed: April 14, 2017
    Publication date: August 3, 2017
    Inventors: Samy Bengio, Jeffrey Adgate Dean, Quoc V. Le, Jonathon Shlens, Yoram Singer
  • Publication number: 20170140272
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a larger neural network from a smaller neural network. In one aspect, a method includes obtaining data specifying an original neural network; generating a larger neural network from the original neural network, wherein the larger neural network has a larger neural network structure including the plurality of original neural network units and a plurality of additional neural network units not in the original neural network structure; initializing values of the parameters of the original neural network units and the additional neural network units so that the larger neural network generates the same outputs from the same inputs as the original neural network; and training the larger neural network to determine trained values of the parameters of the original neural network units and the additional neural network units from the initialized values.
    Type: Application
    Filed: November 11, 2016
    Publication date: May 18, 2017
    Applicant: Google Inc.
    Inventors: Ian Goodfellow, Tianqi Chen, Jonathon Shlens
  • Patent number: 9652695
    Abstract: Systems and techniques for labeling objects within an image. The objects may be labeled by selecting an option from a plurality of options such that each option is a potential label for the object. An option may have an option score associated with. Additionally, a relation score may be calculated for a first option and a second option corresponding to a second object in an image. The relation score may be based on a frequency, probability, or observance corresponding to the co-occurrence of text associated with the first option and the second option in a text corpus such as the World Wide Web. An option may be selected as a label for an object based on a global score calculated based at least on an option score and relation score associated with the option.
    Type: Grant
    Filed: December 20, 2013
    Date of Patent: May 16, 2017
    Assignee: Google Inc.
    Inventors: Samy Bengio, Jeffrey Adgate Dean, Quoc Le, Jonathon Shlens, Yoram Singer
  • Publication number: 20160378863
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for selecting representative frames for videos. One of the methods includes receiving a search query; determining a query representation for the search query; obtaining data identifying a plurality of responsive videos for the search query, wherein each responsive video comprises a plurality of frames, wherein each frame has a respective frame representation; selecting, for each responsive video, a representative frame from the responsive video using the query representation and the frame representations for the frames in the responsive video; and generating a response to the search query, wherein the response to the search query includes a respective video search result for each of the responsive videos, and wherein the respective video search result for each of the responsive videos includes a presentation of the representative video frame from the responsive video.
    Type: Application
    Filed: June 24, 2015
    Publication date: December 29, 2016
    Inventors: Jonathon Shlens, George Dan Toderici, Sami Ahmad Abu-El-Haija
  • Patent number: 9256807
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating labeled images. One of the methods includes selecting a plurality of candidate videos from videos identified in a response to a search query derived from a label for an object category; selecting one or more initial frames from each of the candidate videos; detecting one or more initial images of objects in the object category in the initial frames; for each initial frame including an initial image of an object in the object category, tracking the object through surrounding frames to identify additional images of the object; and selecting one or more images from the one or more initial images and one or more additional images as database images of objects belonging to the object category.
    Type: Grant
    Filed: March 14, 2013
    Date of Patent: February 9, 2016
    Assignee: Google Inc.
    Inventors: Jonathon Shlens, Quoc V. Le, Gregory S. Corrado, Marc'Aurelio Ranzato
  • Publication number: 20150178383
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for classifying data objects. One of the methods includes obtaining data that associates each term in a vocabulary of terms with a respective high-dimensional representation of the term; obtaining classification data for a data object, wherein the classification data includes a respective score for each of a plurality of categories, and wherein each of the categories is associated with a respective category label; computing an aggregate high-dimensional representation for the data object from high-dimensional representations for the category labels associated with the categories and the respective scores; identifying a first term in the vocabulary of terms having a high-dimensional representation that is closest to the aggregate high-dimensional representation; and selecting the first term as a category label for the data object.
    Type: Application
    Filed: December 19, 2014
    Publication date: June 25, 2015
    Inventors: Gregory Sean Corrado, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea L. Frome, Jeffrey Adgate Dean, Mohammad Norouzi
  • Publication number: 20150178596
    Abstract: Systems and techniques are disclosed for labeling objects within an image. The objects may be labeled by selecting an option from a plurality of options such that each option is a potential label for the object. An option may have an option score associated with. Additionally, a relation score may be calculated for a first option and a second option corresponding to a second object in an image. The relation score may be based on a frequency, probability, or observance corresponding to the co-occurrence of text associated with the first option and the second option in a text corpus such as the World Wide Web. An option may be selected as a label for an object based on a global score calculated based at least on an option score and relation score associated with the option.
    Type: Application
    Filed: December 20, 2013
    Publication date: June 25, 2015
    Applicant: Google Inc.
    Inventors: Samy Bengio, Jeffrey Adgate Dean, Quoc Le, Jonathon Shlens, Yoram Singer
  • Patent number: 9009083
    Abstract: A mechanism for automatic quantification of multimedia production quality is presented. A method of embodiments includes assembling data samples from users, the data samples indicating a relative production quality of a set of content items based on a comparison of production quality between content items in the set, extracting content features from each of the content items in the set, and learning, based on the data samples from the plurality of users, a statistical model on the extracted content features, wherein the learned statistical model can predict a production quality of another content item that is not part of the set of content items.
    Type: Grant
    Filed: February 15, 2012
    Date of Patent: April 14, 2015
    Assignee: Google Inc.
    Inventors: Sanketh Shetty, Jonathon Shlens, Hrishikesh Aradhye