Patents by Inventor Sergey Ioffe

Sergey Ioffe has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 10229326
    Abstract: Methods, systems, and media for summarizing a video with video thumbnails are provided.
    Type: Grant
    Filed: September 7, 2018
    Date of Patent: March 12, 2019
    Assignee: Google LLC
    Inventors: Matthias Grundmann, Alexandra Ivanna Hawkins, Sergey Ioffe
  • Publication number: 20190065899
    Abstract: The present disclosure provides systems and methods that enable distance metric learning using proxies. A machine-learned distance model can be trained in a proxy space in which a loss function compares an embedding provided for an anchor data point of a training dataset to a positive proxy and one or more negative proxies, where each of the positive proxy and the one or more negative proxies serve as a proxy for two or more data points included in the training dataset. Thus, each proxy can approximate a number of data points, enabling faster convergence. According to another aspect, the proxies of the proxy space can themselves be learned parameters, such that the proxies and the model are trained jointly. Thus, the present disclosure enables faster convergence (e.g., reduced training time). The present disclosure provides example experiments which demonstrate a new state of the art on several popular training datasets.
    Type: Application
    Filed: September 20, 2017
    Publication date: February 28, 2019
    Inventors: Yair Movshovitz-Attias, King Hong Leung, Saurabh Singh, Alexander Toshev, Sergey Ioffe
  • Publication number: 20190065957
    Abstract: The present disclosure provides systems and methods that enable distance metric learning using proxies. A machine-learned distance model can be trained in a proxy space in which a loss function compares an embedding provided for an anchor data point of a training dataset to a positive proxy and one or more negative proxies, where each of the positive proxy and the one or more negative proxies serve as a proxy for two or more data points included in the training dataset. Thus, each proxy can approximate a number of data points, enabling faster convergence. According to another aspect, the proxies of the proxy space can themselves be learned parameters, such that the proxies and the model are trained jointly. Thus, the present disclosure enables faster convergence (e.g., reduced training time). The present disclosure provides example experiments which demonstrate a new state of the art on several popular training datasets.
    Type: Application
    Filed: August 30, 2017
    Publication date: February 28, 2019
    Inventors: Yair Movshovitz-Attias, King Hong Leung, Saurabh Singh, Alexander Toshev, Sergey Ioffe
  • Publication number: 20190014354
    Abstract: Methods and systems are disclosed for estimating a user's ability to share content that is of interest to recipients, and of informing a recipient of this ability when the user shares content with the recipient. In one embodiment, a computer system receives an indication that a first user wishes to share a content item (e.g., a video clip, a photo, an audio clip, a webpage, etc.) with a second user. In response, the computer system obtains data pertaining to a prior history of interaction by the second user with content that the first user has previously shared with the second user; determines, based on the obtained data, an estimate of an ability of the first user to predict an interest in the content item by the second user; and provides the estimate to the second user.
    Type: Application
    Filed: May 22, 2012
    Publication date: January 10, 2019
    Applicant: Google Inc.
    Inventor: Sergey Ioffe
  • Publication number: 20190005334
    Abstract: Methods, systems, and media for summarizing a video with video thumbnails are provided.
    Type: Application
    Filed: September 7, 2018
    Publication date: January 3, 2019
    Inventors: Matthias Grundmann, Alexandra Ivanna Hawkins, Sergey Ioffe
  • Patent number: 10158893
    Abstract: A method includes dividing a video uploaded to a user's client device into scenes that include one or more frames. The method also includes generating a digital summary for each scene based on content associated with a respective portion of the video spanned by the scene. The method also includes identifying a matching portion of the uploaded video containing third-party content base on a match between the digital summary associated with the matching portion and the digital summary associated with the third-party content. The method also includes identifying an original portion of the video containing the original content and a usage policy associated with a content owner of the third-party content. The method also includes generating a degraded video based on the usage policy, by applying a quality reduction to the matching portion.
    Type: Grant
    Filed: March 22, 2018
    Date of Patent: December 18, 2018
    Assignee: GOOGLE LLC
    Inventor: Sergey Ioffe
  • Patent number: 10102443
    Abstract: An image processing system automatically segments and labels an image using a hierarchical classification model. A global classification model determines initial labels for an image based on features of the image. A label-based descriptor is generated based on the initial labels. A local classification model is then selected from a plurality of learned local classification model based on the label-based descriptor. The local classification model is applied to the features of the input image to determined refined labels. The refined labels are stored in association with the input image.
    Type: Grant
    Filed: August 10, 2016
    Date of Patent: October 16, 2018
    Assignee: Google LLC
    Inventors: Qixing Huang, Mei Han, Bo Wu, Sergey Ioffe
  • Patent number: 10074015
    Abstract: Methods, systems, and media for summarizing a video with video thumbnails are provided.
    Type: Grant
    Filed: April 13, 2016
    Date of Patent: September 11, 2018
    Assignee: Google LLC
    Inventors: Matthias Grundmann, Alexandra Ivanna Hawkins, Sergey Ioffe
  • Publication number: 20180213269
    Abstract: A method includes dividing a video uploaded to a user's client device into scenes that include one or more frames. The method also includes generating a digital summary for each scene based on content associated with a respective portion of the video spanned by the scene. The method also includes identifying a matching portion of the uploaded video containing third-party content base on a match between the digital summary associated with the matching portion and the digital summary associated with the third-party content. The method also includes identifying an original portion of the video containing the original content and a usage policy associated with a content owner of the third-party content. The method also includes generating a degraded video based on the usage policy, by applying a quality reduction to the matching portion.
    Type: Application
    Filed: March 22, 2018
    Publication date: July 26, 2018
    Inventor: Sergey Ioffe
  • Patent number: 9955196
    Abstract: A video server receives an uploaded video and determines whether the video contains third-party content and which portions of the uploaded video match third-party content. The video server determines whether to degrade the matching portions and/or how (e.g., extent, type) to do so. The video server separates the matching portion from original portions in the uploaded video and generates a degraded version of the matching content by applying an effect such as compression, edge distortion, temporal distortion, noise addition, color distortion, or audio distortion. The video server combines the degraded portions with the original portions to output a degraded version of the uploaded video. The video server stores and/or distributes the degraded version of the uploaded video. The video server may offer the uploading user licensing terms with the content owner that the user may accept to reverse the degradation.
    Type: Grant
    Filed: September 14, 2015
    Date of Patent: April 24, 2018
    Assignee: GOOGLE LLC
    Inventor: Sergey Ioffe
  • Patent number: 9940552
    Abstract: A linear function describing a framework for identifying an object of class k in an image sample x may be described by: wk*x+bk, where bk is the bias term. The higher the value obtained for a particular classifier, the better the match or strength of identity. A method is disclosed for classifier and/or content padding to convert dot-products to distances, applying a hashing and/or nearest neighbor technique on the resulting padded vectors, and preprocessing that may improve the hash entropy. A vector for an image, an audio, and/or a video may be received. One or more classifier vectors may be obtained. A padded image, video, and/or audio vector and classifier vector may be generated. A dot product may be approximated and a hashing and/or nearest neighbor technique may be performed on the approximated dot product to identify at least one class (or object) present in the image, video, and/or audio.
    Type: Grant
    Filed: March 14, 2016
    Date of Patent: April 10, 2018
    Assignee: Google LLC
    Inventors: Sergey Ioffe, Alexander Toshkov Toshev
  • Publication number: 20170243085
    Abstract: A neural network system that includes: multiple subnetworks that includes: a first subnetwork including multiple first modules, each first module including: a pass-through convolutional layer configured to process the subnetwork input for the first subnetwork to generate a pass-through output; an average pooling stack of neural network layers that collectively processes the subnetwork input for the first subnetwork to generate an average pooling output; a first stack of convolutional neural network layers configured to collectively process the subnetwork input for the first subnetwork to generate a first stack output; a second stack of convolutional neural network layers that are configured to collectively process the subnetwork input for the first subnetwork to generate a second stack output; and a concatenation layer configured to concatenate the pass-through output, the average pooling output, the first stack output, and the second stack output to generate a first module output for the first module.
    Type: Application
    Filed: December 30, 2016
    Publication date: August 24, 2017
    Inventors: Vincent O. Vanhoucke, Christian Szegedy, Sergey Ioffe
  • Patent number: 9679226
    Abstract: An image processing system automatically segments and labels an image using a hierarchical classification model. A global classification model determines initial labels for an image based on features of the image. A label-based descriptor is generated based on the initial labels. A local classification model is then selected from a plurality of learned local classification model based on the label-based descriptor. The local classification model is applied to the features of the input image to determined refined labels. The refined labels are stored in association with the input image.
    Type: Grant
    Filed: March 27, 2015
    Date of Patent: June 13, 2017
    Assignee: Google Inc.
    Inventors: Qixing Huang, Mei Han, Bo Wu, Sergey Ioffe
  • Patent number: 9659352
    Abstract: A method, computer program product, and computer system for identifying a first portion of a facial image in a first image, wherein the first portion includes noise. A corresponding portion of the facial image is identified in a second image, wherein the corresponding portion includes less noise than the first portion. One or more filter parameters of the first portion are determined based upon, at least in part, the first portion and the corresponding portion. At least a portion of the noise from the first portion is smoothed based upon, at least in part, the one or more filter parameters. At least a portion of face specific details from the corresponding portion is added to the first portion.
    Type: Grant
    Filed: February 5, 2015
    Date of Patent: May 23, 2017
    Assignee: Google Inc.
    Inventors: Sergey Ioffe, Troy Chinen, Vivek Kwatra, Hui Fang, Yichang Shih
  • Publication number: 20170132512
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage medium, for training a neural network, wherein the neural network is configured to receive an input data item and to process the input data item to generate a respective score for each label in a predetermined set of multiple labels. The method includes actions of obtaining a set of training data that includes a plurality of training items, wherein each training item is associated with a respective label from the predetermined set of multiple labels; and modifying the training data to generate regularizing training data, comprising: for each training item, determining whether to modify the label associated with the training item, and changing the label associated with the training item to a different label from the predetermined set of labels, and training the neural network on the regularizing data.
    Type: Application
    Filed: November 4, 2016
    Publication date: May 11, 2017
    Inventor: Sergey Ioffe
  • Publication number: 20170078718
    Abstract: A video server receives an uploaded video and determines whether the video contains third-party content and which portions of the uploaded video match third-party content. The video server determines whether to degrade the matching portions and/or how (e.g., extent, type) to do so. The video server separates the matching portion from original portions in the uploaded video and generates a degraded version of the matching content by applying an effect such as compression, edge distortion, temporal distortion, noise addition, color distortion, or audio distortion. The video server combines the degraded portions with the original portions to output a degraded version of the uploaded video. The video server stores and/or distributes the degraded version of the uploaded video. The video server may offer the uploading user licensing terms with the content owner that the user may accept to reverse the degradation.
    Type: Application
    Filed: September 14, 2015
    Publication date: March 16, 2017
    Inventor: Sergey Ioffe
  • Patent number: 9588990
    Abstract: Image similarity operations are performed in which a seed image is analyzed, and a set of semantic classifications are determined from analyzing the seed image. The set of semantic classifications can include multiple positive semantic classifications. A distance measure is determined that is specific to the set of semantic classifications. The seed image is compared to a collection of images using the distance measure. A set of similar images is determined from comparing the seed image to the collection of images.
    Type: Grant
    Filed: April 3, 2015
    Date of Patent: March 7, 2017
    Assignee: Google Inc.
    Inventors: Sergey Ioffe, Yushi Jing, Stephen C. Holiday
  • Patent number: 9552549
    Abstract: Systems and techniques are provided for a ranking approach to train deep neural nets for multilabel image annotation. Label scores may be received for labels determined by a neural network for training examples. Each label may be a positive label or a negative label for the training example. An error of the neural network may be determined based on a comparison, for each of the training examples, of the label scores for positive labels and negative labels for the training example and a semantic distance between each positive label and each negative label for the training example. Updated weights may be determined for the neural network based on a gradient of the determined error of the neural network. The updated weights may be applied to the neural network to train the neural network.
    Type: Grant
    Filed: July 28, 2014
    Date of Patent: January 24, 2017
    Assignee: Google Inc.
    Inventors: Yunchao Gong, King Hong Thomas Leung, Alexander Toshkov Toshev, Sergey Ioffe, Yangqing Jia
  • Patent number: 9508023
    Abstract: This disclosure relates to transformation invariant media matching. A fingerprinting component can generate a transformation invariant identifier for media content by adaptively encoding the relative ordering of interest points in media content. The interest points can be grouped into subsets, and stretch invariant descriptors can be generated for the subsets based on ratios of coordinates of interest points included in the subsets. The stretch invariant descriptors can be aggregated into a transformation invariant identifier. An identification component compares the identifier against a set of identifiers for known media content, and the media content can be matched or identified as a function of the comparison.
    Type: Grant
    Filed: April 21, 2014
    Date of Patent: November 29, 2016
    Assignee: Google Inc.
    Inventors: Matthew Sharifi, Sergey Ioffe, Jay Yagnik, Gheorghe Postelnicu, Dominik Roblek, George Tzanetakis
  • Patent number: 9501510
    Abstract: Systems and methods for facilitating media fingerprinting are provided. In one aspect, a system can include: a memory, a microprocessor, a communication component that receives media; and a media fingerprinting component that fingerprints the media. The media fingerprinting component employs a fingerprint generation component stored in the memory and includes: a first hash generation component that generates sets of hashes corresponding to versions of the media; and a second hash generation component that computes a final hash based, at least, on hashing the sets of hashes. In some aspects, the media fingerprinting component can generate a flip-resistant fingerprint based, at least, on the final hash. In some aspects, the flip-resistant fingerprint is the final hash.
    Type: Grant
    Filed: September 25, 2014
    Date of Patent: November 22, 2016
    Assignee: Google Inc.
    Inventor: Sergey Ioffe