Patents by Inventor Sergey Ioffe

Sergey Ioffe has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 9501510
    Abstract: Systems and methods for facilitating media fingerprinting are provided. In one aspect, a system can include: a memory, a microprocessor, a communication component that receives media; and a media fingerprinting component that fingerprints the media. The media fingerprinting component employs a fingerprint generation component stored in the memory and includes: a first hash generation component that generates sets of hashes corresponding to versions of the media; and a second hash generation component that computes a final hash based, at least, on hashing the sets of hashes. In some aspects, the media fingerprinting component can generate a flip-resistant fingerprint based, at least, on the final hash. In some aspects, the flip-resistant fingerprint is the final hash.
    Type: Grant
    Filed: September 25, 2014
    Date of Patent: November 22, 2016
    Assignee: Google Inc.
    Inventor: Sergey Ioffe
  • Patent number: 9443314
    Abstract: An image processing system automatically segments and labels an image using a hierarchical classification model. A global classification model determines initial labels for an image based on features of the image. A label-based descriptor is generated based on the initial labels. A local classification model is then selected from a plurality of learned local classification model based on the label-based descriptor. The local classification model is applied to the features of the input image to determined refined labels. The refined labels are stored in association with the input image.
    Type: Grant
    Filed: March 29, 2012
    Date of Patent: September 13, 2016
    Assignee: Google Inc.
    Inventors: Qixing Huang, Mei Han, Bo Wu, Sergey Ioffe
  • Publication number: 20160217368
    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using a neural network system that includes a batch normalization layer. One of the methods includes receiving a respective first layer output for each training example in the batch; computing a plurality of normalization statistics for the batch from the first layer outputs; normalizing each component of each first layer output using the normalization statistics to generate a respective normalized layer output for each training example in the batch; generating a respective batch normalization layer output for each of the training examples from the normalized layer outputs; and providing the batch normalization layer output as an input to the second neural network layer.
    Type: Application
    Filed: January 28, 2016
    Publication date: July 28, 2016
    Inventors: Sergey Ioffe, Corinna Cortes
  • Publication number: 20160216848
    Abstract: A computer-implemented method, computer program product, and computing system is provided for interacting with images having similar content. In an embodiment, a method may include identifying a plurality of photographs as including a common characteristic. The method may also include generating a flipbook media item including the plurality of photographs. The method may further include associating one or more interactive control features with the flipbook media item.
    Type: Application
    Filed: April 6, 2016
    Publication date: July 28, 2016
    Applicant: Google Inc.
    Inventors: Sergey Ioffe, Vivek Kwatra, Matthias Grundmann
  • Patent number: 9400809
    Abstract: A method and apparatus are provided for performing an image search based on a search query having a portion P1 and a portion P2. Based on the first search query, a second search query is generated that includes a portion P3 and the portion P2 such that the second search query is broader in scope than the first search query, while still retaining the portion P2 of the first query. A first image search is then performed for the first search query to obtain a first set of search results and a second image search is performed for the second search query to obtain a second set of search results. Consequently, an image from the first set of search results is selected for presentation to a user, wherein the selection is based on content of the second set of search results.
    Type: Grant
    Filed: April 30, 2014
    Date of Patent: July 26, 2016
    Assignee: Google Inc.
    Inventor: Sergey Ioffe
  • Patent number: 9311403
    Abstract: Methods, systems and computer program product embodiments for hashing techniques for determining similarity between data sets are described herein. A method embodiment includes, initializing a random number generator with a weighted min-hash value as a seed, wherein the weighted min-hash value approximates a similarity distance between data sets. A number of bits in the weighted min-hash value is determined by uniformly sampling an integer bit value using the random number generator. A system embodiment includes a repository configured to store a plurality of data sets and a hash generator configured to generate weighted min-hash values from the data sets. The system further includes a similarity determiner configured to determine a similarity between the data sets.
    Type: Grant
    Filed: June 16, 2011
    Date of Patent: April 12, 2016
    Assignee: Google Inc.
    Inventor: Sergey Ioffe
  • Patent number: 9311310
    Abstract: A computer-implemented method, computer program product, and computing system is provided for interacting with images having similar content. In an embodiment, a method may include identifying a plurality of photographs as including a common characteristic. The method may also include generating a flipbook media item including the plurality of photographs. The method may further include associating one or more interactive control features with the flipbook media item.
    Type: Grant
    Filed: October 26, 2012
    Date of Patent: April 12, 2016
    Assignee: Google Inc.
    Inventors: Sergey Ioffe, Vivek Kwatra, Matthias Grundmann
  • Patent number: 9288484
    Abstract: A method and apparatus for performing sparse coding dictionary priming are disclosed. Sparse coding dictionary priming may include iteratively training a coding dictionary, which may include a plurality of codewords or bases. Iteratively training the coding dictionary may include identifying a sampling index cardinality, identifying a portion of a video stream, decomposing the portion of the video stream, and updating the codeword based on the portion of the video stream. Decomposing the portion of the video stream may include randomly identifying a set of codewords from the plurality of codewords wherein a cardinality of the set of codewords is the sampling index cardinality and wherein the sampling index cardinality is less a cardinality of the plurality of codewords, and determining a codeword having a maximum correlation with the portion of the video stream from the set of codewords.
    Type: Grant
    Filed: August 30, 2012
    Date of Patent: March 15, 2016
    Assignee: GOOGLE INC.
    Inventors: Sergey Ioffe, Pascal Massimino
  • Patent number: 9286549
    Abstract: A linear function describing a framework for identifying an object of class k in an image sample x may be described by: wk*x+bk, where bk is the bias term. The higher the value obtained for a particular classifier, the better the match or strength of identity. A method is disclosed for classifier and/or content padding to convert dot-products to distances, applying a hashing and/or nearest neighbor technique on the resulting padded vectors, and preprocessing that may improve the hash entropy. A vector for an image, an audio, and/or a video may be received. One or more classifier vectors may be obtained. A padded image, video, and/or audio vector and classifier vector may be generated. A dot product may be approximated and a hashing and/or nearest neighbor technique may be performed on the approximated dot product to identify at least one class (or object) present in the image, video, and/or audio.
    Type: Grant
    Filed: July 15, 2013
    Date of Patent: March 15, 2016
    Assignee: Google Inc.
    Inventors: Sergey Ioffe, Alexander Toshkov Toshev
  • Patent number: 9235875
    Abstract: Systems, methods and computer readable media for image enhancement using learned non-photorealistic effects. In some implementations, a method can include obtaining an original image. The method can also include analyzing the original image to determine one or more characteristics of the original image. The method can further include selecting one or more filters based on the one or more characteristics and applying the one or more filters to the original image to generate a modified image. The method can include causing the modified image to be displayed.
    Type: Grant
    Filed: November 1, 2012
    Date of Patent: January 12, 2016
    Assignee: Google Inc.
    Inventors: Sergey Ioffe, Hui Fang
  • Patent number: 9177350
    Abstract: Systems and methods for facilitating video fingerprinting are provided. In one embodiment, a system can include: a memory, a microprocessor, a communication component that receives a video, and a video fingerprinting component that fingerprints the video with a subfingerprint (SFP). The video fingerprinting component can employ an SFP component stored in the memory and that comprises: a feature extraction component that determines local descriptors for at least one frame of a video; and a quantization component that quantizes the local descriptors to generate first frame information including a set of values for the at least one frame. The SFP component can also include: an accumulation component that accumulates first frame information over a snippet of the video; and an SFP generation component that computes the SFP associated with the snippet. The SFP can be computed based on a hash based on the accumulated first frame information over the snippet.
    Type: Grant
    Filed: January 14, 2014
    Date of Patent: November 3, 2015
    Assignee: Google Inc.
    Inventor: Sergey Ioffe
  • Patent number: 9143784
    Abstract: This disclosure relates to transformation invariant media matching. A fingerprinting component can generate a transformation invariant identifier for media content by adaptively encoding the relative ordering of signal markers in media content. The signal markers can be adaptively encoded via reference point geometry, or ratio histograms. An identification component compares the identifier against a set of identifiers for known media content, and the media content can be matched or identified as a function of the comparison.
    Type: Grant
    Filed: September 12, 2013
    Date of Patent: September 22, 2015
    Assignee: Google Inc.
    Inventors: Jay Yagnik, Sergey Ioffe
  • Patent number: 9135674
    Abstract: A method and system generates and compares fingerprints for videos in a video library. The video fingerprints provide a compact representation of the temporal locations of discontinuities in the video that can be used to quickly and efficiently identify video content. Discontinuities can be, for example, shot boundaries in the video frame sequence or silent points in the audio stream. Because the fingerprints are based on structural discontinuity characteristics rather than exact bit sequences, visual content of videos can be effectively compared even when there are small differences between the videos in compression factors, source resolutions, start and stop times, frame rates, and so on. Comparison of video fingerprints can be used, for example, to search for and remove copyright protected videos from a video library. Furthermore, duplicate videos can be detected and discarded in order to preserve storage space.
    Type: Grant
    Filed: November 27, 2013
    Date of Patent: September 15, 2015
    Assignee: Google Inc.
    Inventors: Jay Yagnik, Henry Rowley, Sergey Ioffe
  • Patent number: 9122705
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for scoring hash functions. In one aspect, a method includes computing one or more first performance indicators for an objective function computed on each of one or more sets of input data elements using a set of hash functions. A first overall performance indicator is computed using each of the computed performance indicators. The candidate hash function is added to the set of hash functions to generate a second set of hash functions. Second performance indicators are computed for the objective function computed on each of the sets of input data elements using the second set of hash functions. A second overall performance indicator is computed using each of the computed second performance indicators, and a score is computed for the candidate hash function using the first overall performance indicator and the second overall performance indicator.
    Type: Grant
    Filed: March 15, 2012
    Date of Patent: September 1, 2015
    Assignee: Google Inc.
    Inventor: Sergey Ioffe
  • Patent number: 9118843
    Abstract: Examples of methods and systems for creating swivel views from handheld video are described. In some examples, a method may be performed by a handheld device to receive or capture a video of a target object and the video may include a plurality of frames and content of the target object from a plurality of viewpoints. The device may determine one or more approximately corresponding frames of the video including content of the target object from a substantially matching viewpoint and may align the approximately corresponding frames of the video based on one or more feature points of the target object to generate an aligned video. The device may provide sampled frames from multiple viewpoints from the aligned video, configured for viewing the target object in a rotatable manner, such as in a swivel view format.
    Type: Grant
    Filed: January 17, 2013
    Date of Patent: August 25, 2015
    Assignee: Google Inc.
    Inventors: Sergey Ioffe, Christian Frueh
  • Patent number: 9110923
    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an image ranking model to rank images based on hashes of their contents using a lookup table. An image training set is received. An image ranking model is trained with the training set by generating an image hash for each image of the ordered pair of images based on one or more features extracted from the image, computing a first score for a first image hash of a first image of the pair and a second score for a second image hash of a second image of the pair using the image ranking model, determining whether to update the image ranking model based on the first score and the second score, and updating the image ranking model using an update value based on the first score and the second score.
    Type: Grant
    Filed: March 3, 2011
    Date of Patent: August 18, 2015
    Assignee: Google Inc.
    Inventors: Yangli Hector Yee, Sergey Ioffe, Samy Bengio
  • Patent number: 9092859
    Abstract: Systems and methods facilitating random number generation of hashes for video and/or audio are provided. In one embodiment, a system can include: a memory, and a microprocessor that executes computer executable components. The components can include a weighted distribution generation component that can generate a sampling distribution of a weighted combination of uniform distributions, and obtain a sample value from the sampling distribution. In one embodiment, horizontal regions of substantially equal area can be identified. The sample value can be obtained by selecting one of the horizontal regions, and uniformly selecting a coordinate from the horizontal region. The coordinate can correspond to a value on a horizontal axis of the sampling distribution, and the value can be equal to a sample value. The sample value can be employed to compute a hash employed in video and/or audio fingerprinting and/or in computing image descriptors for video.
    Type: Grant
    Filed: May 19, 2014
    Date of Patent: July 28, 2015
    Assignee: Google Inc.
    Inventor: Sergey Ioffe
  • Patent number: 9087260
    Abstract: Systems and methods for quantizing a local descriptor in video fingerprinting applications are provided. In one or more embodiments, local features of a video are extracted and characterized by a set of feature dimensions. The feature dimensions are then quantized to yield a quantized local descriptor for the video. To introduce a degree of pseudorandom variation in the quantization grids, a cascaded random quantization technique is employed to quantize the dimensions, wherein a quantized value for a given dimension is used to quantize a next dimension in sequence.
    Type: Grant
    Filed: January 3, 2012
    Date of Patent: July 21, 2015
    Assignee: Google Inc.
    Inventor: Sergey Ioffe
  • Patent number: 9076076
    Abstract: A system and method is provided that determines whether objects in one image are visually similar to objects in another image by replacing the images' backgrounds with other images, such as a solid color or an image with texture, and comparing the resulting histograms.
    Type: Grant
    Filed: August 15, 2012
    Date of Patent: July 7, 2015
    Assignee: Google Inc.
    Inventors: Sergey Ioffe, Troy Chinen
  • Publication number: 20150186793
    Abstract: A computer-implemented method can include receiving training data that includes a set of non-matching pairs and a set of matching pairs. The method can further include calculating a non-matching collision probability for each non-matching pair of the set of non-matching pairs and a matching collision probability for each matching pair of the set of matching pairs. The method can also include generating a machine learning model that includes a first threshold and a second threshold. An unknown item and a particular known item are classified as not matching when their collision probability is less than the first threshold, and as matching when their collision probability is greater than the second threshold. The first threshold and the second threshold can be selected based on a minimization of errors in classification of matching and non-matching pairs in the training data, and a maximization of a retrieval efficiency metric.
    Type: Application
    Filed: December 27, 2013
    Publication date: July 2, 2015
    Applicant: GOOGLE INC.
    Inventors: Sergey Ioffe, Samy Bengio