Patents by Inventor Pietro Perona

Pietro Perona has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Publication number: 20190034831
    Abstract: Systems and methods for the annotation of source data in accordance with embodiments of the invention are disclosed. In one embodiment, a data annotation server system obtains a set of source data, provides at least one subset of source data to at least one annotator device, obtains a set of annotation data from the at least one annotator device for each subset of source data, classifies the source data based on the annotation data using a machine classifier for each subset of source data, generates annotator model data describing the characteristics of the at least one annotator device, and generates source data model data describing at least one piece of source data in the set of source data, where the source data model data includes label data identifying the estimated ground truth for each piece of source data in the set of source data.
    Type: Application
    Filed: March 19, 2018
    Publication date: January 31, 2019
    Inventors: Pietro Perona, Grant Van Horn, Steven J. Branson
  • Patent number: 10157217
    Abstract: Systems and methods for the crowdsourced clustering of data items in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for determining categories for a set of source data includes obtaining a set of source data, determining a plurality of subsets of the source data, where a subset of the source data includes a plurality of pieces of source data in the set of source data, generating a set of pairwise annotations for the pieces of source data in each subset of source data, clustering the set of source data into related subsets of source data based on the sets of pairwise labels for each subset of source data, and identifying a category for each related subset of source data based on the clusterings of source data and the source data metadata for the pieces of source data in the group of source data.
    Type: Grant
    Filed: May 27, 2016
    Date of Patent: December 18, 2018
    Assignee: California Institute of Technology
    Inventors: Ryan Gomes, Peter Welinder, Andreas Krause, Pietro Perona
  • Patent number: 10121064
    Abstract: Systems and methods for performing behavioral detection using three-dimensional tracking and machine learning in accordance with various embodiments of the invention are disclosed. One embodiment of the invention involves a the classification application that directs a microprocessor to: identify at least a primary subject interacting with a secondary subject within a sequence of frames of image data including depth information; determine poses of the subjects; extract a set of parameters describing the poses and movement of at least the primary and secondary subjects; and detect a social behavior performed by at least the primary subject and involving at least the second subject using a classifier trained to discriminate between a plurality of social behaviors based upon the set of parameters describing poses and movement.
    Type: Grant
    Filed: June 15, 2016
    Date of Patent: November 6, 2018
    Assignee: California Institute of Technology
    Inventors: Weizhe Hong, David J. Anderson, Pietro Perona
  • Patent number: 9928278
    Abstract: Systems and methods for distributed data annotation in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a distributed data annotation server system includes a storage device configured to store source data, one or more annotators, annotation tasks and a processor, wherein a distributed data annotation application configures the processor to receive source data including one or more pieces of source data, select one or more annotators, create one or more annotation tasks for the selected annotators and source data, request one or more annotations for the source data using the annotation tasks, receive annotations, determine source data metadata for at least one piece of source data using the received annotations, generate annotator metadata for at least one annotator using the received annotations and the source data, and estimate the ground truth for the source data using the source data metadata and the annotator metadata.
    Type: Grant
    Filed: March 6, 2014
    Date of Patent: March 27, 2018
    Assignee: California Institute of Technology
    Inventors: Peter Welinder, Pietro Perona
  • Patent number: 9898701
    Abstract: Systems and methods for determining annotator performance in the distributed annotation of source data in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for clustering annotators includes obtaining a set of source data, determining a training data set representative of the set of source data, obtaining sets of annotations from a set of annotators for a portion of the training data set, for each annotator determining annotator recall metadata based on the set of annotations provided by the annotator for the training data set and determining annotator precision metadata based on the set of annotations provided by the annotator for the training data set, and grouping the annotators into annotator groups based on the annotator recall metadata and the annotator precision metadata.
    Type: Grant
    Filed: May 27, 2016
    Date of Patent: February 20, 2018
    Assignee: California Institute of Technology
    Inventors: Peter Welinder, Pietro Perona
  • Publication number: 20170287170
    Abstract: Some embodiments of the invention provide a method for identifying geographic locations and for performing a fine-grained classification of elements detected in images captured from multiple different viewpoints or perspectives. In several embodiments, the method identifies the geographic locations by probabilistically combining predictions from the different viewpoints by warping their outputs to a common geographic coordinate frame. The method of certain embodiments performs the fine-grained classification based on image portions from several images associated with a particular geographic location, where the images are captured from different perspectives and/or zoom levels.
    Type: Application
    Filed: April 3, 2017
    Publication date: October 5, 2017
    Applicants: California Institute of Technology, ETH Zurich
    Inventors: Pietro Perona, Steven J. Branson, Jan D. Wegner, David C. Hall
  • Patent number: 9773160
    Abstract: Images are searched to locate faces that are the same as a query face. Images that include a face that is the same as the query face may be presented to a user as search result images. Images also may be sorted by the faces included in the images and presented to the user as sorted search result images. The user may provide explicit or implicit feedback regarding the search result images. Additional feedback may be inferred regarding the search result images based on the user-provided feedback, and the results may be updated based on the user-provided and inferred feedback.
    Type: Grant
    Filed: October 23, 2013
    Date of Patent: September 26, 2017
    Assignees: AOL Inc., California Institute of Technology, Isis Innovation Limited
    Inventors: Keren O Perlmutter, Sharon M Perlmutter, Joshua Alspector, Mark Everingham, Alex Holub, Andrew Zisserman, Pietro Perona
  • Patent number: 9704106
    Abstract: Systems and methods for the annotation of source data using confidence labels in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for determining confidence labels for crowdsourced annotations includes obtaining a set of source data, obtaining a set of training data representative of the set of source data, determining the ground truth for each piece of training data, obtaining a set of training data annotations including a confidence label, measuring annotator accuracy data for at least one piece of training data, and automatically generating a set of confidence labels for the set of unlabeled data based on the measured annotator accuracy data and the set of annotator labels used.
    Type: Grant
    Filed: May 27, 2016
    Date of Patent: July 11, 2017
    Assignee: California Institute of Technology
    Inventors: Peter Welinder, Pietro Perona
  • Publication number: 20170161554
    Abstract: Systems and methods for performing behavioral detection using three-dimensional tracking and machine learning in accordance with various embodiments of the invention are disclosed. One embodiment of the invention involves a the classification application that directs a microprocessor to: identify at least a primary subject interacting with a secondary subject within a sequence of frames of image data including depth information; determine poses of the subjects; extract a set of parameters describing the poses and movement of at least the primary and secondary subjects; and detect a social behavior performed by at least the primary subject and involving at least the second subject using a classifier trained to discriminate between a plurality of social behaviors based upon the set of parameters describing poses and movement.
    Type: Application
    Filed: June 15, 2016
    Publication date: June 8, 2017
    Applicant: California Institute of Technology
    Inventors: Weizhe Hong, David J. Anderson, Pietro Perona
  • Publication number: 20170046567
    Abstract: Systems and methods for performing behavioral detection using three-dimensional tracking and machine learning in accordance with various embodiments of the invention are disclosed. One embodiment of the invention involves a the classification application that directs a microprocessor to: identify at least a primary subject interacting with a secondary subject within a sequence of frames of image data including depth information; determine poses of the subjects; extract a set of parameters describing the poses and movement of at least the primary and secondary subjects; and detect a social behavior performed by at least the primary subject and involving at least the second subject using a classifier trained to discriminate between a plurality of social behaviors based upon the set of parameters describing poses and movement.
    Type: Application
    Filed: June 15, 2016
    Publication date: February 16, 2017
    Applicant: California Institute of Technology
    Inventors: Weizhe Hong, David J. Anderson, Pietro Perona
  • Publication number: 20160275173
    Abstract: Systems and methods for the crowdsourced clustering of data items in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for determining categories for a set of source data includes obtaining a set of source data, determining a plurality of subsets of the source data, where a subset of the source data includes a plurality of pieces of source data in the set of source data, generating a set of pairwise annotations for the pieces of source data in each subset of source data, clustering the set of source data into related subsets of source data based on the sets of pairwise labels for each subset of source data, and identifying a category for each related subset of source data based on the clusterings of source data and the source data metadata for the pieces of source data in the group of source data.
    Type: Application
    Filed: May 27, 2016
    Publication date: September 22, 2016
    Inventors: Ryan Gomes, Peter Welinder, Andreas Krause, Pietro Perona
  • Publication number: 20160275418
    Abstract: Systems and methods for determining annotator performance in the distributed annotation of source data in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for clustering annotators includes obtaining a set of source data, determining a training data set representative of the set of source data, obtaining sets of annotations from a set of annotators for a portion of the training data set, for each annotator determining annotator recall metadata based on the set of annotations provided by the annotator for the training data set and determining annotator precision metadata based on the set of annotations provided by the annotator for the training data set, and grouping the annotators into annotator groups based on the annotator recall metadata and the annotator precision metadata.
    Type: Application
    Filed: May 27, 2016
    Publication date: September 22, 2016
    Inventors: Peter Welinder, Pietro Perona
  • Publication number: 20160275417
    Abstract: Systems and methods for the annotation of source data using confidence labels in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for determining confidence labels for crowdsourced annotations includes obtaining a set of source data, obtaining a set of training data representative of the set of source data, determining the ground truth for each piece of training data, obtaining a set of training data annotations including a confidence label, measuring annotator accuracy data for at least one piece of training data, and automatically generating a set of confidence labels for the set of unlabeled data based on the measured annotator accuracy data and the set of annotator labels used.
    Type: Application
    Filed: May 27, 2016
    Publication date: September 22, 2016
    Inventors: Peter Welinder, Pietro Perona
  • Patent number: 9355360
    Abstract: Systems and methods for determining annotator performance in the distributed annotation of source data in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for clustering annotators includes obtaining a set of source data, determining a training data set representative of the set of source data, obtaining sets of annotations from a set of annotators for a portion of the training data set, for each annotator determining annotator recall metadata based on the set of annotations provided by the annotator for the training data set and determining annotator precision metadata based on the set of annotations provided by the annotator for the training data set, and grouping the annotators into annotator groups based on the annotator recall metadata and the annotator precision metadata.
    Type: Grant
    Filed: June 19, 2013
    Date of Patent: May 31, 2016
    Assignee: California Institute of Technology
    Inventors: Peter Welinder, Pietro Perona
  • Patent number: 9355359
    Abstract: Systems and methods for the annotation of source data using confidence labels in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for determining confidence labels for crowdsourced annotations includes obtaining a set of source data, obtaining a set of training data representative of the set of source data, determining the ground truth for each piece of training data, obtaining a set of training data annotations including a confidence label, measuring annotator accuracy data for at least one piece of training data, and automatically generating a set of confidence labels for the set of unlabeled data based on the measured annotator accuracy data and the set of annotator labels used.
    Type: Grant
    Filed: June 12, 2013
    Date of Patent: May 31, 2016
    Assignee: California Institute of Technology
    Inventors: Peter Welinder, Pietro Perona
  • Patent number: 9355167
    Abstract: Systems and methods for the crowdsourced clustering of data items in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for determining categories for a set of source data includes obtaining a set of source data, determining a plurality of subsets of the source data, where a subset of the source data includes a plurality of pieces of source data in the set of source data, generating a set of pairwise annotations for the pieces of source data in each subset of source data, clustering the set of source data into related subsets of source data based on the sets of pairwise labels for each subset of source data, and identifying a category for each related subset of source data based on the clusterings of source data and the source data metadata for the pieces of source data in the group of source data.
    Type: Grant
    Filed: May 17, 2013
    Date of Patent: May 31, 2016
    Assignee: California Institute of Technology
    Inventors: Ryan Gomes, Peter Welinder, Andreas Krause, Pietro Perona
  • Patent number: 9141878
    Abstract: Similar faces may be determined within images based on human perception of facial similarity. The user may provide an image including a query face to which the user wishes to find faces that are similar. Similar faces may be determined based on similarity information. Similarity information may be generated from information related to a human perception of facial similarity. Images that include faces determined to be similar, based on the similarity information, may be provided to the user as search result images. The user then may provide feedback to indicate the user's perception of similarity between the query face and the search result images.
    Type: Grant
    Filed: August 29, 2013
    Date of Patent: September 22, 2015
    Assignees: AOL Inc., California Institute of Technology, Isis Innovation Limited
    Inventors: Sharon M Perlmutter, Keren O Perlmutter, Joshua Alspector, Alex Holub, Mark Everingham, Pietro Perona, Andrew Zisserman
  • Patent number: 9092458
    Abstract: A system and method of sorting graphics files with a graphic server and image search engine is disclosed. The method preferably comprises the steps of: receiving a plurality of search results where each of the search results comprising one or more associated graphics; using a general purpose computer to identifying one or more groups of said graphics that depict or otherwise possess similar visual features; and returning the plurality of search results to a user in accordance with said identified groups. The preferred embodiment effectively clusters graphics files, particularly image files, based on upon shared scale-invariant features in order to enable the user to readily identify collections of images relevant to the user while skip past collections of less relevant images.
    Type: Grant
    Filed: September 13, 2012
    Date of Patent: July 28, 2015
    Assignee: iRobot Corporation
    Inventors: Pietro Perona, Luis Goncalves, Enrico Di Bernardo
  • Publication number: 20140289246
    Abstract: Systems and methods for the crowdsourced clustering of data items in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a method for determining categories for a set of source data includes obtaining a set of source data, determining a plurality of subsets of the source data, where a subset of the source data includes a plurality of pieces of source data in the set of source data, generating a set of pairwise annotations for the pieces of source data in each subset of source data, clustering the set of source data into related subsets of source data based on the sets of pairwise labels for each subset of source data, and identifying a category for each related subset of source data based on the clusterings of source data and the source data metadata for the pieces of source data in the group of source data.
    Type: Application
    Filed: May 17, 2013
    Publication date: September 25, 2014
    Applicant: California Institute of Technology
    Inventors: Ryan Gomes, Peter Welinder, Andreas Krause, Pietro Perona
  • Publication number: 20140188879
    Abstract: Systems and methods for distributed data annotation in accordance embodiments of the invention are disclosed. In one embodiment of the invention, a distributed data annotation server system includes a storage device configured to store source data, one or more annotators, annotation tasks and a processor, wherein a distributed data annotation application configures the processor to receive source data including one or more pieces of source data, select one or more annotators, create one or more annotation tasks for the selected annotators and source data, request one or more annotations for the source data using the annotation tasks, receive annotations, determine source data metadata for at least one piece of source data using the received annotations, generate annotator metadata for at least one annotator using the received annotations and the source data, and estimate the ground truth for the source data using the source data metadata and the annotator metadata.
    Type: Application
    Filed: March 6, 2014
    Publication date: July 3, 2014
    Applicant: California Institute of Technology
    Inventors: Peter Welinder, Pietro Perona