Patents by Inventor Nicholas E. Apostoloff

Nicholas E. Apostoloff has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11816565
    Abstract: Methods and apparatus are disclosed for interpreting a deep neural network (DNN) using a Semantic Coherence Analysis (SCA)-based interpretation technique. In embodiments, a multi-layered DNN that was trained for one task is analyzed using the SCA technique to select one layer in the DNN that produces salient features for another task. In embodiments, the DNN layers are tested with test samples labeled with a set of concept labels. The output features of a DNN layer are gathered and analyzed according to the concepts. In embodiments, the output is scored with a semantic coherence score, which indicates how well the layer separates the concepts, and one layer is selected from the DNN based on its semantic coherence score. In some embodiments, a support vector machine (SVM) or additional neural network may be added to the selected layer and trained to generate classification results based on the outputs of the selected layer.
    Type: Grant
    Filed: February 17, 2020
    Date of Patent: November 14, 2023
    Assignee: Apple Inc.
    Inventors: Moussa Doumbouya, Xavier Suau Cuadros, Luca Zappella, Nicholas E. Apostoloff
  • Patent number: 11663514
    Abstract: Controlling an automated agent in an environment includes obtaining high-quality data regarding a current state of the automated agent; identifying a behavior model; determining a trajectory estimate for the automated agent based on the current state of the automated agent and the behavior model; determining a final trajectory for the automated agent using the trajectory estimate; and controlling the automated agent according to the final trajectory.
    Type: Grant
    Filed: June 12, 2020
    Date of Patent: May 30, 2023
    Assignee: APPLE INC.
    Inventors: Megan M. Maher, Barry-John Theobald, Nicholas E. Apostoloff
  • Patent number: 11475608
    Abstract: One aspect of the disclosure is a non-transitory computer-readable storage medium including program instructions. Operations performed by execution of the program instructions include obtaining an input image that depicts a face of a subject, having an initial facial expression and an initial pose, determining a reference shape description based on the input image, determining a target shape description based on the reference shape description, a facial expression difference, and a pose difference, generating a rendered target shape image using the target shape description, and generating an output image based on the input image and the rendered target shape using an image generator, wherein the output image is a simulated image of the subject of the input image that has a final expression that is based on the initial facial expression and the facial expression difference, and a final pose that is based on the initial pose and the pose difference.
    Type: Grant
    Filed: August 3, 2020
    Date of Patent: October 18, 2022
    Assignee: Apple Inc.
    Inventors: Barry-John Theobald, Nataniel Ruiz Gutierrez, Nicholas E. Apostoloff
  • Patent number: 11437039
    Abstract: Modifying operation of an intelligent agent in response to facial expressions and/or emotions.
    Type: Grant
    Filed: December 23, 2020
    Date of Patent: September 6, 2022
    Inventors: Siddharth Khullar, Abhishek Sharma, Jerremy Holland, Nicholas E. Apostoloff, Russell Y. Webb, Tai-Peng Tian, Tomas J. Pfister
  • Patent number: 11256958
    Abstract: A method that includes obtaining real training samples that include real images that depict real objects, obtaining simulated training samples that include simulated images that depict simulated objects, defining a training dataset that includes at least some of the real training samples and at least some of the simulated training samples, and training a machine learning model to detect subject objects in unannotated input images using the training dataset.
    Type: Grant
    Filed: July 23, 2019
    Date of Patent: February 22, 2022
    Assignee: Apple Inc.
    Inventors: Melanie S. Subbiah, Jamie R. Lesser, Nicholas E. Apostoloff
  • Publication number: 20210166691
    Abstract: Modifying operation of an intelligent agent in response to facial expressions and/or emotions.
    Type: Application
    Filed: December 23, 2020
    Publication date: June 3, 2021
    Inventors: Siddharth Khullar, Abhishek Sharma, Jerremy Holland, Nicholas E. Apostoloff, Russell Y. Webb, Tai-Peng Tian, Tomas J. Pfister
  • Publication number: 20210117782
    Abstract: In some examples, an individually-pruned neural network can estimate blood pressure from a seismocardiogram (SMG). In some examples, a baseline model can be constructed by training the model with SMG data and blood pressure measurement from a plurality of subjects. One or more filters (e.g., the filters in the top layer of the network) can be ranked by separability, which can be used to prune the model for each unseen user that uses the model thereafter, for example. In some examples, individuals can use individually-pruned models to calculate blood pressure using SMG data without corresponding blood pressure measurements.
    Type: Application
    Filed: July 31, 2020
    Publication date: April 22, 2021
    Inventors: Siddharth KHULLAR, Nicholas E. APOSTOLOFF, Amruta PAI
  • Publication number: 20210117778
    Abstract: Methods and apparatus are disclosed for interpreting a deep neural network (DNN) using a Semantic Coherence Analysis (SCA)-based interpretation technique. In embodiments, a multi-layered DNN that was trained for one task is analyzed using the SCA technique to select one layer in the DNN that produces salient features for another task. In embodiments, the DNN layers are tested with test samples labeled with a set of concept labels. The output features of a DNN layer are gathered and analyzed according to the concepts. In embodiments, the output is scored with a semantic coherence score, which indicates how well the layer separates the concepts, and one layer is selected from the DNN based on its semantic coherence score. In some embodiments, a support vector machine (SVM) or additional neural network may be added to the selected layer and trained to generate classification results based on the outputs of the selected layer.
    Type: Application
    Filed: February 17, 2020
    Publication date: April 22, 2021
    Applicant: Apple Inc.
    Inventors: Moussa Doumbouya, Xavier Suau Cuadros, Luca Zappella, Nicholas E. Apostoloff
  • Publication number: 20210097730
    Abstract: One aspect of the disclosure is a non-transitory computer-readable storage medium including program instructions. Operations performed by execution of the program instructions include obtaining an input image that depicts a face of a subject, having an initial facial expression and an initial pose, determining a reference shape description based on the input image, determining a target shape description based on the reference shape description, a facial expression difference, and a pose difference, generating a rendered target shape image using the target shape description, and generating an output image based on the input image and the rendered target shape using an image generator, wherein the output image is a simulated image of the subject of the input image that has a final expression that is based on the initial facial expression and the facial expression difference, and a final pose that is based on the initial pose and the pose difference.
    Type: Application
    Filed: August 3, 2020
    Publication date: April 1, 2021
    Inventors: Barry-John Theobald, Nataniel Ruiz Gutierrez, Nicholas E. Apostoloff
  • Patent number: 10885915
    Abstract: Modifying operation of an intelligent agent in response to facial expressions and/or emotions.
    Type: Grant
    Filed: June 30, 2017
    Date of Patent: January 5, 2021
    Inventors: Siddharth Khullar, Abhishek Sharma, Jerremy Holland, Nicholas E. Apostoloff, Russell Y. Webb, Tai-Peng Tian, Tomas J. Pfister
  • Publication number: 20190348037
    Abstract: Modifying operation of an intelligent agent in response to facial expressions and/or emotions.
    Type: Application
    Filed: June 30, 2017
    Publication date: November 14, 2019
    Inventors: Siddharth Khullar, Abhishek Sharma, Jerremy Holland, Nicholas E. Apostoloff, Russell Y. Webb, Tai-Peng Tian, Tomas J. Pfister