Patents by Inventor Emmanouil Koukoumidis

Emmanouil Koukoumidis has filed for patents to protect the following inventions. This listing includes patent applications that are pending as well as patents that have already been granted by the United States Patent and Trademark Office (USPTO).

  • Patent number: 11908181
    Abstract: In one embodiment, a method includes receiving a user query inputted on a head-mounted device from the head-mounted device, wherein the user query corresponds to multiple dialog-intents, executing multiple tasks corresponding to the multiple dialog-intents, generating a multi-perspective response by a stitching model based on two or more of execution results of the multiple tasks, wherein the stitching model combines the two or more of the execution results based on natural language processing, and wherein the multi-perspective response comprises a natural-language response combining the two or more execution results, and sending instructions to the head-mounted device for presenting the multi-perspective response on the head-mounted device.
    Type: Grant
    Filed: February 3, 2023
    Date of Patent: February 20, 2024
    Assignee: Meta Platforms, Inc.
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Rajen Subba, Heidi Young, Rushin Shah, Jinsong Yu, Benoit F. Dumoulin, Jeremy Gillmor Kahn, Chandrasekhar Iyer
  • Patent number: 11908179
    Abstract: In one embodiment, a method includes, by a client system, receiving a user input from the first user of an online social network, where the user input is associated with a subject and an intent, determining whether one or more agents associated with the intent are able to resolve the user input, identifying, responsive to determining that the one or more agents are not able to resolve the user input, one or more entities related to the subject, where each identified entity is connected to the first user via the online social network, and presenting, at the client system, one or more recommendations to the first user corresponding to the one or more entities.
    Type: Grant
    Filed: April 11, 2022
    Date of Patent: February 20, 2024
    Assignee: Meta Platforms, Inc.
    Inventors: Jason Schissel, Emmanouil Koukoumidis, Shane Haladuick, Nicholas Robin, Fuchun Peng, Aleksey Sergeyevich Fadeev
  • Publication number: 20240054390
    Abstract: Labels are often over labeled by machine-learning models and under labeled by human labelers. A solution to the over and under labeling problem is to have both a machine-learning model and a human label a document, then send the document to a parser to determine the discrepancies. The discrepancies are then presented to a human to review and decide whether the machine-learning model identified labels are labels. The feedback is then given to the machine-learning model for further improvement in its confidence calculations which via a confidence threshold determine if the identified labels are presented.
    Type: Application
    Filed: August 19, 2022
    Publication date: February 15, 2024
    Inventors: James Bradley Wendt, Sandeep Tata, Lauro Ivo Beltrao Colaco Costa, Emmanouil Koukoumidis
  • Publication number: 20240046686
    Abstract: A method for document extraction includes receiving, from a user device associated with a user, an annotated document that includes one or more fields. Each respective field of the one or more fields of the annotated document is labeled by a respective annotation. The method includes clustering, using a template matching algorithm, the annotated document into a cluster and inducing, using the annotated document, a document template for the cluster. The method includes receiving, from the user device, an unannotated document including the one or more fields. The method includes clustering, using the template matching algorithm, the unannotated document into the cluster and, in response to clustering the unannotated document into the cluster, extracting, using the document template, the one or more fields.
    Type: Application
    Filed: August 3, 2022
    Publication date: February 8, 2024
    Applicant: Google LLC
    Inventors: Tianjun Ye, Younghwan Jung, Xiaoqi Ren, Wael Farhan, Tianjun Fu, Nikolaos Kofinas, Nikolay Alexeevich Glushnev, Matthew Eastberg Persons, Xiao Liu, Evan S. Huang, Emmanouil Koukoumidis, Bhavishya Mittal
  • Publication number: 20230419020
    Abstract: A method includes obtaining a document with textual fields and a visual element. For each textual field, the method includes determining a textual offset for the textual field that indicates a location of the textual field relative to each other textual field in the document. The method includes detecting, using a machine learning vision model, the visual element and determining a visual element offset indicating a location of the visual element relative to each textual field in the document. The method includes assigning the visual element a visual element anchor token and inserting the visual element anchor token into the textual fields in an order based on the visual element offset and the respective textual offsets. The method also includes, after inserting the visual element anchor token, extracting, using a text-based extraction model, from the textual fields, structured entities representing the series of textual fields and the visual element.
    Type: Application
    Filed: June 22, 2022
    Publication date: December 28, 2023
    Applicant: Google LLC
    Inventors: Nikolay Glushnev, Qingze Wang, Emmanouil Koukoumidis, Henry Wahyudi Setiawan, Lauro Ivo Beltrao Colaco Costa, Vincent Perot
  • Patent number: 11842522
    Abstract: In one embodiment, a method includes receiving a user query inputted on a head-mounted device from the head-mounted device, wherein the user query corresponds to multiple dialog-intents, executing multiple tasks corresponding to the multiple dialog-intents, generating a multi-perspective response by a stitching model based on two or more of execution results of the multiple tasks, wherein the stitching model combines the two or more of the execution results based on natural language processing, and wherein the multi-perspective response comprises a natural-language response combining the two or more execution results, and sending instructions to the head-mounted device for presenting the multi-perspective response on the head-mounted device.
    Type: Grant
    Filed: February 3, 2023
    Date of Patent: December 12, 2023
    Assignee: Meta Platforms, Inc.
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Rajen Subba, Heidi Young, Rushin Shah, Jinsong Yu, Benoit F. Dumoulin, Jeremy Gillmor Kahn, Chandrasekhar Iyer
  • Patent number: 11715289
    Abstract: In one embodiment, a method includes receiving a user query associated with dialog-intents at a client system, executing tasks corresponding to the dialog-intents, generating a multi-perspective response by a stitching model based on two or more of execution results of the tasks, wherein the multi-perspective response comprises a natural-language response combining the two or more execution results, and presenting the multi-perspective response at the client system.
    Type: Grant
    Filed: December 6, 2021
    Date of Patent: August 1, 2023
    Assignee: Meta Platforms, Inc.
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Rajen Subba, Heidi Young, Rushin Shah, Jinsong Yu, Benoit F. Dumoulin, Jeremy Gillmor Kahn, Chandrasekhar Iyer
  • Patent number: 11704900
    Abstract: In one embodiment, a method includes, by a client system, receiving, at the client system, a first user input, processing by the client system, the first user input to provide an initial response by identifying one or more entities referenced by the first user input and providing, by the client system, the initial response, where the initial response includes a conversational filler referencing at least one of the one or more identified entities, processing the first user input to provide a complete response by identifying, by the client system, one or more intents and one or more slots associated with the first user input based on a semantic analysis by a natural-language understanding module, and providing, by the client system, the complete response subsequent to the initial response, where the complete response is based on the one or more intents and the one or more slots.
    Type: Grant
    Filed: February 7, 2022
    Date of Patent: July 18, 2023
    Assignee: Meta Platforms, Inc.
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Mohsen M Agsen
  • Patent number: 11688159
    Abstract: In one embodiment, a method includes receiving an indication of a trigger action by a first user at a client system, wherein the trigger action is associated with a priming content object, identifying related content objects associated with the priming content object, selecting recommended content objects based on the priming content object, the related content objects, and profile information of the first user, wherein each of the selected recommended content objects comprises entity information of entities associated with the priming content object, and presenting content suggestions at the client system, wherein each content suggestion comprises one of the selected recommended content objects.
    Type: Grant
    Filed: April 7, 2021
    Date of Patent: June 27, 2023
    Assignee: Meta Platforms, Inc.
    Inventors: Fuchun Peng, Kun Han, Wenhai Yang, Cheng Zhang, Vivek Narayanan, Emmanouil Koukoumidis
  • Publication number: 20230195847
    Abstract: A computer-implemented method includes receiving a document insight request that requests document insights for a corpus of documents. The document insight request includes the corpus of documents, a set of entities contained within each document of the corpus of documents, and document insight request parameters that includes a confidence value threshold. The method also includes generating the document insights for the corpus of documents based on the confidence value threshold. Here, the document insights include an accuracy target and a user review rate target. The method also includes transmitting the document insights to the user device causing a graphical user interface to display the document insights on the user device.
    Type: Application
    Filed: February 16, 2022
    Publication date: June 22, 2023
    Applicant: Google LLC
    Inventors: Emmanouil Koukoumidis, Nikolaos Kofinas, Evan Huang, Kiran Bellare, Xiao Liu, Michael Lanning, Lukas Rutishauser
  • Publication number: 20230186618
    Abstract: In one embodiment, a method includes receiving a user query inputted on a head-mounted device from the head-mounted device, wherein the user query corresponds to multiple dialog-intents, executing multiple tasks corresponding to the multiple dialog-intents, generating a multi-perspective response by a stitching model based on two or more of execution results of the multiple tasks, wherein the stitching model combines the two or more of the execution results based on natural language processing, and wherein the multi-perspective response comprises a natural-language response combining the two or more execution results, and sending instructions to the head-mounted device for presenting the multi-perspective response on the head-mounted device.
    Type: Application
    Filed: February 3, 2023
    Publication date: June 15, 2023
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Rajen Subba, Heidi Young, Rushin Shah, Jinsong Yu, Benoit F. Dumoulin, Jeremy Gillmor Kahn, Chandrasekhar Iyer
  • Patent number: 11651586
    Abstract: In one embodiment, a method includes receiving an indication of a trigger action by a first user at a client system, wherein the trigger action is associated with a priming content object, identifying related content objects associated with the priming content object, selecting recommended content objects based on the priming content object, the related content objects, and profile information of the first user, wherein each of the selected recommended content objects comprises entity information of entities associated with the priming content object, and presenting content suggestions at the client system, wherein each content suggestion comprises one of the selected recommended content objects.
    Type: Grant
    Filed: April 7, 2021
    Date of Patent: May 16, 2023
    Assignee: Meta Platforms, Inc.
    Inventors: Fuchun Peng, Kun Han, Wenhai Yang, Cheng Zhang, Vivek Narayanan, Emmanouil Koukoumidis
  • Patent number: 11625622
    Abstract: Aspects of the present disclosure provide systems and methods for detecting memorable events of a user, storing those memorable events in a user profile, and leveraging those events to the user. In so doing, memorable events with positive emotional weight may be recalled to encourage delightful experiences. Alternatively or additionally, memorable events with negative emotional weight may be recalled to increase user engagement for conversational systems.
    Type: Grant
    Filed: June 15, 2017
    Date of Patent: April 11, 2023
    Assignee: Microsoft Technology Licensing, LLC
    Inventors: Emmanouil Koukoumidis, Joseph Johnson, Jr., Dustin I. Abramson, Maria A. Rangel, Ying Wang, Di Li, Steven G. Zittrower, Hailong Mu, Kelli A. Stuckart, Hudong Wang, Pengfei Hao, Donald F. Brinkman, Kati A. London, Zhan Chen
  • Publication number: 20220284051
    Abstract: In one embodiment, a method includes, by a client system, receiving a user input from the first user of an online social network, where the user input is associated with a subject and an intent, determining whether one or more agents associated with the intent are able to resolve the user input, identifying, responsive to determining that the one or more agents are not able to resolve the user input, one or more entities related to the subject, where each identified entity is connected to the first user via the online social network, and presenting, at the client system, one or more recommendations to the first user corresponding to the one or more entities.
    Type: Application
    Filed: April 11, 2022
    Publication date: September 8, 2022
    Inventors: Jason Schissel, Emmanouil Koukoumidis, Shane Haladuick, Nicholas Robin, Fuchun Peng, Aleksey Sergeyevich Fadeev
  • Publication number: 20220166733
    Abstract: In one embodiment, a method includes, by one or more computing systems, receiving, at the client system, a first user input, processing, by the client system, the first user input to provide an initial response by identifying one or more entities referenced by the first user input and providing, by the client system, the initial response, where the initial response comprises a conversational filler referencing at least one of the one or more identified entities, processing the first user input to provide a complete response by identifying, by the client system, one or more intents and one or more slots associated with the first user input based on a semantic analysis by a natural-language understanding module and providing, by the client system, the complete response subsequent to the initial response, where the complete response is based on the one or more intents and the one or more slots.
    Type: Application
    Filed: February 7, 2022
    Publication date: May 26, 2022
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Mohsen M. Agsen
  • Patent number: 11308169
    Abstract: In one embodiment, a method includes receiving a user query from a client system associated with a first user, determining a plurality of dialog-intents, each being associated with a particular agent of a plurality of agents, executing a plurality of tasks corresponding to the user query via the plurality of agents corresponding to the plurality of dialog-intents, receiving a plurality of execution results corresponding to the plurality of tasks from the plurality of agents, respectively, selecting two or more of the plurality of execution results for combination, generating a multi-perspective response based on the selected execution results by a stitching model, wherein the multi-perspective response comprises a natural-language response combining the selected execution results, and sending instructions for presenting the multi-perspective response to the first user to the client system in response to the user query.
    Type: Grant
    Filed: October 31, 2018
    Date of Patent: April 19, 2022
    Assignee: Meta Platforms, Inc.
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Rajen Subba, Heidi Young, Rushin Shah, Jinsong Yu, Benoit F. Dumoulin, Jeremy Gillmor Kahn, Chandrasekhar Iyer
  • Patent number: 11301521
    Abstract: In one embodiment, a method includes receiving a user input by a user from a client system associated with the user, parsing the user input to identify a subject and intent of the user input, determining whether one or more agents associated with the identified intent are able to resolve the user input, identifying one or more entities related to the identified subject of the user input responsive to determining that the one or more agents are not able to resolve the user input, calculating a subject-association score for each of the one or more entities, ranking the one or more entities based on their respective subject-association scores, and sending, to the client system, instructions for presenting one or more recommendations to the user corresponding to one or more of the entities having a ranking greater than a threshold ranking.
    Type: Grant
    Filed: July 17, 2018
    Date of Patent: April 12, 2022
    Assignee: Meta Platforms, Inc.
    Inventors: Jason Schissel, Emmanouil Koukoumidis, Shane Haladuick, Nicholas Robin, Fuchen Peng, Aleksey Sergeyevich Fadeev
  • Publication number: 20220092131
    Abstract: In one embodiment, a method includes receiving a user query associated with dialog-intents at a client system, executing tasks corresponding to the dialog-intents, generating a multi-perspective response by a stitching model based on two or more of execution results of the tasks, wherein the multi-perspective response comprises a natural-language response combining the two or more execution results, and presenting the multi-perspective response at the client system.
    Type: Application
    Filed: December 6, 2021
    Publication date: March 24, 2022
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Rajen Subba, Heidi Young, Rushin Shah, Jinsong Yu, Benoit F. Dumoulin, Jeremy Gillmor Kahn, Chandrasekhar Iyer
  • Patent number: 11263276
    Abstract: In one embodiment, a method includes receiving a user query from a client system associated with a first user, determining a plurality of dialog-intents, each being associated with a particular agent of a plurality of agents, executing a plurality of tasks corresponding to the user query via the plurality of agents corresponding to the plurality of dialog-intents, receiving a plurality of execution results corresponding to the plurality of tasks from the plurality of agents, respectively, selecting two or more of the plurality of execution results for combination, generating a multi-perspective response based on the selected execution results by a stitching model, wherein the multi-perspective response comprises a natural-language response combining the selected execution results, and sending instructions for presenting the multi-perspective response to the first user to the client system in response to the user query.
    Type: Grant
    Filed: October 31, 2018
    Date of Patent: March 1, 2022
    Assignee: Meta Platforms, Inc.
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Rajen Subba, Heidi Young, Rushin Shah, Jinsong Yu, Benoit F. Dumoulin, Jeremy Gillmor Kahn, Chandrasekhar Iyer
  • Patent number: 11245646
    Abstract: In one embodiment, a method includes, by one or more computing systems, receiving, from a client system associated with a first user, a first user input from the first user, identifying one or more entities referenced by the first user input, determining a classification of the first user input based on a machine-learning classifier model, generating several candidate conversational fillers based on the classification of the first user input and the one or more identified entities, wherein each candidate conversational filler references at least one of the one or more identified entities, ranking the candidate conversational fillers based on a relevancy of the candidate conversational filler to the first user input and a decay model hysteresis, and sending instructions for presenting a top-ranked candidate conversational filler as an initial response to the first user.
    Type: Grant
    Filed: November 15, 2018
    Date of Patent: February 8, 2022
    Assignee: Facebook, Inc.
    Inventors: Emmanouil Koukoumidis, Michael Robert Hanson, Mohsen M Agsen