Situation Aware NLU/NLP
An arrangement and corresponding method are described for natural language processing. A natural language understanding (NLU) arrangement processes a natural language input to determine a corresponding sentence-level interpretation. A user state component maintains user context data that characterizes an operating context of the NLU arrangement. Operation of the NLU arrangement is biased by the user context data.
Latest NUANCE COMMUNICATIONS, INC. Patents:
- INTERACTIVE VOICE RESPONSE SYSTEMS HAVING IMAGE ANALYSIS
- GESTURAL PROMPTING BASED ON CONVERSATIONAL ARTIFICIAL INTELLIGENCE
- SPEECH DIALOG SYSTEM AND RECIPIROCITY ENFORCED NEURAL RELATIVE TRANSFER FUNCTION ESTIMATOR
- Automated clinical documentation system and method
- CROSS-ATTENTION BETWEEN SPARSE EXTERNAL FEATURES AND CONTEXTUAL WORD EMBEDDINGS TO IMPROVE TEXT CLASSIFICATION
The present invention relates to natural language processing that includes user context information.
BACKGROUND ARTNatural Language Processing (NLP) and Natural Language Understanding (NLU) involve using computer processing to extract meaningful information from natural language inputs such as human generated speech and text. One recent application of such technology is processing speech and/or text queries in mobile devices such as smartphones.
Embodiments of the present invention are directed to an arrangement and corresponding method for natural language processing. A natural language understanding (NLU) arrangement processes a natural language input to determine a corresponding sentence-level interpretation. A user state component maintains user context data that characterizes an operating context of the NLU arrangement. Operation of the NLU arrangement is biased by the user context data.
The NLU arrangement may use rule-based NLU and/or statistical-based NLU, semantic templates and/or a mention detection approach for processing the natural language input. In addition, the NLU arrangement may process the natural language input using multiple parallel domain pipelines, each representing a different specific subject domain of related concepts. The natural language input may be developed by a mobile device. The NLU arrangement may process the natural language input with minimum response latency.
Current NLU arrangements are state-less and do not reflect any notion of what the user is currently doing or what he has done in the past. Embodiments of the present invention are directed to an arrangement for natural language processing of natural language inputs, such as a speech input query from a smartphone or other mobile device, based on user state component maintains user context data that characterizes an operating context of the NLU arrangement.
-
- [IntentSearch] search for [/IntentSearch]
- [SportTeam] miami heat [/Sport Team]
Individual mentions may also receive additional semantic attachments (SemAtt) representing additional associated relationship information. For example, the mention, SportTeam =“Miami Heat”, may have a SemAtt, TeamURL=www.nba.com/heat. Domain expert interpreters within each domain pipeline 202 are then used to process the mentions and semantic attachments to determine domain interpretation candidates that represent the query. The outputs of all the parallel domain pipelines 202 are funneled together and applied to a global evidence ranker 203 which is typically (but not necessarily restricted to) a classifier trained on relevant data and using features from the domain experts as well as other external sources—for example, current list of in-theater movies, list of popular songs, etc.—to determine a final output interpretation that represents the initial natural language input 201.
During operation of the NLU arrangement, a user state model 204 collects user context data 205 such as user activities (current and/or past), entities that the user frequently refers to, etc. In different specific embodiments, the user state model 204 may extract any and all meta-data that may be available in the natural language input 201 for use as user context data 205. In addition or alternatively, the stored user context data 205 may include various contextual information cures that are developed at the different processing stages of the NLU arrangement—i.e., by any or all of the processing stages within the individual domain pipelines 202 or by the global evidence ranker 203—including without limitation, classification tags, query intents, semantic interpretations, grammar data, modifier data, ate/time, etc.
Based on this user context data 205 the user state model 204 biases the operation of the NLU arrangement to reflect the user context in which the natural language input 201 is provided. In specific NLU arrangements such as the one shown in
For example, assume that the user of a mobile device has an open eBook application. The user then says: “harry potter”. Even if Harry Potter also is a blockbuster movie series and a video game, the user state model 204 will bias the NLU arrangement in returning this natural language input 201 of “Harry Potter” as referring to a Book.
In another scenario, a user may say “book a table at Rialto tomorrow night at 8 pm for 8”. The reservation is completed. The next day at around 8 pm, the user state model 204 biases the operation of the NLU arrangement towards interpretations reflecting that known user context data 205 that the user is dining with seven other people at a restaurant (of which the NLU arrangement knows the location). So if at 7:30 pm the user says “directions”, the user state model 204 biases the operation of the NLU arrangement to return directions to the rialto restaurant referenced in the reservation stored in the user context data 205. And if the user says the next day: “book a taxi”, the user state model 204 biases the NLU's interpretation to be aware that the person *might* require two taxis.
In another example, a user says “post status on Facebook: the avenger movie is great”. Then the person searches for “Jackson on Wikipedia”. Since the user context data 205 reflects the user's earlier like of “the avenger” movie, the user state model 204 biases the operation of the NLU arrangement to return the Wikipedia page for Samuel L. Jackson rather that Michael Jackson (or a disambiguation page from Wikipedia).
Embodiments of the invention may be implemented in whole or in part in any conventional computer programming language such as VHDL, SystemC, Verilog, ASM, etc. Alternative embodiments of the invention may be implemented as pre-programmed hardware elements, other related components, or as a combination of hardware and software components.
Embodiments can be implemented in whole or in part as a computer program product for use with a computer system. Such implementation may include a series of computer instructions fixed either on a tangible medium, such as a computer readable medium (e.g., a diskette, CD-ROM, ROM, or fixed disk) or transmittable to a computer system, via a modem or other interface device, such as a communications adapter connected to a network over a medium. The medium may be either a tangible medium (e.g., optical or analog communications lines) or a medium implemented with wireless techniques (e.g., microwave, infrared or other transmission techniques). The series of computer instructions embodies all or part of the functionality previously described herein with respect to the system. Those skilled in the art should appreciate that such computer instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Furthermore, such instructions may be stored in any memory device, such as semiconductor, magnetic, optical or other memory devices, and may be transmitted using any communications technology, such as optical, infrared, microwave, or other transmission technologies. It is expected that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the network (e.g., the Internet or World Wide Web). Of course, some embodiments of the invention may be implemented as a combination of both software (e.g., a computer program product) and hardware. Still other embodiments of the invention are implemented as entirely hardware, or entirely software (e.g., a computer program product).
Although various exemplary embodiments of the invention have been disclosed, it should be apparent to those skilled in the art that various changes and modifications can be made which will achieve some of the advantages of the invention without departing from the true scope of the invention.
Claims
1. An arrangement for natural language processing comprising:
- a natural language understanding (NLU) arrangement for processing a natural language input to determine a corresponding sentence-level interpretation; and
- a user state component for maintaining user context data that characterizes an operating context of the NLU arrangement;
- wherein operation of the NLU arrangement is biased by the user context data.
2. An arrangement according to claim 1, wherein the NLU arrangement uses semantic templates for processing the natural language input.
3. An arrangement according to claim 1, wherein the NLU arrangement uses a mention detection approach for processing the natural language input.
4. An arrangement according to claim 1, wherein the NLU arrangement processes the natural language input using a plurality of parallel domain pipelines, each domain pipeline representing a different specific subject domain of related concepts.
5. An arrangement according to claim 1, wherein the NLU arrangement uses both rule-based NLU and statistical-based NLU for processing the natural language input.
6. An arrangement according to claim 1, wherein a mobile device develops the natural language input.
7. An arrangement according to claim 1, wherein the NLU arrangement processes the natural language input with minimum response latency.
8. A method for natural language processing comprising:
- processing a natural language input with a natural language understanding (NLU) arrangement including a user state component for maintaining user context data that characterizes an operating context of the NLU arrangement; and
- biasing operation of the NLU arrangement based on the user context data to determine a corresponding sentence-level interpretation for the natural language input.
9. A method according to claim 8, wherein the NLU arrangement processes the natural language input based on use of semantic templates.
10. A method according to claim 8, wherein the NLU arrangement processes the natural language input using a mention detection approach
11. A method according to claim 8, wherein the NLU arrangement processes the natural language input using a plurality of parallel domain pipelines, each domain pipeline representing a different specific subject domain of related concepts.
12. A method according to claim 8, wherein the NLU arrangement processes the natural language input using both rule-based NLU and statistical-based NLU.
13. A method according to claim 8, wherein the natural language input is developed by a mobile device.
14. A method according to claim 8, wherein the NLU arrangement processes the natural language input with minimum response latency.
Type: Application
Filed: Oct 1, 2012
Publication Date: Apr 3, 2014
Patent Grant number: 9619459
Applicant: NUANCE COMMUNICATIONS, INC. (Burlington, MA)
Inventors: Matthieu Hebert (Melocheville), Jean-Philippe Robichaud (Mercier), Christopher Parisien (Montreal-Quest)
Application Number: 13/632,380