Abstract: Systems and methods are described to automatically generate candidate questions and responses to speed the process of response creation and editing for commercial IVAs and chatbots. Rather than create the questions and responses from scratch for a new IVA, the system uses existing questions and responses from a previous or related IVA to train a model that can generate proposed responses to provided questions. The model, or a different model, can further be trained to generate responses using data taken from company or entity-specific data sources such as websites and knowledge bases. After a set of questions and responses have been generated for an IVA they may be reviewed by one or more human reviewers to ensure they are of a suitable quality. Where no previous or related IVA exists to provide example responses, the model may be trained solely using the company or entity-specific data.

Abstract: A system and method for updating computerized language models is provided that automatically adds or deletes terms from the language model to capture trending events or products, while maximizing computer efficiencies by deleting terms that are no longer trending and use of knowledge bases, machine learning model training and evaluation corpora, analysis tools and databases

Abstract: According to principles described herein, a system applies Active Learning methodology to multiple models simultaneously. The system includes a means to distribute the sample selection algorithm across large pools of unlabeled data and a automatic model training deployed on hardware matched to the model type that scales to large volumes of data without consuming all resources.

Abstract: Provided is a system and method for adapting sentiment analysis to user profiles to reduce bias in customer or user generated content, specifically a system and method that discounts or adjusts bias in sentiment data based on the channel from which the content was received and/or the demographic of the user. The system includes a means to detect sentiment bias for any product, service, or company across multiple channels of customer data; a means to construct models to quantize bias by specific demographics and channels; and a means to adjust sentiment model output to reduce inflation by biased groups.

Abstract: Disclosed is a measure to collect and aggregate performance, sales, call center, custom KPI, and WA data in order to construct a model of correlations and causations between metrics to provide a means to measure associated costs of improving performance metrics for use in an interactive planning model. The interactive planning model allows for allocating budget across performance metrics, leveraging correlations and multiple cost models. The information is provided via a dashboard of IVA performance metrics and allows for comparison of real time IVA performance metrics to target metrics set by the IVA owner and incorporating KPIs provided by the WA owners.

Abstract: Disclosed is a system and method for detecting and addressing bias in training data prior to building language models based on the training data. Accordingly system and method, detect bias in training data for Intelligent Virtual Assistant (IVA) understanding and highlight any found. Suggestions for reducing or eliminating them may be provided This detection may be done for each model within the Natural Language Understanding (NLU) component. For example, the language model, as well as any sentiment or other metadata models used by the NLU, can introduce understanding bias. For each model deployed, training data is automatically analyzed for bias and corrections suggested.

Abstract: A system and method using blockchain for monitoring and tracking customer service representative actions are disclosed. In the system and method, customer service representative actions are encrypted using an encryption key specific to a company on whose behalf the customer service representative is acting. The customer service representative signs the encrypted action with a private key of a public/private key pair. The signed, encrypted action record is placed on the blockchain, which can later be accessed to review the actions of the customer service representative or actions of any customer service representative acting on behalf of the specific company.

Abstract: A system and method for updating computerized language models is provided that automatically adds or deletes terms from the language model to capture trending events or products, while maximizing computer efficiencies by deleting terms that are no longer trending and use of knowledge bases, machine learning model training and evaluation corpora, analysis tools and databases

Abstract: According to principles described herein, unsupervised statistical models, semi-supervised data models, and HITL methods are combined to create a text normalization system that is both robust and trainable with a minimum of human intervention.

Abstract: A method and computer product encoding the method is available for preparing a domain or subdomain specific glossary. The method included using probabilities, word context, common terminology and different terminology to identify domain and subdomain specific language and a related glossary updated according to the method.

Abstract: This disclosure describes techniques and architectures for evaluating conversations. In some instances, conversations with users, virtual assistants, and others may be analyzed to identify potential risks within a language model that is employed by the virtual assistants and other entities. The potential risks may be evaluated by administrators, users, systems, and others to identify potential issues with the language model that need to be addressed. This may allow the language model to be improved and enhance user experience with the virtual assistants and others that employ the language model.

Abstract: Systems and methods are provided to combine data from various internal data sources and external data sources to generate and maintain frequently asked questions and knowledge bases. By doing so, the collective knowledge of a company and customers' questions around it are aggregated and served in a way that increases the availability and relevancy of such knowledge for both self-help channels and computing agents (e.g., customer service representatives (CSRs)).

Abstract: Features, libraries, and techniques are provided herein for determining the kinds of relational language that are present. Applying audio, emojis, and sentiment shifts as features may be used to determine whether the customer is providing backstory, whether there is ranting, etc. Textual features may be considered, as well as audio features may be considered.

Abstract: To allow the human customer service agents to specialize in the instances where human service is preferred, but to scale to the volume of large call centers, systems and methods are provided in which human agents and intelligent virtual assistants (IVAs) co-handle a conversation with a customer. IVAs handle simple or moderate tasks, and human agents are used for those tasks that require or would benefit from human compassion or special handling. Instead of starting the conversation with an IVA and then escalating or passing control of the conversation to a human to complete, the IVAs and human agents work together on a conversation.

Abstract: To prevent intent classifiers from potentially choosing intents that are ineligible for the current input due to policies, dynamic intent classification systems and methods are provided that dynamically control the possible set of intents using environment variables (also referred to as external variables). Associations between environment variables and ineligible intents, referred to as culling rules, are used.

Abstract: Hybrid natural language understanding (NLU) systems and methods are provided that capitalize on the strengths of the rule-based models and the statistical models, lowering the cost of development and increasing the speed of construction, without sacrificing control and accuracy. Two models are used for intent recognition, one statistical and one rule-based. Both models define the same set of intents, but the rule-based model is devoid of any grammars or patterns initially. Each model may or may not be hierarchical in that it may be composed of a set of specialized models that are in a tree form or it may be just a singular model.

Abstract: This disclosure describes techniques and architectures for evaluating conversations. In some instances, conversations with users, virtual assistants, and others may be analyzed to identify potential risks within a language model that is employed by the virtual assistants and other entities. The potential risks may be evaluated by administrators, users, systems, and others to identify potential issues with the language model that need to be addressed. This may allow the language model to be improved and enhance user experience with the virtual assistants and others that employ the language model.

Abstract: As described herein, a system for expanding contractions in electronically stored text includes expanding contractions having only on expanded form. For remaining contractions, a grammar check is performed for all possible expanded forms to determine if an expanded form can be selected based on context and grammar rules. If an expanded form is not evident from the first two steps, all possible expanded forms of the remaining contractions are converted to a vector representation along with the original text. A Word Movers Distance (WMD) for each possible expansion is calculated using the vectors for each possible expansion and the original text. An expanded form is chosen without human intervention based on the grammar score alone or the WMD and the grammar score.

Abstract: Conversation user interfaces that are configured for virtual assistant interaction may include tasks to be completed that may have repetitious entry of the same or similar information. User preferences may be learned by the system and may be confirmed by the user prior to the learned preference being implemented. Learned preferences may be identified in near real-time on large collections of data for a large population of users. Further, the learned preferences may be based at least in part on previous conversations and actions between the system and the user as well as user-defined occurrence thresholds.

Abstract: Conversation user interfaces that are configured for virtual assistant interaction may include tasks to be completed that may have repetitious entry of the same or similar information. User preferences may be learned by the system and may be confirmed by the user prior to the learned preference being implemented. Learned preferences may be identified in near real-time on large collections of data for a large population of users. Further, the learned preferences may be based at least in part on previous conversations and actions between the system and the user as well as user-defined occurrence thresholds.