Abstract: Existing research on cross-lingual retrieval cannot take good advantage of large-scale pretrained language models, such as multilingual BERT and XLM. The absence of cross-lingual passage-level relevance data for finetuning and the lack of query-document style pretraining are some of the key factors of this issue. Accordingly, embodiments of two novel retrieval-oriented pretraining tasks are presented herein to further pretrain cross-lingual language models for downstream retrieval tasks, such as cross-lingual ad-hoc retrieval (CUR) and cross-lingual question answering (CLQA). In one or more embodiments, distant supervision data was constructed from multilingual texts using section alignment to support retrieval-oriented language model pretraining. In one or more embodiments, directly finetuning language models on part of an evaluation collection was performed by making Transformers capable of accepting longer sequences.

Abstract: Deep neural network (DNN) models have been widely used for user-relevance content prediction. Presented herein is a new user-relevance framework, embodiments of which may be referred as Gating-Enhanced Multi-task Neural Networks (GemNN). In one or more, neural network-based multi-task learning model embodiments herein predict user engagement with content in a coarse-to-fine manner, which gradually reduces content candidates and allows parameter sharing from upstream tasks to downstream tasks to improve the training efficiency. Also, in one or more embodiments, a gating mechanism was introduced between embedding layers and multi-layer perceptions to learn feature interactions and control the information flow fed to MLP layers. Tested embodiments demonstrated considerable improvements over prior approaches.

Abstract: Learning disentangled representations is an important topic in machine learning for a wide range of applications. Disentangled latent variables represent interpretable semantic information and reflect separate factors of variation in data. Although generative models may learn latent representations and generate data samples as well, existing models may ignore the structural information among latent representations. Described in the present disclosure are embodiments to learn disentangled latent structural representations from data using decomposable variational auto-encoders, which simultaneously learn component representations and encode component relationships. Embodiments of a novel structural prior for latent representations are disclosed to capture interactions among different data components. Embodiments are applied to data segmentation and latent relation discovery among different data components. Experiments on several datasets demonstrate the utility of the present model embodiments.

Abstract: Presented herein are embodiments of an unsupervised cross-lingual sentiment classification model (which may be referred to as multi-view encoder-classifier (MVEC)) that leverages an unsupervised machine translation (UMT) system and a language discriminator. Unlike previous language model (LM)-based fine-tuning approaches that adjust parameters solely based on the classification error on training data, embodiments employ an encoder-decoder framework of an UMT as a regularization component on the shared network parameters. In one or more embodiments, the cross-lingual encoder of embodiments learns a shared representation, which is effective for both reconstructing input sentences of two languages and generating more representative views from the input for classification. Experiments on five language pairs verify that an MVEC embodiment significantly outperforms other models for 8/11 sentiment classification tasks.

Abstract: Described herein are embodiments for end-to-end reinforcement learning based coreference resolution models to directly optimize coreference evaluation metrics. Embodiments of a reinforced policy gradient model are disclosed to incorporate reward associated with a sequence of coreference linking actions. Furthermore, maximum entropy regularization may be used for adequate exploration to prevent a model embodiment from prematurely converging to a bad local optimum. Experiments on datasets compared with state-of-the-art methods verified the effectiveness of embodiments.

Abstract: Due to the high language use variability in real-life, manual construction of semantic resources to cover all synonyms is prohibitively expensive and may result in limited coverage. Described herein are systems and methods that automate the process of synonymy resource development, including both formal entities and noisy descriptions from end-users. Embodiments of a multi-task model with hierarchical task relationship are presented that learn more representative entity/term embeddings and apply them to synonym prediction. In model embodiments, a skip-gram word embedding model is extended by introducing an auxiliary task “neighboring word/term semantic type prediction” and hierarchically organize them based on the task complexity. In one or more embodiments, existing term-term synonymous knowledge is integrated into the word embedding learning framework.

Abstract: Efficient inner product search is important for many data ranking services, such as recommendation and Information Retrieval. Efficient retrieval via inner product dramatically influences the performance of such data searching and retrieval systems. To resolve deficiencies of prior approaches, embodiments of a new index graph construction approach, referred to generally as Norm Adjusted Proximity Graph (NAPG), for approximate Maximum Inner Product Search (MIPS) are presented. With adjusting factors estimated on sampled data, NAPG embodiments select more meaningful data points to connect with when constructing a graph-based index for inner product search. Extensive experiments verify that the improved graph-based index pushes the state-of-the-art of inner product search forward greatly, in the trade-off between search efficiency and effectiveness.

Abstract: An Internet of Vehicles service activation method comprises: receiving first vehicle information of a vehicle to be activated, which is sent by a mobile terminal; when the received first vehicle information is comprised in a vehicle information database, sending an activation request message to the mobile terminal and starting a countdown of a preset time period; when a vehicle-started event sent by a vehicle-mounted terminal is received before the end of the countdown, extracting, from the vehicle-started event, second vehicle information corresponding to the vehicle-mounted terminal; and when the first vehicle information is the same as the second vehicle information, activating an Internet of Vehicles service of the vehicle to be activated.

Abstract: Current pretrained vision-language models for cross-modal retrieval tasks in English depend upon on the availability of many annotated image-caption datasets for pretraining to have English text. However, the texts are not necessarily in English. Although machine translation (MT) tools may be used to translate text to English, the performance largely relies on MT's quality and may suffer from high latency problems in real-world applications. Embodiments herein address these problems by learning cross-lingual cross-modal representations for matching images and their relevant captions in multiple languages. Embodiments seamlessly combine cross-lingual pretraining objectives and cross-modal pretraining objectives in a unified framework to learn image and text in a joint embedding space from available English image-caption data, monolingual corpus, and parallel corpus. Embodiments are shown to achieve state-of-the-art performance in retrieval tasks on multimodal multilingual image caption datasets.

Abstract: Described herein are embodiments for systems and methods to incorporate skip-gram convolution to extract non-consecutive local n-gram patterns for comprehensive information for varying text expressions. In one or more embodiments, one or more recurrent neural networks are employed to extract long-range features from localized level to sequential and global level via a chain-like architecture. Comprehensive experiments on large-scale datasets widely used for the text classification task were conducted to demonstrate the effectiveness of the presented deep skip-gram network embodiments. Performance evaluation on various datasets demonstrates that embodiments of the skip-gram network are powerful for general text classification task set. The skip-gram models are robust and may be generalized well on different datasets, even without tuning the hyper-parameters for specific dataset.

Abstract: Existing research on cross-lingual retrieval cannot take good advantage of large-scale pretrained language models, such as multilingual BERT and XLM. The absence of cross-lingual passage-level relevance data for finetuning and the lack of query-document style pretraining are some of the key factors of this issue. Accordingly, embodiments of two novel retrieval-oriented pretraining tasks are presented herein to further pretrain cross-lingual language models for downstream retrieval tasks, such as cross-lingual ad-hoc retrieval (CUR) and cross-lingual question answering (CLQA). In one or more embodiments, distant supervision data was constructed from multilingual texts using section alignment to support retrieval-oriented language model pretraining. In one or more embodiments, directly finetuning language models on part of an evaluation collection was performed by making Transformers capable of accepting longer sequences.

Abstract: Previous neural network models that perform named entity recognition (NER) typically treat the input sentences as a linear sequence of words but ignore rich structural information, such as the coreference relations among non-adjacent words, phrases, or entities. Presented herein are novel approaches to learn coreference-aware word representations for the NER task. In one or more embodiments, a “CNN-BiLSTM-CRF” neural architecture is modified to include a coreference layer component on top of the BiLSTM layer to incorporate coreferential relations. Also, in one or more embodiments, a coreference regularization is added during training to ensure that the coreferential entities share similar representations and consistent predictions within the same coreference cluster. A model embodiment achieved new state-of-the-art performance when tested.

Abstract: Learning disentangled representations is an important topic in machine learning for a wide range of applications. Disentangled latent variables represent interpretable semantic information and reflect separate factors of variation in data. Although generative models may learn latent representations and generate data samples as well, existing models may ignore the structural information among latent representations. Described in the present disclosure are embodiments to learn disentangled latent structural representations from data using decomposable variational auto-encoders, which simultaneously learn component representation and encodes component relationships. Embodiments of a novel structural prior for latent representations are disclosed to capture interactions among different data components. Embodiments are applied to data segmentation and latent relation discovery among different data components. Experiments on several datasets demonstrate the utility of the present model embodiments.

Abstract: Presented herein are embodiments of an unsupervised cross-lingual sentiment classification model (which may be referred to as multi-view encoder-classifier (MVEC)) that leverages an unsupervised machine translation (UMT) system and a language discriminator. Unlike previous language model (LM)-based fine-tuning approaches that adjust parameters solely based on the classification error on training data, embodiments employ an encoder-decoder framework of an UMT as a regularization component on the shared network parameters. In one or more embodiments, the cross-lingual encoder of embodiments learns a shared representation, which is effective for both reconstructing input sentences of two languages and generating more representative views from the input for classification. Experiments on five language pairs verify that an MVEC embodiment significantly outperforms other models for 8/11 sentiment classification tasks.

Abstract: Presented are systems and methods that allow healthcare providers and governments to infer demand for healthcare resources to ensure effective and timely healthcare services to patients by reducing healthcare supply shortages, emergencies, and healthcare costs. In embodiments, this is accomplished by gathering data from a number of sources to generate labeled records from which entity features and relationships between entities are extracted, correlates, and/or combined with other external healthcare data. In embodiments, this information is used to train a model that predicts healthcare resource demands given a set of input conditions or factors.

Abstract: Systems and methods are disclosed for question generation to obtain more related medical information based on observed symptoms from a patient. In embodiments, possible diseases associated with the observed symptoms are generated by querying a knowledge graph. In embodiments, candidate symptoms associated with the possible diseases are also identified and are combined with the observed symptoms to obtain combined symptom sets. In embodiments, discriminative scores for the candidate symptom sets are determined and candidate symptoms with top discriminative scores are selected. In embodiments, these selected candidate symptoms may be checked for conflicts with observed symptoms and removed from further consideration if a conflict exists. In embodiments, one or more questions may be generated based on the remaining selected candidate systems to aid in collecting information about the patient. In embodiments, the process may be repeated with the updated observed symptoms.

Abstract: Described herein are embodiments for end-to-end reinforcement learning based coreference resolution models to directly optimize coreference evaluation metrics. Embodiments of a reinforced policy gradient model are disclosed to incorporate reward associated with a sequence of coreference linking actions. Furthermore, maximum entropy regularization may be used for adequate exploration to prevent a model embodiment from prematurely converging to a bad local optimum. Experiments on datasets compared with state-of-the-art methods verified the effectiveness of embodiments.

Abstract: Causal performance analysis for store merchandising may be provided. A clustering technique may be performed based on target store location data and existing store data. Based on the clustering technique, a peer selection group is determined comprising a group of stores determined to have similar attributes to the target store location. Sales distortions for a plurality of divisions associated with the group of stores in the peer selection group may be determined. A distortion matrix may be generated comprising a ranked list of the plurality of divisions. A merchandise mix recommendation for the target store location may be presented via a user interface device.

Abstract: Previous neural network models that perform named entity recognition (NER) typically treat the input sentences as a linear sequence of words but ignore rich structural information, such as the coreference relations among non-adjacent words, phrases, or entities. Presented herein are novel approaches to learn coreference-aware word representations for the NER task. In one or more embodiments, a “CNN-BiLSTM-CRF” neural architecture is modified to include a coreference layer component on top of the BiLSTM layer to incorporate coreferential relations. Also, in one or more embodiments, a coreference regularization is added during training to ensure that the coreferential entities share similar representations and consistent predictions within the same coreference cluster. A model embodiment achieved new state-of-the-art performance when tested.

Abstract: Due to the high language use variability in real-life, manual construction of semantic resources to cover all synonyms is prohibitively expensive and may result in limited coverage. Described herein are systems and methods that automate the process of synonymy resource development, including both formal entities and noisy descriptions from end-users. Embodiments of a multi-task model with hierarchical task relationship are presented that learn more representative entity/term embeddings and apply them to synonym prediction. In model embodiments, a skip-gram word embedding model is extended by introducing an auxiliary task “neighboring word/term semantic type prediction” and hierarchically organize them based on the task complexity. In one or more embodiments, existing term-term synonymous knowledge is integrated into the word embedding learning framework.