Abstract: Embodiments described herein provide a method of evaluating a natural language processing model. The method includes receiving an evaluation dataset that may include a plurality of unit tests, the unit tests having: an input context, and a first candidate and a second candidate that are generated in response to the input context, where the first test candidate is associated with a first quality notation, and the second candidate is associated with a second quality notation. The method includes determining, via a model, a first likelihood of generating the first candidate and a second likelihood of generating the second candidate in response to the input context. The method also includes determining whether the first likelihood being greater than the second likelihood. The method also includes determining whether the first model passed the unit test, where the first quality notation indicates a higher quality candidate and the second quality notation indicate a lower quality candidate.

Abstract: In view of the need for a conversational recommender system (CRS) in guiding purchasing processes of complex items, embodiments described herein provide a CRS system that creates a realistic purchase scenario and agent evaluation for fulfilling the recommendation objective. Specifically, the CRS system utilizes existing buying guides as a knowledge source for the recommendation model.

Abstract: Embodiments described herein provide systems and methods for training neural network based language models using human feedback. An existing (or generated) summary of a document is provided, and that summary may be used to generate a number of other summaries. A human annotator may reject the summary if there is any factuality issue with the summary. Summaries which are agreed to have no factuality problems are used as baseline summaries. Small atomic edits are made to the baseline summaries (e.g., replacing a single word or phrase) to create a group of summaries. Human annotators label each of these summaries as factual or not. The annotated summaries are used to train a summarization model and/or a factual detector model.

Abstract: A method of training a text simplification model is provided. A training dataset including a first set of original textual samples and original revision histories and a second set of simplified textual samples and simplified revision histories is received via a data interface. A training pair including an original textual sample and corresponding original revision history from the first set and a counterpart simplified textual sample and corresponding simplified revision history from the second set are identified. An alignment label for a first revision in the corresponding original revision history and a second revision in the corresponding simplified revision history are generated using a neural network-based alignment model from a score. A revision category label for each of the first revision and second revision is generated using a neural network-based classification model. A neural network-based text simplification model is trained based on the updated training dataset.

Abstract: Embodiments described herein provide system of a neural network based news reading tool. The system includes a communication interface receiving one or more text passages from data sources. The system also includes a memory storing a discord question neural model that generates a discord question according to the text passages. The system further includes processors executing processor-executable instructions to perform operations that include determining, by the discord question neural model, respective locations within the one or more text passages for inserting the discord question; receiving, via the communication interface from a client component, an indication that a user device is accessing a first text passage from a first data source; and sending, via the communication interface, the discord question, and a first location corresponding to the first text passage to the user device thereby causing the client component to generate and display a reading interface widget.

Abstract: Embodiments described herein provide a method and system for generating a reading interface for a user. The method includes receiving a first text passage from a first data source and a second text passage from a second data source. The method also includes generating a candidate question relating to contents of the first and the second text passages. The method further includes generating a first answer to the candidate question and a second answer to the candidate question. The method further includes determining that the candidate question qualifies as a discord question when the first answer and the second answer are both available and exhibit semantic diversity. The method further includes constructing a reading interface comprising a first component displaying the discord question and one or more user-engageable components each displaying the first or the second answer and linking to the first or the second data source, respectively.

Abstract: Embodiments described herein provide a method of evaluating a natural language processing model. The method includes receiving an evaluation dataset that may include a plurality of unit tests, the unit tests having: an input context, and a first candidate and a second candidate that are generated in response to the input context, where the first test candidate is associated with a first quality notation, and the second candidate is associated with a second quality notation. The method includes determining, via a model, a first likelihood of generating the first candidate and a second likelihood of generating the second candidate in response to the input context. The method also includes determining whether the first likelihood being greater than the second likelihood. The method also includes determining whether the first model passed the unit test, where the first quality notation indicates a higher quality candidate and the second quality notation indicate a lower quality candidate.

Abstract: An electrical connection device for electrically connecting an electronic component of the type having electrical connection points. The electrical connection device includes electrical connection members that each have a tip that can retract against an elastic means, a support carrying the electrical connection members, and a movable receptacle. The movable receptacle can house an electronic component and can bring the electronic component into a connection position in which the tips of the electrical connection members bear on the electrical connection points of the electronic component. In a preferred embodiment, the electrical connection members are removable from the device. Additionally, a testing apparatus that includes such an electrical connection device is provided.