Grading Of Response Form Patents (Class 434/353)
  • Patent number: 11310381
    Abstract: An image forming apparatus includes a reader, a wait part, and a processor. The reader reads an image from a first sheet. In the wait part, the first sheet is made to wait. The processor is configured to make the first sheet wait in the wait part after the reading performed by the reader, and, after the first sheet is made to wait in the wait part, output the first sheet and a second sheet continuously. The second sheet records first information determined on the basis of an evaluation result obtained from evaluation of the first sheet.
    Type: Grant
    Filed: September 15, 2020
    Date of Patent: April 19, 2022
    Assignee: FUJIFILM Business Innovation Corp.
    Inventor: Taishi Asumi
  • Patent number: 11270526
    Abstract: A teaching assistance method and a teaching assistance system using said method, the teaching assistance method comprising implementing behaviour detection of students in classroom images by means of using a trained depth tensor column network model, thus providing higher image recognition precision and reducing the hardware requirements for algorithms, and being able to be used on an embedded device, reducing the usage costs of the teaching assistance method; in addition, a teaching assistance system using said teaching assistance method has the same advantages.
    Type: Grant
    Filed: August 7, 2017
    Date of Patent: March 8, 2022
    Assignees: SHENZHEN INSTITUTES OF ADVANCED TECHNOLOGY CHINESE ACADEMY OF SCIENCES, SHENZHEN SIBIKU TECHNOLOGY CO., LTD
    Inventors: Shuqiang Wang, Yongcan Wang, Yue Yang, Yanyan Shen, Minghui Hu
  • Patent number: 11227230
    Abstract: Mechanisms are provided for modifying response of cognitive systems to user requests based on a determined knowledge level of the user and knowledge level of the concepts referenced in the cognitive system responses. The mechanisms receive a response to a user submitted request received from a client computing system and determine a first knowledge level of the user for a domain of the user submitted request. The mechanisms determine a second knowledge level associated with concepts referenced in the response and whether the first knowledge level is different from the second knowledge level. The mechanism, in response to the first knowledge level being different than the second knowledge level, modify content of the response to include concepts associated with the first knowledge level, to thereby generate a modified response. The modified response is then output to the client computing system associated with the user.
    Type: Grant
    Filed: March 17, 2017
    Date of Patent: January 18, 2022
    Assignee: International Business Machines Corporation
    Inventors: Rhonda L. Childress, Darren L. Lawless, Crystal N. Moody, Daniel S. Riley, Michael J. Spisak
  • Patent number: 11210306
    Abstract: A method of obtaining a response to a query inputted by a user, the method comprising: receiving a user inputted query; representing the user inputted query as a sequence of embedding vectors using a first model; encoding the sequence of embedding vectors to produce a context vector using a second model; retrieving responses with associated response vectors; scoring response vectors against the context vector, wherein the scoring is a measure of the similarity between the context vector and a response vector; and outputting the responses with the closest response vectors, wherein the first model is configured to segment a user inputted query into a sequence of units from a vocabulary of units and represent each unit in the sequence as an embedding vector, wherein at least one of the units in the vocabulary is an incomplete word, and wherein the first model comprises parameters that are stored using eight bits per parameter; and wherein the second model has been trained using corresponding queries and response
    Type: Grant
    Filed: December 30, 2020
    Date of Patent: December 28, 2021
    Assignee: PolyAI Limited
    Inventors: Matthew Steedman Henderson, Pei-Hao Su, Tsung-Hsien Wen, Inigo Casanueva Perez, Nikola Mrksic, Samuel John Coope, Ivan Vulic
  • Patent number: 11195431
    Abstract: Systems and methods for defining and using an optimal burst boundary threshold to assess the reliability of a manual/automatic writing score are presented. Keystroke data, including inter-key interval data, such as inter-word interval data, may be gathered from writings. Clustering analyses may be performed on the inter-key interval data to determine an optimal number of bursts for the writings. An optimal burst boundary may be determined from the optimal number of bursts. Other burst-related measures and statistics, including the average and maximum burst lengths, may be determined from the writings based on the optimal burst boundary threshold. A score may be received for each of the writings. A validation indication metric may be generated for each of the writings based on the received score and the optimal burst boundary threshold. The resulting measures and statistics may be used or applied in different ways and provide personalized feedback as learning analytics.
    Type: Grant
    Filed: March 12, 2019
    Date of Patent: December 7, 2021
    Assignee: Educational Testing Service
    Inventors: Mo Zhang, Jiangang Hao, Paul Deane, Chen Li
  • Patent number: 11176163
    Abstract: Embodiments of the present invention are directed to a two-way data transfer and synchronization system. The data transfer and synchronization system includes a provider system and a customer system. The provider system includes a web service hosted by a web server connected to a computer network and a first database. The customer system includes a client application, connected to the computer network and a second database (e.g., SIS database). The provider system cannot read data from the second database. The client application is configured to send requests to the web service and to receive HTTP responses from the web service. The responses from the web service are executed by the client application so that changes made to the second database are also made to the first database, and vice versa.
    Type: Grant
    Filed: September 26, 2017
    Date of Patent: November 16, 2021
    Assignee: COLLEGENET, INC.
    Inventors: Tuan Nguyen Thanh Pham, Jeffery Scott Sanders, Herts Chen, Mikhail Anatolevich Krughkov, Devin Christopher Sills
  • Patent number: 11158203
    Abstract: A combination of knowledge representation and retrieval, assessments and personalization for a tutoring platform that provides an automated phased word exposition method for vocabulary learning and enable learners to learn words (or any learning objective) with sufficient proficiency and without excessive repetition. The phased learning objective method implements a moving average model that models the learner across multiple dimensions (e.g. reading, listening, writing, speaking) such that the platform is confident of the learner's understanding, while using minimal assessment doses, and such that the understanding transfers across experiences. The method implemented provides for multiple learning and assessment activities for a learning objective and leverages the same learner model for each learning dimension. Each word of a word set passes through learning and assessment-only phases for statistically significant evidence of learning.
    Type: Grant
    Filed: February 14, 2018
    Date of Patent: October 26, 2021
    Assignees: International Business Machines Corporation, Sesame Workshop
    Inventors: Ravi Kokku, Tamer E. Abuelsaad, Prasenjit Dey, Satyanarayana V. Nitta, Miles Ludwig, Emily Reardon, James Gray
  • Patent number: 11151895
    Abstract: Accordingly, embodiments of the present invention advantageously account for item discrimination in a single-parameter logistic model used for measuring a test-taker's ability and an item's difficulty. Accounting for item discrimination improves the reliability of a test without increasing the number of test items. To account for item discrimination in a single-parameter logistic model, this invention uses the correlation between item response (correct or incorrect) and total test score or other measure of test-taker ability to obtain a Bayesian estimate of the correct-response probability (between zero and one). This correlation is a measure of item discrimination. The numerator in the formula for this correlation contains the difference between the average test score of test-takers who got the item right and the average test score of test-takers who got the item wrong.
    Type: Grant
    Filed: February 4, 2019
    Date of Patent: October 19, 2021
    Inventor: Ronald Weitzman
  • Patent number: 11068793
    Abstract: An inference device disclosed herein includes an identifying unit, and an inferring unit. The identifying unit identifies a set of a predetermined phenomenon, an effect having a causal relation with the predetermined phenomenon, and a polarity of the effect, from concept information in which a phenomenon, an effect having a causal relation with the phenomenon, a polarity to be an indicator of advantages and disadvantages caused by the effect to a user. The inferring unit infers an effect caused when the predetermined phenomenon occurs, and advantages and disadvantages caused by the effect based on the set identified by the identifying unit.
    Type: Grant
    Filed: August 30, 2017
    Date of Patent: July 20, 2021
    Assignee: YAHOO JAPAN CORPORATION
    Inventor: Ikuo Kitagishi
  • Patent number: 11049409
    Abstract: Systems and methods are provided for automatically scoring a response and statistically revaluating whether it can be considered as aberrant. In one embodiment, a constructed response is evaluated via a pre-screening stage and a post-hoc screening stage. The pre-screening stage attempts to determine whether the constructed response is aberrant based on a variety of aberration metrics and criteria. If the constructed response is deemed not to be aberrant, then the post-hoc screening stage attempts to predict a discrepancy between what score an automated scoring system would assigned and what score a human rater would assign to the response. If the discrepancy is sufficiently low, then the constructed response may be scored by an automated scoring engine. On the other hand, if the constructed response failed to pass either of the two stages, then a flag may be raised to indicate that additional human review may be needed.
    Type: Grant
    Filed: December 18, 2015
    Date of Patent: June 29, 2021
    Assignee: Educational Testing Service
    Inventors: Mo Zhang, Jing Chen, Andre Alexander Rupp, David Michael Williamson
  • Patent number: 10957212
    Abstract: Methods, computer program products, and systems are presented. The methods include, for instance: obtaining sample essays, sample annotations corresponding to the sample essays, and a subject content for building a subject domain comprehension model and an essay annotation model, by use of one or more neural network. The nodes of the subject domain comprehension model and the essay annotation model are interconnected based on respective relevancies for automatically annotating student works according to a standard of review corresponding to submitting students.
    Type: Grant
    Filed: April 4, 2018
    Date of Patent: March 23, 2021
    Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
    Inventors: Laurence Plant, James D. Cleaver, Stefan Harrer
  • Patent number: 10885803
    Abstract: The present disclosure features systems and methods for analyzing student learning and calibrating the difficulty of questions on a test or examination. In one embodiment, a method for analyzing the learning of a student includes administering, by an assessment agent, a task to a student, the task comprising a question having an associated difficulty. The assessment agent receives a response to the question from the student and evaluates the response to generate an observable, the observable comprising information related to the response. A posterior determination of the student's ability is then calculated by incorporating the observable into an ability model associated with the student, and the posterior determination of ability may be compared with the difficulty of the question, a skill acquisition probability, or other measure. The student's response, or a plurality of responses from students within a cohort, may be used to determine the difficulty of each question.
    Type: Grant
    Filed: January 23, 2015
    Date of Patent: January 5, 2021
    Assignee: Massachusetts Institute of Technology
    Inventors: Emery Neal Brown, Demba Ba, Anne Caroline Smith
  • Patent number: 10878711
    Abstract: The systems, methods and associated devices electronically collect, report and generate normalized educational outcome summaries of multiple different educational inputs, including didactic, experiential and problem solving events and/or assessments.
    Type: Grant
    Filed: October 26, 2017
    Date of Patent: December 29, 2020
    Assignee: East Carolina University
    Inventor: Robert Todd Watkins, Jr.
  • Patent number: 10878197
    Abstract: A machine-learning component of a conversational user-interface system retrieves records of human question-and-answer sessions from heterogenous sources. A sequence of image-processing operations converts each source into a set of grayscale images. Each image is segmented into disjoint sections as a function of the textures, edges, and contours that make up the image's content. Each section is tagged with a vector of parametric values that identify characteristics of the section's content from which may be inferred semantic meaning. A cognitive function intelligently analyzes the vectors to classify each section as containing a question, an answer to a question, non-textual media, or other types of content. Another cognitive function merges and organizes the sections into question-answer pairs and the vectors associated with each pair are stored in a corpus that is submitted to the self-learning user interface during a machine-learning training session.
    Type: Grant
    Filed: November 27, 2018
    Date of Patent: December 29, 2020
    Assignee: International Business Machines Corporation
    Inventors: Kun Yan Yin, He Li, Yuan Lin Yang, Jian Hui Bj Chen, Shun Sp Wen, Zhong Fang Yuan
  • Patent number: 10762377
    Abstract: A method processes scanned floating forms by comparing topological structures of an empty form and a corresponding filled form. The topological structure of each form includes vertical and horizontal ordered sequences of text phrases in the form, which describe directional relationships among text phrases but not their distances. A partial structure alignment method is used in the comparison, where insertions of text in the topological structure of the filled form relative to that of the empty form is not penalized, but deletions and substitutions are penalized. Based on this comparison, some text phrases in the filled form are matched to the those in the empty form, and unmatched text in the filled form is deemed user-filled data. The method further associates user-filled data with fields of the form based on a settings file of the empty form.
    Type: Grant
    Filed: December 29, 2018
    Date of Patent: September 1, 2020
    Assignee: KONICA MINOLTA LABORATORY U.S.A., INC.
    Inventor: Maral Mesmakhosroshahi
  • Patent number: 10741092
    Abstract: A computer implemented method for evaluating examination responses includes receiving, with an examination evaluation server, an examination response, tokenizing the examination response into one or more n-word segments, extracting, from the response, a set of construct-relevant features indicating one or more scoring metrics, generating a set of scoring rules, and generating a score estimate as a function of one or more scoring rules from the set of scoring rules and one or more of the scoring metrics.
    Type: Grant
    Filed: June 9, 2017
    Date of Patent: August 11, 2020
    Assignee: ACT, INC.
    Inventors: Richard Meisner, Bradley Bolender, Daniel Shaw
  • Patent number: 10741093
    Abstract: A computer implemented method for identifying and removing similar examination items in a digital examination includes obtaining, with an examination evaluation server, a digital examination with a plurality of examination items, tokenizing a first examination item of the plurality of examination items into a first set of n-grams, tokenizing a second examination item of the plurality of examination items into a second set of n-grams, generating a first item vector from the first set of n-grams, generating a second item vector from the second set of n-grams, generating multiple item congruence values for corresponding similarity dimensions by comparing the first item vector with the second item vector, and identifying that the first and second examination items are a potential enemy item pair if one or more of the item congruence values exceed corresponding threshold values.
    Type: Grant
    Filed: July 19, 2019
    Date of Patent: August 11, 2020
    Assignee: ACT, INC.
    Inventor: Richard Meisner
  • Patent number: 10665122
    Abstract: A computer implemented method of examination scoring includes receiving, with an examination evaluation server, a language corpus with a plurality of corpus words, deriving, for each corpus word, a corpus sematic vector within a multi-dimensional semantic vector space, receiving first and second examination responses, tokenizing each examination response into a plurality of response words, identifying a set of matching response words from the plurality of response words from each examination response, identifying a set of semantic vectors corresponding to the matching response words for each examination response, and comparing the semantic vectors from the first examination response with the semantic vectors from the second examination response to derive a response semantic similarity value for use as an input for automated scoring.
    Type: Grant
    Filed: June 9, 2017
    Date of Patent: May 26, 2020
    Assignee: ACT, INC.
    Inventor: Richard Meisner
  • Patent number: 10646155
    Abstract: The invention provides a method of generating an adaptive partial report for an observer with an apparatus comprising a display, a user interface, and a processor. The apparatus can be a computer system or an electronic device, for example. The method includes the processor characterizing an iconic memory decay function for the observer. The characterization includes determining a prior for a plurality of parameters. The method further includes the processor determining a first stimulus for a first trial based on the prior for the plurality of parameters, the display generating the stimulus for viewing by the observer, the user interface receiving input for the first trial and in response to the stimulus, the processor revising respective parameter values for the parameters based on the received input, and the processor determining a new stimulus for a next trial based on the revised parameter values.
    Type: Grant
    Filed: April 14, 2015
    Date of Patent: May 12, 2020
    Assignees: OHIO STATE INNOVATIVE FOUNDATION, ADAPTIVE SENSORY TECHNOLOGY
    Inventors: Zhong-Lin Lu, Jongsoo Baek, Luis A. Lesmes
  • Patent number: 10529245
    Abstract: A method for test item selection is provided that includes a computer implemented test battery having at least two or more sections with a plurality of test items. An ability estimate is calculated from an earlier section(s) of the at least two or more sections and an initial item and subsequent items for a subsequent section are selected from the plurality of test items based upon the ability estimate(s) from the earlier section(s). Use of a more informative initial ability estimate in the item selection process can improve interim ability estimation accuracy and item selection efficiency while keeping item exposure and item usage rates at acceptable levels.
    Type: Grant
    Filed: December 5, 2014
    Date of Patent: January 7, 2020
    Assignee: ACT, INC.
    Inventors: Lisa Gawlick, Changhui Zhang, Nancy Petersen, Lingyun Gao, Wugen Dai
  • Patent number: 10460618
    Abstract: A non-transitory computer-readable recording medium stores a scoring rule application target specification program that causes a computer to execute a process including: receiving specification of one or more areas in an image; and storing the one or more areas in a storage in a manner associated with a question number in order of being received.
    Type: Grant
    Filed: May 10, 2016
    Date of Patent: October 29, 2019
    Assignee: FUJITSU LIMITED
    Inventors: Shinichi Tashiro, Masanori Isobe, Shintaro Kida, Aiko Koyago
  • Patent number: 10446044
    Abstract: A computer-implemented method of calibrating an assessment model for assessing responses includes accessing a plurality of training responses with a processing system for training an assessment model. The processing system analyzes the plurality of training responses to derive values of multiple features of the training responses. The processing system trains the assessment model based on the values of the multiple features of the training responses and a portfolio score for each individual associated with the plurality of training responses utilized in the training. The portfolio score for each individual corresponds to a measure of proficiency based on multiple writing samples constructed by the individual. The processing system determines, based on said training, a weight for each of the multiple features. The processing system calibrates the assessment model to include the weights for at least some of the features such that the assessment model is configured to generate scores for responses.
    Type: Grant
    Filed: June 17, 2014
    Date of Patent: October 15, 2019
    Assignee: Educational Testing Service
    Inventors: Paul Douglas Deane, Brent Bridgeman, Chaitanya Ramineni
  • Patent number: 10373512
    Abstract: Mechanisms for automatically grading a large number of solutions provided by learners in response to an open response mathematical question. Each solution is mapped to a corresponding feature vector based on the mathematical expressions occurring in the solution. The feature vectors are clustered using a conventional clustering method, or alternatively, using a presently-disclosed Bayesian nonparametric clustering method. A representative solution is selected from each solution cluster. An instructor supplies a grade for each of the representative solutions. Grades for the remaining solutions are automatically generated based on their cluster membership and the instructor supplied grades. The Bayesian method may also automatically identify the location of an error in a given solution. The error location may be supplied to the learner as feedback. The error location may also be used to extract information from correct solutions. The extracted information may be supplied to a learner as a solution hint.
    Type: Grant
    Filed: December 11, 2015
    Date of Patent: August 6, 2019
    Assignee: William Marsh Rice University
    Inventors: Shiting Lan, Divyanshu Vats, Andrew E. Waters, Richard G. Baraniuk
  • Patent number: 10335079
    Abstract: Methods, systems, and apparatus, including medium-encoded computer program products, for analyzing data include: receiving data including responses, and lack thereof, for items of a cognitive test including multiple item-recall trials; processing the data using a stochastic model of a cognitive process, in which a conditional probability distribution of future states of the cognitive process depend upon a present state; and encoding a result of the processing on a non-transitory computer-readable medium for use in an assessment related to cognition; wherein the processing using the stochastic model includes representing recall or recognition of an item in the multiple item-recall trials using distinct cognitive states; and wherein the processing using the stochastic model includes adjusting separate memory storage and retrieval parameters for each of the distinct cognitive states in the modeled cognitive process to account for position of the items in each respective trial of the multiple item-recall trials.
    Type: Grant
    Filed: February 10, 2015
    Date of Patent: July 2, 2019
    Assignee: Medical Care Corporation
    Inventors: Gregory E. Alexander, William Rodman Shankle
  • Patent number: 10339824
    Abstract: This disclosure describes a system that delivers content (e.g. multimedia video), and analyzes and aggregates feedback of a user viewing the content. The user may interact with the multimedia video in the form of answering questions which are dynamically placed between video segments. The system may determine the response type of the user for a given question. The system may handle both positive and negative responses (e.g. correct and incorrect responses to questions). The system may help the user navigate the segments of the video.
    Type: Grant
    Filed: November 21, 2016
    Date of Patent: July 2, 2019
    Assignee: KONINKLIJKE PHILIPS N.V.
    Inventors: Anand Srinivasan Srinivasan Natesan, Rithesh Sreenivasan, Rajendra Singh Sisodia
  • Patent number: 10339826
    Abstract: Systems and methods are provided for automatically scoring essay responses to a prompt using a scoring model. A relevant word corpus and an irrelevant word corpus are accessed. A scoring model is generated by, for each of a plurality of words in the relevant word corpus, determining a topic signature score based on a number of appearances of that word in the relevant word corpus and a number of appearances of that word in the irrelevant word corpus. For each of a plurality of words in an essay response, a topic signature score is determined for that word. A score for the essay response is determined based on the identified topic signature scores.
    Type: Grant
    Filed: October 12, 2016
    Date of Patent: July 2, 2019
    Assignee: Educational Testing Service
    Inventors: Swapna Somasundaran, Martin Chodorow, Jill Burstein
  • Patent number: 10025799
    Abstract: A method for managing and rendering one or more information nodes relative to a current focus. The method receives one or more principal topics from an analysis of the information nodes. The method determines a topic vector from a similarity of a first information node to each of the principal topics and a map from the topic vector to a storage location of the first information node. The method determines one or more current topics from a second information node that has been recently accessed by a user and a current focus vector from a similarity of the one or more current topics to the one or more principal topics. The method renders the current focus vector and the first information node according to a rendering algorithm that is applied to the current focus vector and the first information node.
    Type: Grant
    Filed: July 22, 2015
    Date of Patent: July 17, 2018
    Assignee: International Business Machines Corporation
    Inventors: Brian P. Gaucher, Jeffrey O. Kephart, Jonathan Lenchner, David O. S. Melville
  • Patent number: 9971976
    Abstract: Selecting candidates from a set of candidates by receiving a set of input parameters about a single aspect of the candidates and applying multiple scoring methods to those input parameters to compute a candidate score via each scoring method. A subset of zero or more candidates is then selected from the set of candidates by applying one or more selection criteria to the set of scores. There is not necessarily a value for every input parameter of every candidate.
    Type: Grant
    Filed: September 23, 2014
    Date of Patent: May 15, 2018
    Assignee: International Business Machines Corporation
    Inventors: Wesley M. Gifford, Pavankumar Murali, Anshul Sheopuri
  • Patent number: 9959776
    Abstract: Systems and methods are provided for measuring a user's English language proficiency. A constructed response generated by a user is received, the constructed response being based on a picture. The constructed response is processed to determine a first numerical measure indicative of a presence of one or more grammar errors in the constructed response. The constructed response is processed to determine a second numerical measure indicative of a degree to which the constructed response describes a subject matter of the picture. The constructed response is processed to determine a third numerical measure indicative of a degree of awkward word usage in the constructed response. A model is applied to the first, second, and third numerical measures to determine a score for the constructed response indicative of the user's English language proficiency. The model includes first, second, and third variables with associated first, second, and third weighting factors, respectively.
    Type: Grant
    Filed: October 31, 2017
    Date of Patent: May 1, 2018
    Assignee: Educational Testing Service
    Inventors: Swapna Somasundaran, Martin Chodorow, Joel Tetreault
  • Patent number: 9928754
    Abstract: Computer-implemented systems and methods are provided for automatically generating recitation items. For example, a computer performing the recitation item generation can receive one or more text sets that each includes one or more texts. The computer can determine a value for each text set using one or more metrics, such as a vocabulary difficulty metric, a syntactic complexity metric, a phoneme distribution metric, a phonetic difficulty metric, and a prosody distribution metric. Then the computer can select a final text set based on the value associated with each text set. The selected final text set can be used as the recitation items for a speaking assessment test.
    Type: Grant
    Filed: March 17, 2014
    Date of Patent: March 27, 2018
    Assignee: Educational Testing Service
    Inventors: Su-Youn Yoon, Lei Chen, Keelan Evanini, Klaus Zechner
  • Patent number: 9911161
    Abstract: A method and survey server for analyzing social media content based on survey participation data related to a website. The survey server collects survey participation data comprising text and a metric. The survey participation data correspond to survey information received from users of a plurality of user devices in relation to a visit of the website. The survey server analyzes the text of the survey participation data, to generate a correlation between words or groups of words present in the text and the metric. Social media content comprising text and related to a visit of the website is also collected. The text of the social media content is analyzed, to identify the words or groups of words. The social media content comprising the words or groups of words is associated with the metric. The analysis of the text of the survey participation data may comprise performing a semantic categorization.
    Type: Grant
    Filed: December 19, 2014
    Date of Patent: March 6, 2018
    Assignee: iPerceptions Inc.
    Inventors: Lane Cochrane, Matthew Butler, Naoko Tomioka
  • Patent number: 9886869
    Abstract: A method/apparatus/system for educational intervention based on a response metric is disclosed. The notice is generated in response to the collection of user and question data, the sending of questions, the receipt of answers, the evaluation of the correctness of the answers, the generation of a response metric, the comparison of the response metric to a threshold, and the generation of the report or notice. The response metric can be reflect the scatter, randomness, and/or slope of student provided answer data, and can be a fractal dimension.
    Type: Grant
    Filed: September 23, 2016
    Date of Patent: February 6, 2018
    Assignee: PEARSON EDUCATION, INC.
    Inventors: William M. Galen, Rasil Warnakulasooriya
  • Patent number: 9824604
    Abstract: A method, a system, and a computer program product for creating an assessment model are disclosed. Marked regions in an electronic document are identified. Thereafter, mark-ups within the marked regions are identified. Based on the identified marked regions and mark-ups the assessment model is created.
    Type: Grant
    Filed: September 4, 2012
    Date of Patent: November 21, 2017
    Assignee: Conduent Business Services, LLC
    Inventor: Dennis L Venable
  • Patent number: 9767707
    Abstract: An optimization engine builds a set of test forms, called a pallet for use in a qualification test setting. The pallet is generated using a optimization engine programmed with constraints and goals for each test form in the pallet and for the pallet in general. A test information floor is set at the cut point of the test which causes test items to be focused at the area of the test where the pass/fail decision is made. The test items may be separated into families by assigning test items selected for a test form in each test generation cycle to one of several families. Because each family has unique test items compared to other families, a person re-taking a qualification test from a particular pallet may be given a test form from another family, ensuring that no item from the first administration will appear on the subsequent test.
    Type: Grant
    Filed: February 22, 2013
    Date of Patent: September 19, 2017
    Assignee: NATIONAL ASSOC. OF BOARDS OF PHARMACY
    Inventor: Paul Edward Jones
  • Patent number: 9715657
    Abstract: A medical diagnosis support apparatus includes a training data obtaining unit that obtains training data, an inference means candidate creating unit that creates a plurality of inference means candidates based on the training data, an inference performance evaluation unit that evaluates the performance of the plurality of inference means candidates based on the training data, an information validity evaluation unit that evaluates the validity of information presented by each of the plurality of inference means candidates based on the training data, and an inference means selection unit that selects an inference means from the plurality of inference means candidates based on the performance of the plurality of inference means candidates and the validity of the information presented by each of the plurality of inference means candidates.
    Type: Grant
    Filed: January 6, 2014
    Date of Patent: July 25, 2017
    Assignee: CANON KABUSHIKI KAISHA
    Inventor: Masami Kawagishi
  • Patent number: 9679256
    Abstract: Embodiments herein provide automated assessment of examination scripts, such as English for Speakers of Other Languages (ESOL) examination scripts, written in response to prompts eliciting free text answers. In an embodiment, the task may be defined as discriminative preference ranking. Further, a system employing such methodology may be trained and tested on a corpus of manually-graded scripts. Embodiments herein, unlike extant solutions, are relatively prompt-insensitive and resistant to subversion, even if the operating principles are known. Embodiments may also detect scripts which are linguistically good but non-responsive to prompts such as memorized responses.
    Type: Grant
    Filed: October 6, 2010
    Date of Patent: June 13, 2017
    Assignee: THE CHANCELLOR, MASTERS AND SCHOLARS OF THE UNIVERSITY OF CAMBRIDGE
    Inventors: Ted Briscoe, Ben Medlock, Oeistein Andersen
  • Patent number: 9665565
    Abstract: A semantic similarity evaluation method includes performing word vectorization processing separately on words in a first sentence and a word in a second sentence to obtain a first word vector and a second word vector; performing, in a preset word vector compression order, compression coding processing on the first word vector according to a first compression coding parameter to obtain a first statement vector; performing, in the preset word vector compression order, compression coding processing on the second word vector according to a second compression coding parameter to obtain a second statement vector; and determining a vector distance between the first statement vector and the second statement vector, and evaluating a semantic similarity between the first sentence and the second sentence according to the vector distance. The method is used to evaluate a semantic similarity.
    Type: Grant
    Filed: December 29, 2015
    Date of Patent: May 30, 2017
    Assignee: Huawei Technologies Co., Ltd.
    Inventors: Lin Ma, Kai Liu, Hao Xiong
  • Patent number: 9607524
    Abstract: A method, a system, and a computer program product for creating and evaluating assessment sheets are disclosed. The method includes identifying one or more first location identifiers based on one or more markings provided in a scanned master assessment sheet. A machine-readable code (MRC) is generated based on the one or more first location identifiers. The MRC is then split into a plurality of portions and embedded at various locations on a scanned pre-assessment sheet. The scanned pre-assessment sheet with the embedded MRC is then printed to generate the assessment sheets. Thereafter, one or more second location identifiers are identified based on one or more markings provided by evaluatees in each of the assessment sheets provided to them. The one or more second location identifiers are then compared with the one or more first location identifiers. The result of evaluation is provided based on the comparison.
    Type: Grant
    Filed: September 3, 2013
    Date of Patent: March 28, 2017
    Assignee: Xerox Corporation
    Inventor: Nikesh Anand Rajagopalan
  • Patent number: 9576043
    Abstract: A computer program product for managing and rendering one or more information nodes relative to a current focus. The computer program product receives one or more principal topics from an analysis of the information nodes. The computer program product determines a topic vector from a similarity of a first information node to each of the principal topics and a map from the topic vector to a storage location of the first information node. The computer program product determines one or more current topics from a second information node that has been recently accessed by a user and a current focus vector from a similarity of the one or more current topics to the one or more principal topics. The computer program product renders the current focus vector and the first information node according to a rendering algorithm that is applied to the current focus vector and the first information node.
    Type: Grant
    Filed: September 1, 2016
    Date of Patent: February 21, 2017
    Assignee: International Business Machines Corporation
    Inventors: Brian P. Gaucher, Jeffrey O. Kephart, Jonathan Lenchner, David O.S. Melville
  • Patent number: 9576297
    Abstract: A method and apparatus of communicating with multiple mobile station devices in a concerted effort is disclosed. According to one example method of operation the method may include identifying a number of mobile station devices as being present at the pre-defined venue location via a server and initiating an application on the server that establishes a feedback menu display. The method may also include transmitting the feedback menu display to each of the identified mobile station devices receiving feedback messages from a majority of the mobile station devices responsive to the feedback menu display questions, and processing the feedback messages and providing a result to the moderator device.
    Type: Grant
    Filed: May 4, 2015
    Date of Patent: February 21, 2017
    Assignee: West Corporation
    Inventor: Jason H Groenjes
  • Patent number: 9552742
    Abstract: A teaching aid to aid in evaluating data may include a frame member and an eye guide movably connected to the frame member to guide the user of the teaching aid in evaluating the data. The frame member may include a top frame wall, a pair of opposing a side frame walls to connect to the top frame wall and a bottom frame wall to connect to the pair of opposing side frame walls, and the frame member may include a divider wall to define a cavity for the data to be evaluated.
    Type: Grant
    Filed: June 27, 2012
    Date of Patent: January 24, 2017
    Inventor: Earl Dwight McArthur
  • Patent number: 9542854
    Abstract: A system and method of communication based on the Reverse Multiple-Choice Method of teaching and testing is disclosed where at least one communicant is a machine. The method is applicable for training a machine for knowledge engineering and artificial intelligence oriented applications, as well as for a trained machine to assist a human being engaged in the activity of teaching or testing.
    Type: Grant
    Filed: February 11, 2016
    Date of Patent: January 10, 2017
    Inventors: Indu M. Anand, Pranav Anand
  • Patent number: 9483955
    Abstract: A method/apparatus/system for educational intervention based on a response metric is disclosed. The notice is generated in response to the collection of user and question data, the sending of questions, the receipt of answers, the evaluation of the correctness of the answers, the generation of a response metric, the comparison of the response metric to a threshold, and the generation of the report or notice. The response metric can be reflect the scatter, randomness, and/or slope of student provided answer data, and can be a fractal dimension.
    Type: Grant
    Filed: June 12, 2014
    Date of Patent: November 1, 2016
    Assignee: PEARSON EDUCATION, INC.
    Inventors: William M. Galen, Rasil Warnakulasooriya
  • Patent number: 9460191
    Abstract: A method for managing and rendering one or more information nodes relative to a current focus. The method receives one or more principal topics from an analysis of the information nodes. The method determines a topic vector from a similarity of a first information node to each of the principal topics and a map from the topic vector to a storage location of the first information node. The method determines one or more current topics from a second information node that has been recently accessed by a user and a current focus vector from a similarity of the one or more current topics to the one or more principal topics. The method renders the current focus vector and the first information node according to a rendering algorithm that is applied to the current focus vector and the first information node.
    Type: Grant
    Filed: April 14, 2016
    Date of Patent: October 4, 2016
    Assignee: International Business Machines Corporation
    Inventors: Brian P. Gaucher, Jeffrey O. Kephart, Jonathan Lenchner, David O. S. Melville
  • Patent number: 9396255
    Abstract: Methods and systems for facilitating evaluation of one or more electronic documents. One or more templates are extracted from the one or more electronic documents. At least one area in the one or more templates is identified. The at least one area includes a response provided by a user associated with the electronic document. One or more groups are generated based on similarity between the responses in the at least one area in the one or more templates. The one or more groups are distributed to one or more remote workers or a crowdsourcing platform for the evaluation.
    Type: Grant
    Filed: August 22, 2013
    Date of Patent: July 19, 2016
    Assignee: Xerox Corporation
    Inventors: Pallavi Manohar, Shourya Roy
  • Patent number: 9368044
    Abstract: A remote art jurying system in which art jurors independently evaluate works of art at different locations, times and places. A central database presents information about each work of art and/or the artist to an individual juror. The database provides images of the artwork along with relevant information. Each juror can access the database independent of other jurors in terms of location, or time. The system provides enhanced voting methods. The juror can review or change their votes at any time prior to final submission. They can review all works of art, by categories of art, or by scores. The juror reviews artwork thumbnail images, which can be used to select high resolution images with extended information. Selected categories of art can be enhanced by eliminating the lowest scores to balance artwork finalists across multiple categories of art.
    Type: Grant
    Filed: February 4, 2014
    Date of Patent: June 14, 2016
    Inventor: Paul Fisher
  • Patent number: 9355373
    Abstract: A test results screening tool allows capture of test item and test results data and provides a reviewer with a robust set of tools for entering screening criteria and display of test results by both test subject and test center. The tool includes parametrically sets flags for unwanted testing behaviors including test item harvesting, test subjects with pre-knowledge of the test subject matter, and proxy test takers for some a portion of a test.
    Type: Grant
    Filed: August 16, 2012
    Date of Patent: May 31, 2016
    Assignee: NATIONAL ASSOC. OF BOARDS OF PHARMACY
    Inventors: Joy L. Matthews-López, Paul E. Jones
  • Patent number: 9352232
    Abstract: The present invention provides a technique for preferring a season character by users by increasing a skill capability value of the season character in which a limitation capability value of a character is differentiated according to a time provided in a game compared with a general character in an online game managing the character.
    Type: Grant
    Filed: November 26, 2012
    Date of Patent: May 31, 2016
    Assignee: INTELLECTUAL DISCOVERY CO., LTD.
    Inventors: Jeong Hun Kim, Ki Cheol Jang
  • Patent number: 9299266
    Abstract: A system for educational assessment without testing is provided that includes one or more client systems that are connected to a network allowing students or school officials to communicate with an education framework that performs and manages educational assessment. The one or more client systems issue a message to the education framework requesting a task to be performed. The educational assessment is administered independent of one or more educators so as to avoid interruption of instruction time. A server system receives the message and the education framework proceeds to process the contents of the message. The education framework includes a plurality of programming modules being executed on the server system that provides to educators specific information used for the educational assessment based on the contents of the message.
    Type: Grant
    Filed: August 15, 2011
    Date of Patent: March 29, 2016
    Assignee: LEXIA LEARNING SYSTEMS LLC
    Inventors: Elizabeth Catherine Crawford, Yaacov Petscher, Christopher Schatschneider
  • Patent number: 9208509
    Abstract: A system, method, and computer program product are provided for personalizing content for a user based on a size of a working vocabulary of the user. In use, text is identified from content that is one of consumed and output by a user. Additionally, a size of a working vocabulary of the user is identified using the text. Further, the content is personalized based on the size of the working vocabulary of the user.
    Type: Grant
    Filed: July 13, 2012
    Date of Patent: December 8, 2015
    Assignee: Amdocs Software Systems Limited
    Inventors: David Curran, Juraj Sofranko, Keith Bradley, Michael Kerrigan