Abstract: A method of automatically generating a prompt for a large language model includes receiving a test set comprising a plurality of human ratings and a plurality of large language model ratings. Each human rating of the plurality of human ratings includes a selection of a first product or a second product, and each large language model rating of the plurality of large language model ratings includes a selection of the first product or the second product using the prompt. The method further includes comparing the plurality of human ratings and the plurality of large language model ratings against a metric. When the metric is satisfied, the method includes transmitting the prompt as a final prompt. When the metric is not satisfied, the method includes generating, by an evaluation large language model, a refined prompt.

Abstract: In one embodiment, a method of generating an electronic document includes receiving a source document, receiving a summary document of the source document, generating a residual model includes information present in the source document and not present in the summary document, receiving at least a portion of the residual model and at least a portion of the summary document by a trained model as an input, and outputting, by the trained model an inverse summary document, where the inverse summary document includes at least some information present in the source document but not present in the summary document.

Abstract: A database clustering search system includes a processor; and a non-transitory, processor readable storage medium communicatively coupled to the processor, the non-transitory, processor readable storage medium including one or more instructions stored thereon that, when executed, cause the processor to: receive input comprising a search query; deploy a search based on the input; populate a list that refers to one or more content types; and output the list to a user device.

Abstract: System and methods for managing a clauses database for use with creating electronic documents are disclosed. In one embodiment, a computer-implemented method for updating a database includes receiving, from a graphical user interface, a clause, the graphical user interface comprising an electronic document panel and a clauses panel, receiving, from the clauses panel, a clause approval assignment that assigns the clause to an approver; receiving, from the approver, an approval or a denial of the clause, and when an approval is received, storing the clause in the database, wherein the clause is marked as approved.

Abstract: System and methods for managing a clauses database for use with creating electronic documents are disclosed. In one embodiment, a computer-implemented method for updating a database includes receiving, from a graphical user interface, a clause, the graphical user interface comprising an electronic document panel and a clauses panel, receiving, from the clauses panel, a clause approval assignment that assigns the clause to an approver; receiving, from the approver, an approval or a denial of the clause, and when an approval is received, storing the clause in the database, wherein the clause is marked as approved.

Abstract: In embodiment, a method of displaying an answer of a question-answer pair in response to a natural language search query includes receiving, from a Bidirectional Encoder Representations from Transformers (BERT) model, an array of attention matrices for the question-answer pair, where each attention matrix of the array of attention matrices includes an array of attribution values, generating a total attribution value for each word of an answer of the question-answer pair from the array of attention matrices, and displaying the answer on an electronic display, wherein one or more words of the answer is highlighted based on the total attribution values for each word.

Abstract: A database clustering search system includes a processor; and a non-transitory, processor readable storage medium communicatively coupled to the processor, the non-transitory, processor readable storage medium including one or more instructions stored thereon that, when executed, cause the processor to: receive input comprising a search query; deploy a search based on the input; populate a list that refers to one or more content types; and output the list to a user device.

Abstract: A method for expanding a hierarchical taxonomy associated with a corpus of documents may include performing cluster analysis on documents associated with a leaf node of the taxonomy to determine a plurality of first clusters, each cluster of the plurality of first clusters being associated with one or more documents associated with the leaf node; determining one or more topics associated with the documents in each cluster of the plurality of first clusters; performing cluster analysis on the topics to determine a plurality of second clusters, each cluster of the plurality of second clusters being associated with one or more of the topics; determining a name for each cluster of the plurality of second clusters; and expanding the hierarchical taxonomy based on the topics associated with the documents in each cluster of the plurality of first clusters and the determined name for each cluster of the plurality of second clusters.

Abstract: A system including a search engine configured to: determine a search result based on a search query and search query context, extract query tokens from the search query, determine, query token hit set(s) within each search result document, each query token hit set including query token hit(s) within a defined proximity range of a query token centrally-located within the defined proximity range, determine a query token density value (QTDV) between each query token hit and the centrally-located query token for each query token hit set within each document, each QTDV based on a distance between each query token hit and the centrally-located query token, determine a query token density score (QTDS) for each query token hit set, determine a document density score (DDS) for each document, rank or re-rank each document within the search result based on the DDSs, and transmit a ranked/re-ranked search engine result page for presentation.

Abstract: In one embodiment, a method of displaying electronic documents relevant to a matter includes receiving matter information relating to the matter, extracting key phrases from the matter information, generating a query from the key phrases, searching one or more data sources for electronic documents using the query, and displaying, on an electronic display, one or electronic documents relevant to the query.

Abstract: In one embodiment, a method of automatically assigning a matter includes receiving matter information, generating one or more new matter vectors from the matter information, comparing the one or more new matter vectors to historical matter vectors of historical matter information relating to historical matters, selecting a plurality of historical matters from the historical matters, where each historical matter of the plurality of historical matters has a similarity between the one or more new matter vectors and one or more historical matter vectors that is above a threshold, determining a subset of employees associated with the plurality of historical matters, where the subset of employees is a subset of a plurality of employees, applying one or more rules to the subset of employees to generate a score for each employee of the subset of employees, and assigning the matter to an assigned employee having a highest score.

Abstract: Systems and methods for automatic extraction of electronic data are disclosed. In one aspect, a method of automatically generating a matter form, the method includes receiving input data from one or more sources, the input data relating to a matter, classifying, using a trained model, a matter classification based at least in part on the input data, selecting one or more matter forms based on the matter classification, automatically extracting field data from the input data, and automatically populating fields of the one or more matter forms with the extracted field data.

Abstract: Systems and methods for displaying estimated relevance indicators for result sets of documents and for displaying query visualizations are disclosed. A method includes receiving a search query including a plurality of query terms. The method further includes searching a database using the search query to identify the result set of documents and calculating an estimated relevance score for the result set of documents. The estimated relevance score is indicative of a degree to which the result set of documents are relevant to the search query. The method further includes providing for display the estimated relevance indicator based on the estimated relevance score. The estimated relevance indicator provides a visual indication of the degree to which the result set of documents are relevant to the search query. Query visualizations including a plurality of nodes and a plurality of connectors are also disclosed.

Abstract: System and method for displaying a user interface of an evaluation system configured to evaluate predicted answers generated by a machine learning system. For example, the method includes receiving textual data and a predicted answer to a question associated with a text object. The text object includes a structured data field of the textual data. The predicted answer includes a confidence level. The confidence level is determined by a machine learning system. In response to determining the confidence level being larger than or equal to a predetermined confidence threshold, the predicted answer and a reference is stored in a storage for retrieval and display. The reference indicates a location of the text object in the textual data. In response to determining the confidence level being smaller than the predetermined confidence threshold, the question and the text object associated with the question is displayed.

Abstract: In one embodiment, a method for adaptive training of a machine learning system configured to predict answers to questions associated with textual data includes receiving predicted answers to questions associated with textual data. The predicted answers are generated based at least in part on one or more first models of a machine learning system. The one or more first models are associated with a first accuracy score. The method further includes determining based at least in part on a quality control parameter whether an evaluation of the questions by one or more external entities is required. In response to determining based at least in part on the quality control parameter that an evaluation of the questions by one or more external entities is required, the questions associated with the textual data and the textual data are sent to the one or more external entities for evaluation.

Abstract: A method comprises receiving a natural language search query, identifying a first set of semantic concepts in the query, creating a vector representation of the first set of semantic concepts, identifying a second set of semantic concepts having a vector representation within a predetermined threshold of similarity to the first set of semantic concepts, performing a search of documents based on the first set of semantic concepts, presenting a result set of documents and the first, second, and third sets of semantic concepts to a user, receiving input from the user, performing a second search of the documents based on the input from the user to obtain a second result set of documents, identifying a fourth set of semantic concepts based on the second result set of documents, and presenting the second result set of documents and the fourth set of semantic concepts to the user.

Abstract: Embodiments of the present disclosure are directed to systems, methods, and computer product programs to identify one or more seminal cases within a database containing legal case data. The disclosed systems and methods provide an approach to identify one or more seminal cases for particular legal issues by mining a text database containing electronic legal documents for the reasons for citing and mining the text within to determine whether the legal issue addressed in the reasons for citing is directed to a seminal case. The data is created through data mining and obtained from the plurality the reasons for citing identifying the seminal cases for a particular legal issue such that the output data corresponding to the seminal cases causes an external device to distinguish the seminal cases when one or more of the seminal cases are returned as the result of a search.

Abstract: System and method for displaying a user interface of an evaluation system configured to evaluate predicted answers generated by a machine learning system. For example, the method includes receiving textual data and a predicted answer to a question associated with a text object. The text object includes a structured data field of the textual data. The predicted answer includes a confidence level. The confidence level is determined by a machine learning system. In response to determining the confidence level being larger than or equal to a predetermined confidence threshold, the predicted answer and a reference is stored in a storage for retrieval and display. The reference indicates a location of the text object in the textual data. In response to determining the confidence level being smaller than the predetermined confidence threshold, the question and the text object associated with the question is displayed.

Abstract: A method and system for maintaining the order of arrangement of a modified object list is provided. The method includes receiving selection of an object from a first plurality of objects of an object list, wherein the object list has an order of arrangement and is included on a first digital page displayed on a user interface of a device, receiving selection of a category icon from a plurality of category icons corresponding, to a plurality of categories, wherein the plurality of category icons are included on the first digital page, modifying the object list to exclude the object from the first digital page, and maintaining the order of arrangement of the object list that is modified, wherein the maintaining includes displaying a second plurality of objects included in the object list that is modified on the first digital page in accordance with the order of arrangement.

Abstract: System and methods for managing a clauses database for use with creating electronic documents are disclosed. In one embodiment, a computer-implemented method for updating a database includes receiving, from a graphical user interface, a clause, the graphical user interface comprising an electronic document panel and a clauses panel, receiving, from the clauses panel, a clause approval assignment that assigns the clause to an approver; receiving, from the approver, an approval or a denial of the clause, and when an approval is received, storing the clause in the database, wherein the clause is marked as approved.