Abstract: Systems and methods for displaying estimated relevance indicators for result sets of documents and for displaying query visualizations are disclosed. A method includes receiving a search query including a plurality of query terms. The method further includes searching a database using the search query to identify the result set of documents and calculating an estimated relevance score for the result set of documents. The estimated relevance score is indicative of a degree to which the result set of documents are relevant to the search query. The method further includes providing for display the estimated relevance indicator based on the estimated relevance score. The estimated relevance indicator provides a visual indication of the degree to which the result set of documents are relevant to the search query. Query visualizations including a plurality of nodes and a plurality of connectors are also disclosed.

Abstract: System and method for displaying a user interface of an evaluation system configured to evaluate predicted answers generated by a machine learning system. For example, the method includes receiving textual data and a predicted answer to a question associated with a text object. The text object includes a structured data field of the textual data. The predicted answer includes a confidence level. The confidence level is determined by a machine learning system. In response to determining the confidence level being larger than or equal to a predetermined confidence threshold, the predicted answer and a reference is stored in a storage for retrieval and display. The reference indicates a location of the text object in the textual data. In response to determining the confidence level being smaller than the predetermined confidence threshold, the question and the text object associated with the question is displayed.

Abstract: In one embodiment, a method for adaptive training of a machine learning system configured to predict answers to questions associated with textual data includes receiving predicted answers to questions associated with textual data. The predicted answers are generated based at least in part on one or more first models of a machine learning system. The one or more first models are associated with a first accuracy score. The method further includes determining based at least in part on a quality control parameter whether an evaluation of the questions by one or more external entities is required. In response to determining based at least in part on the quality control parameter that an evaluation of the questions by one or more external entities is required, the questions associated with the textual data and the textual data are sent to the one or more external entities for evaluation.

Abstract: A method comprises receiving a natural language search query, identifying a first set of semantic concepts in the query, creating a vector representation of the first set of semantic concepts, identifying a second set of semantic concepts having a vector representation within a predetermined threshold of similarity to the first set of semantic concepts, performing a search of documents based on the first set of semantic concepts, presenting a result set of documents and the first, second, and third sets of semantic concepts to a user, receiving input from the user, performing a second search of the documents based on the input from the user to obtain a second result set of documents, identifying a fourth set of semantic concepts based on the second result set of documents, and presenting the second result set of documents and the fourth set of semantic concepts to the user.

Abstract: Embodiments of the present disclosure are directed to systems, methods, and computer product programs to identify one or more seminal cases within a database containing legal case data. The disclosed systems and methods provide an approach to identify one or more seminal cases for particular legal issues by mining a text database containing electronic legal documents for the reasons for citing and mining the text within to determine whether the legal issue addressed in the reasons for citing is directed to a seminal case. The data is created through data mining and obtained from the plurality the reasons for citing identifying the seminal cases for a particular legal issue such that the output data corresponding to the seminal cases causes an external device to distinguish the seminal cases when one or more of the seminal cases are returned as the result of a search.

Abstract: System and method for displaying a user interface of an evaluation system configured to evaluate predicted answers generated by a machine learning system. For example, the method includes receiving textual data and a predicted answer to a question associated with a text object. The text object includes a structured data field of the textual data. The predicted answer includes a confidence level. The confidence level is determined by a machine learning system. In response to determining the confidence level being larger than or equal to a predetermined confidence threshold, the predicted answer and a reference is stored in a storage for retrieval and display. The reference indicates a location of the text object in the textual data. In response to determining the confidence level being smaller than the predetermined confidence threshold, the question and the text object associated with the question is displayed.

Abstract: A method and system for maintaining the order of arrangement of a modified object list is provided. The method includes receiving selection of an object from a first plurality of objects of an object list, wherein the object list has an order of arrangement and is included on a first digital page displayed on a user interface of a device, receiving selection of a category icon from a plurality of category icons corresponding, to a plurality of categories, wherein the plurality of category icons are included on the first digital page, modifying the object list to exclude the object from the first digital page, and maintaining the order of arrangement of the object list that is modified, wherein the maintaining includes displaying a second plurality of objects included in the object list that is modified on the first digital page in accordance with the order of arrangement.

Abstract: System and methods for managing a clauses database for use with creating electronic documents are disclosed. In one embodiment, a computer-implemented method for updating a database includes receiving, from a graphical user interface, a clause, the graphical user interface comprising an electronic document panel and a clauses panel, receiving, from the clauses panel, a clause approval assignment that assigns the clause to an approver; receiving, from the approver, an approval or a denial of the clause, and when an approval is received, storing the clause in the database, wherein the clause is marked as approved.

Abstract: The method of no fail searching may include receiving a set of keywords from an input set of search fields within a GUI, retrieving a set of search results based on the set of keywords from the input set of search fields, organizing the set of search results according to a set of content types, ranking each search field according to the set of content types, removing the lowest ranked search field from the set of keywords to create a broadened set of search fields when at least one content type has zero results, repeating the retrieving, organizing, ranking, and removing steps until either each content type contains a threshold amount of search results or all search fields have been removed, wherein the broadened set of search fields is used as the input set of search fields in subsequent retrieving steps.

Abstract: In one embodiment, a method for adaptive training of a machine learning system configured to predict answers to questions associated with textual data includes receiving predicted answers to questions associated with textual data. The predicted answers are generated based at least in part on one or more first models of a machine learning system. The one or more first models are associated with a first accuracy score. The method further includes determining based at least in part on a quality control parameter whether an evaluation of the questions by one or more external entities is required. In response to determining based at least in part on the quality control parameter that an evaluation of the questions by one or more external entities is required, the questions associated with the textual data and the textual data are sent to the one or more external entities for evaluation.

Abstract: Systems and methods include receiving textual data and a predicted answer to a question associated with a text object. The text object includes a structured data field of the textual data. The predicted answer includes a confidence level. The confidence level is determined by a machine learning system based at least in part on one or more models of the machine learning system and the textual data. In response to determining the confidence level being larger than or equal to a predetermined confidence threshold, the predicted answer and a reference is stored in a storage for retrieval and display. The reference indicates a location of the text object in the textual data. In response to determining the confidence level being smaller than the predetermined confidence threshold, the question and the text object associated with the question is displayed, at the user interface, to a user for inputting a true answer.

Abstract: The method of no fail searching may include receiving a set of keywords from an input set of search fields within a GUI, retrieving a set of search results based on the set of keywords from the input set of search fields, organizing the set of search results according to a set of content types, ranking each search field according to the set of content types, removing the lowest ranked search field from the set of keywords to create a broadened set of search fields when at least one content type has zero results, repeating the retrieving, organizing, ranking, and removing steps until either each content type contains a threshold amount of search results or all search fields have been removed, wherein the broadened set of search fields is used as the input set of search fields in subsequent retrieving steps.

Abstract: System and method for adaptive training of a machine learning system processing textual data. For example, a method for adaptive training of a machine learning system configured to predict answers to questions associated with textual data includes receiving predicted answers to questions associated with textual data. The predicted answers are generated based at least in part on one or more first models of a machine learning system. The one or more first models are associated with a first accuracy score. The method further includes determining based at least in part on a quality control parameter whether an evaluation of the questions by one or more external entities is required. In response to determining based at least in part on the quality control parameter that an evaluation of the questions by one or more external entities is required, the questions associated with the textual data and the textual data are sent to the one or more external entities for evaluation.

Abstract: Systems and methods for displaying estimated relevance indicators for result sets of documents and for displaying query visualizations are disclosed. A method includes receiving a search query including a plurality of query terms. The method further includes searching a database using the search query to identify the result set of documents and calculating an estimated relevance score for the result set of documents. The estimated relevance score is indicative of a degree to which the result set of documents are relevant to the search query. The method further includes providing for display the estimated relevance indicator based on the estimated relevance score. The estimated relevance indicator provides a visual indication of the degree to which the result set of documents are relevant to the search query. Query visualizations including a plurality of nodes and a plurality of connectors are also disclosed.

Abstract: Systems and methods generate hierarchical displays of search results from a document corpus or documents. The hierarchical displays include parent nodes and children nodes. The parent nodes identify broader concepts and children nodes identify narrower concepts. Concepts can be terms of art, areas of law, or the like. Hierarchical displays may be modified to add or remove nodes. Analysis tools may provide metrics for assessing a document corpus, search query, or search results.

Abstract: Systems and methods for displaying estimated relevance indicators for result sets of documents and for displaying query visualizations are disclosed. A method includes receiving a search query including a plurality of query terms. The method further includes searching a database using the search query to identify the result set of documents and calculating an estimated relevance score for the result set of documents. The estimated relevance score is indicative of a degree to which the result set of documents are relevant to the search query. The method further includes providing for display the estimated relevance indicator based on the estimated relevance score. The estimated relevance indicator provides a visual indication of the degree to which the result set of documents are relevant to the search query. Query visualizations including a plurality of nodes and a plurality of connectors are also disclosed.

Abstract: In one embodiment, a method of evaluating a term in a patent-related document includes receiving a selected term, generating one or more variant suggestions related to the selected term, displaying the one or more variant suggestions, receiving one or more selected variant suggestions from the one or more variant suggestions, and associating the one or more selected variant suggestions with the selected term. In another embodiment, a method of evaluating a term in a patent-related document includes displaying a list of terms appearing in a first patent-related document and a second patent-related document, displaying a graphical representation of whether or not individual terms in the list of terms appear in the first patent-related document or the second patent-related document, and receiving a selected term from the list of terms. The method further includes displaying one or more variant suggestions related to the selected term.

Abstract: Systems and methods for generating logical documents for a document evaluation system are provided. For example, a method for generating logical documents for a document evaluation system includes receiving a first child document associated with a master document. The master document includes one or more master terms. The first child document includes one or more first child terms and a first date. The method further includes generating a logical document based at least in part on the master document. The logical document includes one or more current document values. Each current document value corresponds to one master term and is associated with a current date and a current reference. The current reference identifies a region in the master document related to the one master term. The method further includes selecting a first child term from the one or more first child terms.

Abstract: A method includes receiving, at a computing device, an electronic document, wherein the electronic document is a legal brief comprising a plurality of paragraphs; identifying one or more argument paragraphs from among the plurality of paragraphs, wherein each of the one or more argument paragraphs includes one or more legal arguments; performing a textual search of a corpus of legal documents based on a selected set of the identified argument paragraphs; performing a semantic search of the corpus of legal documents based on the selected set of the identified argument paragraphs; combining results of the textual search and the semantic search; and presenting the combined results to a user.

Abstract: A computing device generates a patent characteristic space including patent vectors having a first number of features. Each of the patent vectors represents a respective patent family of one or more respective patents, and each of the features represents a respective property of the respective patent families represented by the patent vectors. The device performs a linear dimensionality reduction on the patent characteristic space to obtain an intermediate characteristic space that includes the patent vectors having a number of features reduced to a second number of features selected based on the respective properties of the patent families. The device performs a non-linear dimensionality reduction on the intermediate characteristic space to obtain, and present via a user interface according to a visualization preference, a reduced characteristic space that includes the patent vectors having a number of features reduced to a third number of features selected based on the visualization preference.