METHOD OF AUTOMATED DISCOVERY OF TOPIC RELATEDNESS

Abstract

A computer system and method for automated discovery of topic relatedness are disclosed. According to an embodiment, topics within documents from a corpus may be discovered by applying multiple topic identification (ID) models, such as multi-component latent Dirichlet allocation (MC-LDA) or similar methods. Each topic model may differ in a number of topics. Discovered topics may be linked to the associated document. Relatedness between discovered topics may be determined by analyzing co-occurring topic IDs from the different models, assigning topic relatedness scores, where related topics may be used for matching/linking a feature of interest. The disclosed method may have an increased disambiguation precision, and may allow the matching and linking of documents using the discovered relationships.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Utility application Ser. No. 14/557,906, entitled “METHOD OF AUTOMATED DISCOVERY OF TOPICS RELATEDNESS,” filed Dec. 2, 2014, which claims a benefit of U.S. Provisional Application No. 61/910,754, entitled “METHOD FOR AUTOMATED DISCOVERY OF TOPICS RELATEDNESS,” filed Dec. 2, 2013, which are hereby fully incorporated by reference herein for all purposes.

This application is related to U.S. application Ser. No. 14/557,794, entitled “METHOD FOR DISAMBIGUATING FEATURES IN UNSTRUCTURED TEXTS,” filed Dec. 2, 2014, now U.S. Pat. No. 9,239,875 issued Jan. 19, 2016; U.S. application Ser. No. 14/558,300, entitled “EVENT DETECTION THROUGH TEXT ANALYSIS USING TRAINED EVENT TEMPLATE MODELS,” filed Dec. 2, 2014 now U.S. Pat. No. 9,177,254 issued Nov. 3, 2015; and U.S. application Ser. No. 14/558,076, entitled “METHOD FOR AUTOMATED DISCOVERY OF NEW TOPICS,” filed Dec. 2, 2014, now U.S. Pat. No. 9,177,262 issued Nov. 3, 2015; each of which are hereby fully incorporated by reference herein for all purposes.

TECHNICAL FIELD

The present disclosure relates, in general, to data management; and, more specifically, to text analytics.

BACKGROUND

Using multiple topic models without determining their relatedness offers limited value. It becomes more difficult to find and discover a feature of interest as the collective knowledge continues to be digitized and stored in the form of news, blogs, Web pages, scientific articles, books, images, sound, video, and social networks. New computational tools to help organize, search, and understand these vast amounts of information are needed. Current tools to work with online information include search and links, automated and human generated topics.

One of the limitations of automatically generating topics is that models may be too broad or too specific, decreasing analytics accuracy that may be achieved as a result. In addition, automatically generated topics do not strongly define any taxonomy, ontology or other semantic meaning.

One of the limitations of human generated topics is they have human-bias, thus, topics and taxonomies represent a specific viewpoint. Human generated topics and topic taxonomies may be expensive and time consuming to create and maintain. In addition to being difficult and expensive to create and maintain, human generated topics may not meet the needs of various users.

Therefore, there is still a need for automatically discovering related topics from a corpus ranging from broad topics to more specific topics based on the content of a large corpus. Automatically discovered topic relationships can improve search results, content navigation and provide increased precision for text analytics tasks including entity disambiguation and document linking using the discovered topic relationships.

SUMMARY

An aspect of the present disclosure is a system and method for automated discovery of topic relatedness, using probabilistic modeling, such as a multi-component extension of latent Dirichlet allocation (MC-LDA) or similar methods, to discover topics in a corpus, employing multiple topic identification (ID) models, with differing numbers of topics, tagging topics in all docs, and analyzing co-occurring topic IDs from the different models to discover relatedness by assigning topic relatedness scores, where related topics may be employed for matching/linking topics with documents.

Benefits of the disclosed system and method may be an increased disambiguation precision, allowing matching and linking of documents using the discovered relationships.

One aspect of the present disclosure may be the development of multiple topic models (e.g., based on latent Dirichlet allocation (LDA) or a similar method) against a large corpus of documents, in order to automatically generate topics at different levels of granularity or other model differences and build a hierarchy of related topics dynamically from the corpus.

Topic models may differ in terms of the topics they identify and other parameters. One model may have broad base topics; other models may have greater level of granularity.

Another aspect of the present disclosure may be to classify a large corpus of documents using all the topic models, in order to determine topic relatedness among all the topics using the topic co-occurrence within the corpus.

The disclosed method for automated discovery of topic relatedness may be employed to perform entity disambiguation, as a tool for search navigation, to find related articles, and to discover information from an unknown corpus.

In one embodiment, a method comprises generating, via a first topic model computer, a first term vector identifying a first topic in a plurality of documents in a document corpus; generating, via a second topic model computer, a second term vector identifying a second topic in the plurality of documents in the document corpus; linking, via a topic detection computer, each of the first and second topics across the plurality of documents in the document corpus; assigning, via the topic detection computer, a relatedness score to each of the linked first and second topics based on co-occurrence of each of the linked first and second topics across the plurality of documents in the document corpus; and determining, via the topic detection computer, whether the first and second linked topics are related across the plurality of documents in the document corpus based at least in part on the relatedness score.

In another embodiment, a system comprises a first topic model computer comprising a processor configured to generate a first term vector identifying a first topic in a plurality of documents in a document corpus; a second topic model computer comprising a processor configured to generate a second term vector identifying a second topic in the plurality of documents in the document corpus; and a topic detection computer comprising a processor configured to: (a) link each of the first and second topics across the plurality of documents in the document corpus, (b) assign a relatedness score to each of the linked first and second topics based on co-occurrence of each of the linked first and second topics across the plurality of documents in the document corpus, and (c) determine whether the first and second linked topics are related across the plurality of documents in the document corpus based at least in part on the relatedness score.

In yet another embodiment, a non-transitory computer readable medium is provided having stored thereon computer executable instructions. The instructions comprise generating, via a first topic model computer module of a computer, a first term vector identifying a first topic in a plurality of documents in a document corpus, generating, via a second topic model computer module of the computer, a second term vector identifying a second topic in the plurality of documents in the document corpus, and linking, via a topic detection computer module of the computer, each of the first and second topics across the plurality of documents in the document corpus. The instructions further include assigning, via the topic detection computer module of the computer, a relatedness score to each of the linked first and second topics based on co-occurrence of each of the linked first and second topics across the plurality of documents in the document corpus, and determining, via the topic detection computer module of the computer, whether the first and second linked topics are related across the plurality of documents in the document corpus based at least in part on the relatedness score.

Numerous other aspects, features and benefits of the present disclosure may be made apparent from the following detailed description taken together with the drawing figures.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure can be better understood by referring to the following figures. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the disclosure. In the figures, reference numerals designate corresponding parts throughout the different views.

FIG. 1 is a block diagram of a computer system for discovering topics, according to an exemplary embodiment.

FIG. 2 is a flowchart of a method for discovering topic relatedness, according to an exemplary embodiment.

FIG. 3 is diagram of a graphical representation of topic relatedness among documents, according to an exemplary embodiment.

DEFINITIONS

As used here, the following terms may have the following definitions:

“Document” refers to a discrete electronic representation of information having a start and end.

“Corpus” refers to a collection, including a computer database, of one or more documents.

“Feature” refers to any information which is at least partially derived from a document.

“Feature attribute” refers to metadata associated with a feature; for example, location of a feature in a document, among others.

“Memory” refers to any hardware component suitable for storing information and retrieving said information at a sufficiently high speed.

“Topic” refers to a set of thematic information which is at least partially derived from a corpus.

“Topic Model” refers to a computer based hypothetical description of a complex entity or process.

“Term Vector” refers to a computer based algebraic model for representing text documents (and any objects, in general) as vectors of identifiers, e.g., index terms.

“Topic” refers to a set of thematic information which is at least partially derived from a corpus.

DETAILED DESCRIPTION

The present disclosure is here described in detail with reference to embodiments illustrated in the drawings, which form a part hereof. Other embodiments may be used and/or other changes may be made without departing from the spirit or scope of the present disclosure. The illustrative embodiments described in the detailed description are not meant to be limiting of the subject matter presented here.

The present disclosure describes a system and method for automated discovery of topic relatedness.

Some embodiments may develop multiple computer executed topic models against a large corpus of documents, including various types of knowledge bases from sources such as the internet and internal networks, among others.

The embodiments herein recite a notion of “topic” relatedness and methods for the automatic discovery of topic relatedness in a large document corpus. In exemplary method, the topic relatedness among the documents can be computed based on pre-built topic models. This “topic” relatedness can be used as a supplementary feature to enhance various existing applications related to data analytics in general and text analytics. Additionally, these different pre-built topic models (employed for computing topic relatedness) are built with different levels of granularity of topics, vocabulary, and converging parameters, thereby providing a vertical hierarchy/scalability over a specific domain of interest. LDA topic modeling can be extended to support multi-component LDA, where each component is treated as conditionally-independent, given document topic proportions. This components can include features, such as terms, key phrases, entities, facts, etc. This approach can provide a concept of horizontal scalability of the topic models over a specific domain. Further, analytics application can benefit from the described notion of topic relatedness.

FIG. 1 is a block diagram of a computer system 100 for automated discovery of new topics, according to an embodiment. Computer system 100 comprises at least one processor configured to execute at least one module. In this exemplary embodiment, the system 100 for automated discovery of new topics may include a master topic model (MTM) computer module 102 that executes computer readable instructions for building a master topic computer model based on data 104 extracted from a large corpus of documents. MTM module 102 may produce a set of topics 106 defined by a set 108 of term vectors. As newer data 104 is uploaded to the corpus, a periodic new model (PNM) computer module 110 executes computer readable instructions for creating a periodic new computer model to periodically analyze newer data 104. PNM computer module 110 produces another set of topics 112 defined by a new set 114 of term vectors. Change detection computer module 116 executes computer readable instructions for measuring differences between topics 106 and topics 112 to identify new topics that are not represented. Change detection computer module 116 may also use term vector differences to compare and measure the significance of topics based on established thresholds. Change detection computer module 116 may produce zero or more topics that are not represented in the old model. In this way, the system may periodically add the new topics to the MTM module 102.

According to one embodiment, the respective computer topic models executed by the MTM module 102 and PNM module 110 may be developed using a multi-component extension of latent Dirichlet allocation (MC-LDA) or similar methods.

The models may differ in terms of the topics identified and other parameters, such as the multi-document components used, the vocabulary size for each component, and hyperparameter settings for the prior Dirichlet distributions on topic-term and document-topic proportions, among other parameters. In an embodiment, one model may have broad based topics, while the other model may have a greater level of topic granularity.

According to various embodiments, a large corpus of documents may be classified employing all the models generated in the system 100 for automated discovery of topics.

According to various embodiments, the topics may be automatically generated at different levels of granularity or by other model differences, such as the multi-document components used, the vocabulary size for each component, and hyperparameter settings for the prior Dirichlet distributions on topic-term and document-topic proportions, among others. The generated topics may be used to build a hierarchy of related topics dynamically from a corpus.

Subsequently, topic relatedness may be determined among all the topic co-occurrence within the corpus.

FIG. 2 is a flowchart of a method 200 for automated discovery of topic relatedness, according to an embodiment.

In step 202, topics in a document corpus are identified, via the computer system 100, for automated discovery of new topics. In step 204, the identified topics may be linked with documents within a corpus under analysis (e.g., via the change detection computer module 116). The linked topics may be compared, in step 206, across all documents, in order to assign relatedness scores in step 208.

According to one embodiment, the change detection computer module 116 creates a graph of co-occurring topics as topics are linked with documents in the document corpus.

In step 210, depending on the relatedness score assigned to each topic in step 208, the change detection computer module 116 determines topic relatedness employing weighting algorithms. According to various embodiments, the weighting algorithms may use the topic co-occurrence within a document and across documents of the corpus. The higher the weight assigned to a topic, the higher the relatedness score may be assigned between topics.

FIG. 3 is a diagram of a graphical representation of records in a database 300, showing co-occurring topics between documents, which may be employed to determine topic relatedness, according to an embodiment. The database may comprise a plurality of records, each record storing information pulled from a machine readable computer file (e.g., a document). The information may include a topic. The graphical representation shows relationships between these records.

The graph of co-occurring topics may include the representation of each document analyzed within a corpus, as well as the models and topics associated to the document.

After generating topic models by computer system 100, a number of topics may be identified within the corpus data 104. The generated topics may be linked with related documents. Each topic model may include different parameters and, thus, different number of topics. In an embodiment, one model may have 64 topics, another model may have 1024 topics, and other models may have 16,000 topics.

For example, for a corpus of documents, computer system 100 can build three computer-generated topic models. Document one 302 may include topic five 304 from model one 306, topic seven 308 from model two 310, and topic nine 312 from model three 314. Document two 316 may include topic five 304 from model one 306, topic seven 308 from model two 310, and topic eight 318 from model three 314. Document three 320 may include topic five 304 from model one 306, topic seven 308 from model two 310, and topic ten 322 from model three 314.

Because document one 302, document two 316, and document three 320 include topic five 304 from model one 306 and topic seven 308 from model two 310, it implies that a relatedness may occur between topic five 304 and topic seven 308. In this example, there may be some relatedness between topic nine 312 (from model three 314) and topic seven 308, or topic eight 318 and topic seven 308, but it may be a weaker relatedness than the relatedness between topic five 304 and topic seven 308.

The resulting web may be represented by methods known for those skilled in the art, such as bipartite graphs, weighted graphs, and probabilistic graphs, among others. The representation may be employed to identify topic co-occurrences among the documents within the corpus.

Therefore, an entire document corpus may be analyzed to find the strongest relatedness between the discovered topics 202, which may be represented by mechanisms known for those skilled in the art such as community detection in networks, Markov clustering in graphs, and spectral clustering in graphs, among others.

Applying topic relatedness within documents of a corpus, topic relatedness may be determined by employing topic co-occurrence within a corpus.

Example #1 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to perform feature disambiguation within documents of a corpus.

Example #2 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed as a tool for search navigation or browsing.

Example #3 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to find related articles.

Example #4 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to perform information discovery of related topics from an unknown corpus.

Example #5 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed by varying the quantity of models and/or topics.

Example #6 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may use any domain of data.

Example #7 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to find related features or entities.

Example #8 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to discover a hierarchy of relations.

Example #9 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to determine the relatedness/mapping of independently defined hierarchies (i.e. medical coding, product codes, ISO/FIPS codes).

Example #10 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to map a human taxonomy to an automatically discovered topic hierarchy. This mapping can provide human friendly names to the topics.

Example #11 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to measure the relatedness of extracted features.

Example #12 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may employ textual data/video data/image data.

Example #13 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to perform signal analysis.

Example #14 is an application of a method for automated discovery of topic relatedness, where the method of automated discovery may be employed to relate multiple independent signatures.

The foregoing method descriptions and the process flow diagrams are provided merely as illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the order presented. As will be appreciated by one of skill in the art the steps in the foregoing embodiments may be performed in any order. Words such as “then,” “next,” etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Although process flow diagrams may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed here may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

Embodiments implemented in computer software may be implemented in software, firmware, middleware, microcode, hardware description languages, or any combination thereof. A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

The actual software code or specialized control hardware used to implement these systems and methods is not limiting of the invention. Thus, the operation and behavior of the systems and methods were described without reference to the specific software code being understood that software and control hardware can be designed to implement the systems and methods based on the description here.

When implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of a method or algorithm disclosed here may be embodied in a processor-executable software module which may reside on a computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable media includes both computer storage media and tangible storage media that facilitate transfer of a computer program from one place to another. A non-transitory processor-readable storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, such non-transitory processor-readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible storage medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer or processor. Disk and disc, as used here, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a non-transitory processor-readable medium and/or computer-readable medium, which may be incorporated into a computer program product.

The preceding description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined here may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown here but is to be accorded the widest scope consistent with the following claims and the principles and novel features disclosed here.

Claims

1. A method comprising:

in response to generating, by a processor, a first term vector defining a first topic in a set of documents based on a first topic model and a second term vector defining a second topic in the set of documents based on a second topic model, identifying, by the processor, a third term vector defining a third topic in the set of documents and linking, by the processor, the first term vector, the second term vector, and the third term vector with the set of documents, wherein the first topic model and the second topic model are formed based on a set of data extracted from the set of documents, wherein the first topic model is implemented based on a generative statistical model with a first set of parameters, wherein the second topic model is implemented based on the generative statistical model with a second set of parameters, wherein the first set of parameters is distinct from the second set of parameters, and wherein the third topic is not represented by the first topic and the second topic in the set of documents;

in response to comparing, by the processor, the first term vector, the second term vector, and the third term vector across the set of documents, assigning, by the processor, a relatedness score to each of the first term vector, the second term vector, and the third term vector based on co-occurrence of each of the first term vector, the second term vector, and the third term vector across the set of documents, and determining, by the processor, whether the first term vector, the second term vector, and the third term vector are topic related across the set of documents based on the relatedness score.

2. The method of claim 1, wherein the first model is based a multi-component extension of the generative statistical model.

3. The method of claim 2, wherein the generative statistical model includes a latent Dirichlet allocation.

4. The method of claim 1, wherein the second model is based on a multi-component extension of the generative statistical model.

5. The method of claim 4, wherein the generative statistical model includes a latent Dirichlet allocation.

6. The method of claim 1, further comprising:

generating, by the processor, a hierarchy of related topics in the document corpus, wherein the hierarchy of related topics is based on the first term vector, the second term vector, and the third term vector.

7. The method of claim 1, wherein the linking further comprises generating, by the processor, a graphical representation of co-occurring linked topics across the documents based on the first term vector, the second term vector, and the third term vector.

8. The method of claim 1, wherein the first set of parameters includes at least one of a multi-document component, a vocabulary size, or a parameter setting for a prior Dirichlet distribution on a topic term.

9. The method of claim 1, wherein the second set of parameters includes at least one of a multi-document component, a vocabulary size, or a parameter setting for a prior Dirichlet distribution on a topic term.

10. The method of claim 1, further comprising:

adding, by the processor, the third term vector to the first set of parameters after the determining.

11. A system comprising:

a processor and a memory, wherein the memory stores a set of instructions executable via the processor, wherein the set of instructions enables: in response to generating, by the processor, a first term vector defining a first topic in a set of documents based on a first topic model and a second term vector defining a second topic in the set of documents based on a second topic model, identifying, by the processor, a third term vector defining a third topic in the set of documents and linking, by the processor, the first term vector, the second term vector, and the third term vector with the set of documents, wherein the first topic model and the second topic model are formed based on a set of data extracted from the set of documents, wherein the first topic model is implemented based on a generative statistical model with a first set of parameters, wherein the second topic model is implemented based on the generative statistical model with a second set of parameters, wherein the first set of parameters is distinct from the second set of parameters, wherein the third topic is not represented by the first topic and the second topic in the set of documents; in response to comparing, by the processor, the first term vector, the second term vector, and the third term vector across the set of documents, assigning, by the processor, a relatedness score to each of the first term vector, the second term vector, and the third term vector based on co-occurrence of each of the first term vector, the second term vector, and the third term vector across the set of documents, and determining, by the processor, whether the first term vector, the second term vector, and the third term vector are topic related across the set of documents based on the relatedness score.

12. The system of claim 11, wherein the first model is based a multi-component extension of the generative statistical model.

13. The system of claim 12, wherein the generative statistical model includes a latent Dirichlet allocation.

14. The system of claim 11, wherein the second model is based on a multi-component extension of the generative statistical model.

15. The system of claim 14, wherein the generative statistical model includes a latent Dirichlet allocation.

16. The system of claim 11, wherein the method further comprises:

generating, by the processor, a hierarchy of related topics in the document corpus, wherein the hierarchy of related topics is based on the first term vector, the second term vector, and the third term vector.

17. The system of claim 11, wherein the linking further comprises generating, by the processor, a graphical representation of co-occurring linked topics across the documents based on the first term vector, the second term vector, and the third term vector.

18. The system of claim 11, wherein the first set of parameters includes at least one of a multi-document component, a vocabulary size, or a parameter setting for a prior Dirichlet distribution on a topic term.

19. The system of claim 11, wherein the second set of parameters includes at least one of a multi-document component, a vocabulary size, or a parameter setting for a prior Dirichlet distribution on a topic term.

20. The system of claim 11, wherein the method further comprises:

adding, by the processor, the third term vector to the first set of parameters after the determining.