Systems and methods for determining an article association measure

- Google

Systems and methods for determining an article association measure are described. In one described method, a computer program receives a first article identifier associated with a first attribute and a second article identifier associated with a second attribute and determines a similarity measure of the first article identifier to the second article identifier based at least in part on the first attribute and on the second attribute. The similarity measure may provide a means for associating various article identifiers so that they may be, for example, retrieved and/or ranked in subsequent search queries.

Skip to: Description  ·  Claims  ·  References Cited  · Patent History  ·  Patent History
Description
RELATED APPLICATION

This application relates to U.S. patent application Ser. No. 10/882,059, filed herewith, titled “Systems and Methods for Associative Information Retrieval,” the entirety of which is incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates generally to systems and methods for information retrieval. The present invention relates particularly to systems and methods for determining an article association measure.

BACKGROUND

Conventional search engines receive a search query from a user and execute a search against a global index. Such conventional search engines typically use one or more conventional methods for performing a search. For example, one known method, described in an article entitled “The Anatomy of a Large-Scale Hypertextual Search Engine,” by Sergey Brin and Lawrence Page, assigns a degree of importance to an article, such as a web page, based on the link structure of the web. The search results are often presented in a list format, comprising article identifiers and brief snippets about the articles in a web page that can be resized.

Often, a user mentally associates articles that the user accesses close in time, that comprise similar content, or that originate from similar sources. In conventional information retrieval systems, the user is often unable to retrieve the articles since the association between the articles that is of interest to the user may not be captured by the information retrieval system. For example, the conventional search engines are unlikely to capture the fact that a user has printed two articles in rapid succession.

SUMMARY

Embodiments of the present invention provide systems and methods for determining an article association measure. In one embodiment of the present invention, a computer program receives a first article identifier associated with a first attribute and a second article identifier associated with a second attribute and determines an association measure of the first article identifier to the second article identifier based at least in part on the first attribute and on the second attribute. The association measure provides a means for associating various article identifiers so that they may be, for example, retrieved and/or ranked in subsequent search queries.

These illustrative embodiments are mentioned not to limit or define the invention, but to provide examples of embodiments of the invention to aid understanding thereof. Illustrative embodiments are discussed in the Detailed Description, and further description of the invention is provided there. Advantages offered by the various embodiments of the present invention may be further understood by examining this specification.

BRIEF DESCRIPTION OF THE FIGURES

These and other features, aspects, and advantages of the present invention are better understood when the following Detailed Description is read with reference to the accompanying drawings, wherein:

FIG. 1 is a block diagram illustrating an illustrative environment in which one embodiment of the present invention may operate;

FIG. 2 is a flowchart illustrating a method of computing an association measure for an article pair in one embodiment of the present invention;

FIG. 3 is a flowchart illustrating a method of associative retrieval in one embodiment of the present invention.

 DETAILED DESCRIPTION

Embodiments of the present invention provide systems and methods for determining an article association measure. Referring now to the drawings in which like numerals indicate like elements throughout the several figures, FIG. 1 is a block diagram illustrating an illustrative environment for implementation of an embodiment of the present invention. While the environment shown reflects a client-side search engine architecture embodiment, other embodiments are possible.

System Architecture

The system 100 shown in FIG. 1 includes multiple client devices 102a-n in communication with a server device 150 over a wired or wireless network 106. The network 106 shown comprises the Internet. In other embodiments, other networks, such as an intranet, may be used instead. Moreover, methods according to the present invention may operate within a single client device.

The client devices 102a-n shown each includes a computer-readable medium 108. The embodiment shown includes a random access memory (RAM) 108 coupled to a processor 110. The processor 110 executes computer-executable program instructions stored in memory 108. Such processors may include a microprocessor, an ASIC, a state machine, or other processor, and can be any of a number of computer processors, such as processors from Intel Corporation of Santa Clara, Calif. and Motorola Corporation of Schaumburg, Ill. Such processors include, or may be in communication with, media, for example computer-readable media, which stores instructions that, when executed by the processor, cause the processor to perform the steps described herein.

Embodiments of computer-readable media include, but are not limited to, an electronic, optical, magnetic, or other storage or transmission device capable of providing a processor, such as the processor 110 of client 102a, with computer-readable instructions. Other examples of suitable media include, but are not limited to, a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ROM, RAM, an ASIC, a configured processor, all optical media, all magnetic tape or other magnetic media, or any other medium from which a computer processor can read instructions. Also, various other forms of computer-readable media may transmit or carry instructions to a computer, including a router, private or public network, or other transmission device or channel, both wired and wireless. The instructions may comprise code from any suitable computer-programming language, including, for example, C, C++, C#, Visual Basic, Java, Python, Perl, and JavaScript.

Client devices 102a-n can be connected to a network 106 as shown, or can be stand-alone machines. Client devices 102a-n may also include a number of external or internal devices such as a mouse, a CD-ROM, DVD, a keyboard, a display, or other input or output devices. Examples of client devices 102a-n are personal computers, digital assistants, personal digital assistants, cellular phones, mobile phones, smart phones, pagers, digital tablets, laptop computers, Internet appliances, and other processor-based devices. In general, the client devices 102a-n may be any type of processor-based platform that operates on any operating system, such as Microsoft® Windows® or Linux, capable of supporting one or more client application programs. For example, the client device 102a shown comprises a personal computer executing client application programs, also known as client applications 120. The client applications 120 can be contained in memory 108 and can include, for example, a word processing application, a spreadsheet application, an e-mail application, an instant messenger application, a presentation application, an Internet browser application, a calendar/organizer application, and any other application capable of being executed by a client device.

The user 112a can interact with the various client applications 120 and articles associated with the client applications 120 via various input and output devices of the client device 102a. Articles include, for example, word processor, spreadsheet, presentation, e-mail, instant messenger, database, and other client application program content files or groups of files, web pages of various formats, such as Hypertext Markup Language (HTML), eXtensible Markup Language (XML), eXtensible Hypertext Markup Language (XHTML), Portable Document Format (PDF) files, and audio files, video files, or any other documents or groups of documents or information of any type whatsoever.

The memory 108 of the client device 102a shown also contains a capture processor 124, a queue 126, and a search engine 122. The client device 102a shown also contains or is in communication with a data store 140. The search engine 122 can receive an explicit query from the user 112a or generate an implicit query and retrieve information from the data store 140 in response to the query.

The search engine 122 shown contains an indexer 130, a query system 132, and a formatter 134. Events and performance data can be sent by the queue 126 to the query system 132 to provide the query system 132 with information concerning current user context. The query system 132 can use this information to generate an implicit query. The query system 132 can also receive and process explicit queries from the user 112a.

The data store 140 can be any type of computer-readable media and can be integrated with the client device 102a, such as a hard drive, or external to the client device 102a, such as an external hard drive or on another data storage device accessed through the network 106. The data store 140 may include any one or combination of methods for storing data, including without limitation, arrays, hash tables, lists, and pairs.

In the embodiment shown in FIG. 1, a user 112a can input an explicit query into a search engine interface displayed on the client device 102a, which is received by the search engine 122. The search engine 122 can also generate an implicit query based on a current user context or state, which can be determined by the query system 132 from contextual real time events. Based on the query, the query system 132 can locate relevant information in the data store 140 and provide a result set. In one embodiment, the result set comprises article identifiers identifying articles associated with the client applications 120 or client articles. Client articles stored in the data store 140 include articles associated with the user 112a or client device 102a, such as word-processing documents, previously viewed web pages and any other article associated with the client device 102a or user 112a. In another embodiment, the result set also comprises identifiers identifying articles located on the network 106 or network articles located by a search engine on a server device. Network articles include articles located on the network 106 not previously viewed or otherwise referenced by the user 112a, such as web pages not previously viewed by the user 112a.

The result sets comprise one or more article identifiers. An article identifier may be, for example, a Uniform Resource Locator (URL), a file name, a link, an icon, a path for a local file, or anything else that identifies an article. In the embodiment shown, an article identifier comprises a URL associated with an article.

Messaging articles stored in the data store 140 include user's emails, chat messages, and instant messaging messages. Each time a message is received, sent, modified, printed, or otherwise accessed, a record is stored in the data store 140. This information can later be searched to identify messages that should be displayed in the user interface.

An embodiment of the present invention may also store message threads in the data store 140. In such an embodiment, messages are related together by various attributes, including, for example, the sender, recipient, date/time sent and received, the subject, the content, or any other attribute of the message. The related messages can then be retrieved as a thread, which may be treated as a document by the display processor 128.

The data store 140 comprises a local index. The local index in the embodiment shown in FIG. 1 may comprise information, such as articles, which are associated with the client device 102a, a user 112a of the client device 102a, or a group of users of the client device 102a. For example, the local index in the data store 140 shown in FIG. 1 may comprise an index of articles created, edited, received, or stored by the client user 112a using the client machine 102a, or articles otherwise associated with the client user 102a or the client machine 112a. The local index may be stored in a client machine, such as in data store 140, in a data store on a local network in a manner accessible by the client machine, on a server accessible to the client machine through the Internet, or in another accessible location.

In contrast, a global index may comprise information relevant to many users or many servers, such as, for example, an index of web pages located on multiple servers in communication with the World Wide Web. One example of a global index is an index used by the Google™ search engine to provide search results in response to a search query.

A single index may comprise both a local and a global index. For example, in one embodiment, an index may comprise both local and global information, and include a user or client identifier with the local information so that it may be identified with the user(s) or client(s) to which it pertains. Moreover, an index, local or global, may be present in one or multiple logical or physical locations.

The formatter 134 can receive the search result set from the query system 132 of the search engine 122 and can format the results for output to a display processor 128. In one embodiment, the formatter 134 formats the results in XML or HTML. The display processor 128 can be contained in memory 108 and can control the display of the result set on a display device associated with the client device 102a. The display processor 128 may comprise various components. For example, in one embodiment, the display processor 128 comprises a Hypertext Transfer Protocol (HTTP) server that receives requests for information and responds by constructing and transmitting HTML pages. In one such embodiment, the HTTP server comprises a scaled-down version of the Apache Web server. In various embodiments, the functions described herein may be performed by various other components and devices.

Through the client devices 102a-n, users 112a-n can communicate over the network 106, with each other and with other systems and devices coupled to the network 106. As shown in FIG. 1, a server device 150 is also coupled to the network 106. In the embodiment shown, the search engine 122 can transmit a search query comprised of an explicit or implicit query or both to the server device 150. The user 112a can also enter a search query in a search engine interface, which can be transmitted to the server device 150. In another embodiment, the query signal may instead be sent to a proxy server (not shown), which then transmits the query signal to server device 150. Other configurations are also possible.

The server device 150 shown includes a server executing a search engine application program, such as the Google™ search engine. Similar to the client devices 102a-n, the server device 150 shown includes a processor 160 coupled to a computer-readable memory 162. Server device 150, depicted as a single computer system, may be implemented as a network of computer processors. Examples of a server device 150 are servers, mainframe computers, networked computers, a processor-based device, and similar types of systems and devices. The server processor 160 can be any of a number of computer processors, such as processors from Intel Corporation of Santa Clara, Calif. and Motorola Corporation of Schaumburg, Ill.

Memory 162 contains the search engine application program, also known as a search engine 170. The search engine 170 locates relevant information in response to a search query from a client device 102a. The search engine 122 then provides the result set to the client device 102a via the network 106. The result set 134 comprises one or more article identifiers. An article identifier may be, for example, a URL, a file name, a link, an icon, a path for a local file, or anything else that identifies an article. In the embodiment shown, an article identifier comprises a URL associated with an article.

In the embodiment shown, the server device 150, or related device, has previously performed a crawl of the network 106 to locate articles, such as web pages, stored at other devices or systems connected to the network 106, and indexed the articles in memory 162 or on another data storage device.

It should be noted that embodiments of the present invention may comprise systems having different architecture than that which is shown in FIG. 1. For example, in some systems according to the present invention, server device 150 may comprise a single physical or logical server, or there may be no server device. The system 100 shown in FIG. 1 is merely illustrative, and is used to explain the illustrative methods shown in FIGS. 2 and 3.

Process

Various methods may be implemented in the environment shown in FIG. 1 and other environments, according to the present invention. Methods according to the present invention may be implemented by, for example, a processor-executable program code stored on a computer-readable medium.

In one embodiment of the present invention, a computer program, such as the query system 132, receives a first article identifier associated with a first attribute and a second article identifier associated with a second attribute and determines an association measure of the first article identifier to the second article identifier based at least in part on the first attribute and on the second attribute. The association measure provides a method of associating various article identifiers so that they may be, for example, retrieved and/or ranked in subsequent search queries. The association measure may also be referred to as an association score or similarity measure. The process may approximate a method or association that a user would perform implicitly. In other words, a user may draft a word-processing document and send it to another user. The two events and the two articles, the word-processing document and email message, are related or associated from the user's perspective.

The determination of the association measure may be performed in various ways. For example, the indexer 130 may create the associations between articles and/or events associated with the articles, i.e., the indexer 130 may determine the associated events/articles during indexing and store that information in the data store 140. The information can then be used at query time. In another embodiment, the query system 132 generates association information at query time. Alternatively, part of the processing may be done by the indexer 130 at indexing time and part done by the query system 132 at query time. For example, in one embodiment, association-in-content scores may be computed at indexing time by the indexer 130, while association-in-time scores are computed at query time by the query system 132. Association in time and association in content are described in further detail below.

The attribute may comprise any of a number of attributes of the article associated with the article identifier or with events associated with the article. The association may be between attributes of the article. In one embodiment, the association is between attributes of events associated with the article. Each article may be associated with one or more events. For instance, one article may be associated with four events, which correspond to the user viewing the article at times t1, t2, t3, and printing the article at time t4.

FIG. 2 is a flowchart illustrating a method of computing an association measure for an article pair in one embodiment of the present invention. The query system 132 receives a first article identifier 202. For example, the query system 132 may receive a first article identifier as part of a result set received in response to an explicit or implicit query submitted to a search engine.

The query system 132 determines an attribute associated with the first article identifier 204. The attribute may comprise, for example, source, metadata, or content attributes, including, for example, the creation time, content, or source of the article associated with the article identifier.

Each article may be associated with one or more attributes directly or indirectly. Embodiments of the present invention may include event capture.

As one example, an event may be captured when an email message is sent. The email message may comprise the article in such an embodiment. The email message or the associated event (e.g., sending the email message) may include attributes, such as the recipient of the email message, the time the email message was sent, the subject, the content, and other attributes. These attributes are stored with the event, and the query system 132 may make use of these attributes to search for associated events and articles associated with the events. Another event may be captured when an article comprising a word-processing document is printed. If the article is printed soon after the email message is received, the email message and word-processing document may be associated.

The query system 132 receives a second article identifier 206. For instance, by searching for events that occur near in time to an event associated with the first article. The query system then determines an attribute of the second article identifier 208. For example, if the first attribute is a date-time attribute, the second attribute may be a date-time attribute as well. The query system 132 then calculates an association measure or for the pair of articles based on the attributes 210. The association measure is a measure corresponding the similarity, relevancy, or other suitable measure indicating that two or more articles or events associated with the articles are associated. In the embodiment shown in FIG. 2, the query system then stores the association measure with the identifiers of the two associated articles 212.

For example, an email message may include the term “budget” in the subject line. A word-processing article may comprise the term “budget” fifty times in the text. The query system 132 determines that these articles are similar. The query system 132 performs the association calculation for each pair of articles. In alternative embodiments, the indexer 130 may perform all or part of the computation instead of, or in addition to, the query system 132.

Association in Time

Articles may be associated because events associated with the articles occur nearby in time. For example, in one embodiment, the query system identifies a first article, article a, finds all events related to article a, and then for each event e does the following: scan the other events ei occurring nearby in time to event e, computes an association measure (e.g., an association score), and adds the association measure to a sum for the article associated with the other event ei. For instance, events e1 and e2 are associated with article a, and events e3 and e4 are associated with article b. Event e1 is nearby in time to e3, and e2 is nearby in time to e4. If the score for the time difference in each case is 1, article b would have an association score of 1+1=2 for its association with article a. Similarly, if events e5 and e6 are related to article c, and they occur nearby in time to e1 and e2 but not as close in time as events e3 and e4. The score for the time difference may be less, e.g., the score for the time difference between e1 and e3 and between e2 and e4 is 0.5, then, the association score for article c with respect to article a is 0.5+0.5=1.

The process of calculating the association score may be repeated for every article that is related to events nearby in time to events associated with the article of interest. In one embodiment, the articles may be ranked by the association score. In the example above, for article a, article b has an association score of 2 and article c has a score of 1. In one embodiment, scores are determined for a plurality of associated articles, and only the top n (e.g., twenty) articles are displayed or otherwise considered. The associated articles may be combined with regular search results or they may be displayed separately, e.g., below each search result, to the right of the search results, or in the sidebar. The database is organized such that is easy and efficient to retrieve events that occur nearby in time, e.g., for a given event, the n events occurring before/after that event may be easily retrieved and the association score accordingly computed accordingly.

In one embodiment, the association measure is determined to be 1 if the time difference between two events is below a threshold. Otherwise, the association measure is determined to be 0. In another embodiment, the association measure may vary linearly from 0 at a maximum time difference threshold, to a maximum of 1 at a difference of 0 in time. In yet another embodiment, another suitable function is used, such as a polynomial, linear, piecewise linear, or exponential function. The association measure is higher for lower time differences and lower for higher time differences.

Other association measures may be associated with events and types of events. For example, an article may be associated with a bookmarking event and a viewing event. In one embodiment, bookmarking an article may have a higher relevancy or interest score than viewing an article. The score derived in relation to these events may be combined with the time-difference score. For example, this score may be multiplied by the time difference score to arrive at the association measure.

For example, the article identifier is associated with an article, and the article includes a date and time on which the article was created, edited, loaded, viewed, saved, or otherwise acted upon, interacted with, or accessed or is associated with an event that includes one or more attributes. The query system 132 may use the date-time attributes to determine the association measure. The query system 132 may calculate the association measure by determining the difference between a date-time attribute associated with the first article identifier and a second date-time attribute associated with the second article identifier.

In one embodiment, two events, a and b, occurring at times ta and tb, and corresponding to two articles, a1 and a2, result in an association between a1 and a2 according to the difference in time, with smaller time differences receiving a higher association score. The association score for these two articles is summed over all occurrences of the articles in events. The articles may be stored in relation to multiple events and may be more or less associated with one another depending on the event. For instance, two articles may be printed one after another on a first day, resulting in a high association score. On another day, one may be printed in the morning and the second in the afternoon, resulting in a lower association score.

The association score may be based on various attributes of the articles. In one embodiment, the association score is based on the time attributes of the articles. For example, the association of a1 to a2 may be equal to the absolute value of the difference in the time attribute for each article, subtracted from a maximum time difference threshold, and divided by a normalization factor. If a1 is printed at 9:00 am, and a2 is printed at 9:15 am, the association score may be equal to 60 (maximum time difference threshold) minus 15 divided by the normalization factor, which may be equal to the maximum time difference threshold, e.g., 60, resulting in an association score of 0.75. In one embodiment, if the difference in the time attribute is greater than some maximum (e.g., one hour), the association score is set to zero. In other words, the articles are determined to be dissimilar based on the attribute.

Association in Content

Articles may be associated because attributes associated with the articles, such as the content (e.g., text), or content associated with events that are associated with the articles, is similar. Each article may be associated with one or more events, and the content may be different for different events. For example, an article a may have four events associated with it. Two of the events contain text t1, and two contain text t2; article b is associated with one event containing text t3. In one embodiment, the query system 132, indexer 130, or other program computes an association measure between articles a and b by comparing the text t1 with t3 and comparing text t2 with t3. The results are then combined. In another embodiment, the program creates a combined or average representation of article a by combining t1 and t2 to create combined text t4. The program then compares t3 and t4 to create an association measure between articles a and b in terms of content.

In one embodiment, the content association measure is computed based on an event e without referring to the associated article. For example, event e contains or is associated with text t, and text t can be used to locate associated events or associated articles without reference to the article associated with event e.

Association in Metadata

In yet another embodiment, the attributes comprise source or metadata attributes. The source or metadata attribute may comprise, for example, an author of an article associated with an article identifier, an organization, a physical location, or any other attribute that relates to the source of the article. The source or metadata attributes may alternatively be associated with events, which are associated with article identifiers. In one embodiment, articles may be associated because they share the same author, the same to: field, the same recipient, etc. As with content associations described above, an association measure in metadata may be computed without reference to the article to which an event refers.

Combined Association Measures

In one embodiment of the present invention, the query system 132 computes a plurality of association measures between two articles based on a plurality of attributes. The plurality of association measures may be combined to create a combined association measure. In another embodiment, the association measures for a pair of articles are normalized using a normalization factor to create a normalized association measure. The normalized association measure may increase the ease with which various article identifiers are ranked.

Using an Association Measure

Given one or more articles or one or more events, the query system 132 can locate related articles or related events based on the association measures. For example, a user performs a search and is provided with a list of events. Each event is associated with an article. The query system 132 finds related articles based on the association measures, and presents the related articles to the user. The query system 132 may present articles to the user (e.g., file f), or events (e.g., file f was saved at time t). The query system 132 may present associated items for each individual result or may combine them into one list of associated items. For instanced, a search returns two events, which may each reference different articles. The query system 132 may show two sets of associated items or may create a combined list of items associated to the two events.

In another embodiment, the query system 132 may also analyze “related event objects”, e.g., a sequence of messages in a conversation may be in a related event object and the related event object may be thought of as an “article”.

An article may be dissimilar (i.e., not associated) based on a first attribute and similar or associated based on a second attribute. For example, two articles may comprise very similar content but never be accessed in close time proximity. Accordingly, the association scores for the first and second article that are related to content would be high and those related to time would be low. The articles may be related by other attributes, including for example, the source of the article. The source may be the author, location, organization, or some other attribute that describes how or where the article was originated. The overall association score may be calculated as the sum of the various association scores.

The overall association score may combine the individual association scores with different weights. For example, in one embodiment, the overall association measure is calculated as follows: overall association=w1*time_association+w2*content association+w3*metadata_association. The weights w1, w2, and w3 may be set to specific values. In an alternative embodiment, the weights may be adjusted based on user interest measures, such as clickthrough. For example, if the user clicks on many articles that were presented to the user because of association in time, the weight for association in time may be increased.

FIG. 3 is a flowchart illustrating a method of associative retrieval in one embodiment of the present invention. In the embodiment shown, the query system 132 receives a search query 302. The search query may be, for example, a keyword query and is a search query for events. The search query 132 may have been generated in response to various actions by the user 112a or in response to some other activity on the client 102a. For example, the user 112a may submit an explicit query that triggers the execution of the search query. The query system 132 submits the search query 132 to an event database, such as database 140, to retrieve potentially related events 304. In response, the query system 132 receives a result set comprising one or more events that are related to the search query 306.

As described herein in relation to FIG. 1, the query system 132 captures events. One example of an event is printing a word-processing document; another example is sending an email message. Each of the events is associated with one or more articles. Each pair of articles may include an association score as described above in relation to FIG. 2.

Referring still to FIG. 3, the query system 132 locates one or more events that are related to each event in the result set 308. The event(s) located by the query system may relate to the original event to a greater or lesser degree. In order to determine the relevance, the query system 132 computes a relevance score for each related event 210. The relevance score is based on the association scores of the articles associated with the events. The query system 132 uses the relevance score to rank the events in the result set 312. The query system 132 then generates displays the related events to the user 314.

An embodiment of the present invention may allow the user to retrieve associated content that would otherwise be difficult or impossible to retrieve as a group. For example, a user may spend several hours researching digital cameras. Later, the user wishes to review results from the searches. Since the searches were performed during a relatively contiguous period of time and related in article identifiers associated with digital cameras, the article identifiers would have high association measures. When the user performs the subsequent explicit query, the query system 132 uses the high association measures to present the user with the previous search results.

In one embodiment, the user specifies the period of time during which the queries of interest were performed and may also specify keywords. For example, the user may know that the “digital camera” queries were performed two days ago in the morning. The user provides this information to the query system 132, which uses the information to identify events and associated article identifiers.

In one embodiment of the present invention, the query system 132 executes a query based on the content attribute associated with an article identifier. For example, the query system 132 identifies an email message as related in time to an article that has been received in response to a search query. The query system 132 extracts the content from the email message and executes a search query using keywords from the content. The query system 132 provides the results of the search query to the user, either combined with the original results or in a separate list.

The foregoing description of embodiments of the invention has been presented only for the purpose of illustration and description and is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Numerous modifications and adaptations thereof will be apparent to those skilled in the art without departing from the spirit and scope of the present invention.

Claims

1. A method comprising:

capturing a plurality of events, wherein the events comprise user interactions with articles on a client device, and wherein the captured events are generated by user interactions involving multiple ones of the plurality of client applications;
receiving a first article identifier identifying a first article and associated with a first date-time attribute, the first date-time attribute associated with a first one of the plurality of events;
receiving a second article identifier identifying a second article and associated with a second date-time attribute, the second date-time attribute associated with a second one of the plurality of events; and
determining an association measure of the first article identifier to the second article identifier based at least in part on the first date-time attribute and on the second date-time attribute, the determining comprising: determining a difference between the first date-time attribute and the second date-time attribute, comparing the difference to a threshold, setting the association measure equal to a first value if the difference exceeds the threshold, and setting the association measure equal to a second value if the difference does not exceed the threshold.

2. The method of claim 1, wherein the difference is inversely proportional to the association measure.

3. The method of claim 1, wherein the association measure comprises a plurality of association measures, and further comprising combining the plurality of association measures to create a combined association measure.

4. The method of claim 1, further comprising normalizing the association measure to create a normalized association measure.

5. The method of claim 1, wherein the association measure is one of a linear, piecewise, or exponential function, wherein the measure indicates higher association for smaller differences.

6. The method of claim 1, wherein the first article identifier is further associated with a first content attribute and the second article identifier is further associated with a second content attribute, the method further comprising determining the association measure of the first article identifier to the second article identifier based at least in part on the first content attribute and on the second content attribute.

7. The method of claim 1, wherein the first article identifier is further associated with a first metadata attribute and the second article identifier is further associated with a second metadata attribute, the method further comprising determining the association measure of the first article identifier to the second article identifier based at least in part on the first metadata attribute and on the second metadata attribute.

8. The method of claim 1, wherein the association score is determined by an indexer as part of indexing and applied as part of execution of a query.

9. The method of claim 1, wherein the association score is determined as part of execution of a query.

10. The method of claim 1, further comprising:

receiving a query;
receiving search results as a result of query execution, the search results comprising the first article identifier and omitting the second article identifier;
identifying the second article identifier as being related to the first article identifier based at least in part on the association measure; and
displaying the second article identifier along with the received search results.

11. The method of claim 7, wherein the first metadata attribute comprises an author.

12. The method of claim 7, wherein the first metadata attribute comprises an organization.

13. The method of claim 3, wherein combining the plurality of association measures to create the combined association measure comprises weighting each association measure according to a weight corresponding to a type of the association measure.

14. A non-transitory computer-readable storage medium on which is encoded program code, the program code comprising:

program code for capturing a plurality of events, wherein the events comprise user interactions with articles on a client device, and wherein the events are generated by user interactions involving multiple ones of the plurality of client applications;
program code for receiving a first article identifier identifying a first article and associated with a first date-time attribute, the first date-time attribute associated with a first one of the plurality of events;
program code for receiving a second article identifier identifying a second article and associated with a second date-time attribute, the second date-time attribute associated with a second one of the plurality of events; and
program code for determining an association measure of the first article identifier to the second article identifier based at least in part on the first date-time attribute and on the second date-time attribute, the determining comprising; determining a difference between the first date-time attribute and the second date-time attribute, comparing the difference to a threshold, setting the association measure equal to a first value if the difference exceeds the threshold, and setting the association measure equal to a second value if the difference does not exceed the threshold.

15. The computer-readable storage medium of claim 14, wherein the association measure comprises a plurality of association measures, and further comprising program code for combining the plurality of association measures to create a combined association measure.

16. The computer-readable storage medium of claim 14, further comprising program code for normalizing the association measure to create a normalized association measure.

17. A computer system, comprising:

a computer processor; and
a non-transitory computer-readable medium storing processor-executable program instructions that when executed by the processor perform actions comprising:
capturing a plurality of events, wherein the events comprise user interactions with articles on a client device, and wherein the captured events are generated by user interactions involving multiple ones of the plurality of client applications;
receiving a first article identifier identifying a first article and associated with a first date-time attribute, the first date-time attribute associated with a first one of the plurality of events;
receiving a second article identifier identifying a second article and associated with a second date-time attribute, the second date-time attribute associated with a second one of the plurality of events; and
determining an association measure of the first article identifier to the second article identifier based at least in part on the first date-time attribute and on the second date-time attribute, the determining comprising:
determining a difference between the first date-time attribute and the second date-time attribute, comparing the difference to a threshold, setting the association measure equal to a first value if the difference exceeds the threshold, and setting the association measure equal to a second value if the difference does not exceed the threshold.

18. The computer system of claim 17, wherein the first article identifier is further associated with a first content attribute and the second article identifier is further associated with a second content attribute, the actions further comprising determining the association measure of the first article identifier to the second article identifier based at least in part on the first content attribute and on the second content attribute.

19. The computer system of claim 17, wherein the first article identifier is further associated with a first metadata attribute and the second article identifier is further associated with a second metadata attribute, the actions further comprising determining the association measure of the first article identifier to the second article identifier based at least in part on the first metadata attribute and on the second metadata attribute.

20. The computer system of claim 19, wherein the first metadata attribute comprises an author.

Referenced Cited
U.S. Patent Documents
5418948 May 23, 1995 Turtle
5678038 October 14, 1997 Dockter et al.
5696962 December 9, 1997 Kupiec
5701469 December 23, 1997 Brandli et al.
5717913 February 10, 1998 Driscoll
5754938 May 19, 1998 Herz et al.
5796393 August 18, 1998 MacNaughton et al.
5826261 October 20, 1998 Spencer
5890152 March 30, 1999 Rapaport et al.
5911139 June 8, 1999 Jain et al.
5933827 August 3, 1999 Cole et al.
5940821 August 17, 1999 Wical
5964839 October 12, 1999 Johnson et al.
5987446 November 16, 1999 Corey et al.
6006222 December 21, 1999 Culliss
6012067 January 4, 2000 Sarkar
6014665 January 11, 2000 Culliss
6070158 May 30, 2000 Kirsch et al.
RE36727 June 6, 2000 Kageneck et al.
6078916 June 20, 2000 Culliss
6112203 August 29, 2000 Bharat et al.
6122647 September 19, 2000 Horowitz et al.
6167434 December 26, 2000 Pang
6182068 January 30, 2001 Culliss
6199059 March 6, 2001 Dahan et al.
6272507 August 7, 2001 Pirolli et al.
6321228 November 20, 2001 Crandall et al.
6397221 May 28, 2002 Greef et al.
6421675 July 16, 2002 Ryan et al.
6460036 October 1, 2002 Herz
6473752 October 29, 2002 Fleming
6484164 November 19, 2002 Nikolovska et al.
6490575 December 3, 2002 Berstis
6505191 January 7, 2003 Baclawski
6546388 April 8, 2003 Edlund et al.
6571234 May 27, 2003 Knight et al.
6581056 June 17, 2003 Rao
6583798 June 24, 2003 Hoek et al.
6587856 July 1, 2003 Srinivasan et al.
6602300 August 5, 2003 Ushioda et al.
6633868 October 14, 2003 Min et al.
6651054 November 18, 2003 De Judicibus
6665666 December 16, 2003 Brown et al.
6687704 February 3, 2004 Russell
6697799 February 24, 2004 Neal et al.
6697840 February 24, 2004 Godefroid et al.
6745178 June 1, 2004 Emens et al.
6766320 July 20, 2004 Wang et al.
6772188 August 3, 2004 Cloutier
6778951 August 17, 2004 Contractor
6785671 August 31, 2004 Bailey et al.
6795825 September 21, 2004 Rishe
6803906 October 12, 2004 Morrison et al.
6820093 November 16, 2004 de la Huerga
6820237 November 16, 2004 Abu-Hakima et al.
6834287 December 21, 2004 Folk-Williams et al.
6850934 February 1, 2005 Bates et al.
6853998 February 8, 2005 Biebesheimer et al.
6874126 March 29, 2005 Lapidous
6925456 August 2, 2005 Nakano et al.
6948134 September 20, 2005 Gauthier et al.
6950791 September 27, 2005 Bray et al.
6961910 November 1, 2005 Lee et al.
6961954 November 1, 2005 Maybury et al.
6963830 November 8, 2005 Nakao
6976053 December 13, 2005 Tripp et al.
6976090 December 13, 2005 Ben-Shaul et al.
7007085 February 28, 2006 Malik
7022905 April 4, 2006 Hinman et al.
7027975 April 11, 2006 Pazandak et al.
7032174 April 18, 2006 Montero et al.
7039650 May 2, 2006 Adams et al.
7043492 May 9, 2006 Neal et al.
7054860 May 30, 2006 Inaba et al.
7054870 May 30, 2006 Holbrook
7062442 June 13, 2006 Berg et al.
7082428 July 25, 2006 Denny et al.
7099860 August 29, 2006 Liu et al.
7146399 December 5, 2006 Fox et al.
7162473 January 9, 2007 Dumais
7171352 January 30, 2007 Chang et al.
7181459 February 20, 2007 Grant et al.
7194455 March 20, 2007 Zhou et al.
7194485 March 20, 2007 Kaipa et al.
7231395 June 12, 2007 Fain et al.
7293014 November 6, 2007 Subramaniam et al.
7293015 November 6, 2007 Zhou
7305129 December 4, 2007 Chellapilla et al.
7318049 January 8, 2008 Iannacci
7412708 August 12, 2008 Khan et al.
7421645 September 2, 2008 Reynar
7437353 October 14, 2008 Marmarmos et al.
7451136 November 11, 2008 Chua et al.
7478089 January 13, 2009 Henkin et al.
7516118 April 7, 2009 Badros et al.
20010037328 November 1, 2001 Pustejovsky et al.
20010037377 November 1, 2001 Nakano et al.
20020016786 February 7, 2002 Pitkow et al.
20020040311 April 4, 2002 Douglass et al.
20020059272 May 16, 2002 Porter
20020065800 May 30, 2002 Morlitz
20020069194 June 6, 2002 Robbins
20020095427 July 18, 2002 Kaplan
20020099700 July 25, 2002 Li
20020103698 August 1, 2002 Cantrell
20020103737 August 1, 2002 Briere
20020103806 August 1, 2002 Yamanoue
20020116291 August 22, 2002 Grasso et al.
20020129059 September 12, 2002 Eck
20020174101 November 21, 2002 Fernley et al.
20030014398 January 16, 2003 Ohta
20030020749 January 30, 2003 Abu-Hakima et al.
20030033296 February 13, 2003 Rothmuller et al.
20030046311 March 6, 2003 Baidya et al.
20030055831 March 20, 2003 Ryan et al.
20030069877 April 10, 2003 Grefenstette et al.
20030079185 April 24, 2003 Katariya et al.
20030093276 May 15, 2003 Miller et al.
20030093790 May 15, 2003 Logan et al.
20030115552 June 19, 2003 Jahnke et al.
20030123443 July 3, 2003 Anwar
20030130982 July 10, 2003 Kasriel et al.
20030135490 July 17, 2003 Barrett et al.
20030135499 July 17, 2003 Schirmer et al.
20030154071 August 14, 2003 Shreve
20030158855 August 21, 2003 Farnham et al.
20030167266 September 4, 2003 Saldanha et al.
20030208493 November 6, 2003 Hall et al.
20030220913 November 27, 2003 Doganata et al.
20040001104 January 1, 2004 Sommerer et al.
20040003097 January 1, 2004 Willis et al.
20040030741 February 12, 2004 Wolton et al.
20040036716 February 26, 2004 Jordahl
20040059564 March 25, 2004 Zhou
20040064447 April 1, 2004 Simske et al.
20040068486 April 8, 2004 Chidlovskii
20040073534 April 15, 2004 Robson
20040122656 June 24, 2004 Abir
20040133560 July 8, 2004 Simske
20040139106 July 15, 2004 Bachman et al.
20040143569 July 22, 2004 Gross et al.
20040225667 November 11, 2004 Hu et al.
20040267700 December 30, 2004 Dumais et al.
20040267730 December 30, 2004 Dumais et al.
20040267813 December 30, 2004 Rivers-Moore et al.
20050065909 March 24, 2005 Musgrove et al.
20050114306 May 26, 2005 Shu et al.
20050125382 June 9, 2005 Karnawat et al.
20050125390 June 9, 2005 Hurst-Hiller et al.
20050187920 August 25, 2005 Tenembaum et al.
20050198026 September 8, 2005 Dehlinger et al.
20050222987 October 6, 2005 Vadon
20050262073 November 24, 2005 Reed
20060010150 January 12, 2006 Shaath et al.
20060136405 June 22, 2006 Ducatel et al.
Other references
  • U.S. Appl. No. 10/749,440, filed Dec. 31, 2003, Badros et al.
  • 80-20 Software—Products—80-20 One Search, http://www.80-20.com/products/one-search/retriever.asp printed Mar. 16, 2004.
  • “askSam™ Making Information Useful,” askSam,—Organize your Information with askSam, http://www.asksam.com/brochure.asp, printed Mar. 15, 2004.
  • Alexa® Web Search—Toolbar Quick Tour, http://pages.alexa.com/prod—serv/quicktour.html pp. 1-5, printed Mar. 16, 2004.
  • Barrett, R. et al., “How to Personalize the Web,”IBM Research, http://www.almaden.ibm.com/cs/wbi/papers/chi97/wbipaper.html, pp. 1-13, printed Mar. 16, 2004.
  • Battelle, J., CNN.com “When geeks go camping, ideas hatch,” http://www.cnn.com/2004/TECH/ptech/01/09/bus2.feat.geek.camp/index.html, pp. 1-3, printed Jan. 13, 2004.
  • Boyan, J., et al., “A Machine Learning Architecture for Optimizing Web Search Engines,” School of Computer Science, Carnegie Mellon University, May 10, 1996, pp. 1-8.
  • Bradenbaugh, F., “Chapter 1 The Client-Side Search Engine,” JavaScript Cookbook, 1st Ed., Oct. 1999, O'Reilly™ Online Catalog, http://www.oreilly.com/catalog/iscook/chapter/ch01.html, pp. 1-30, printed Dec. 29, 2003.
  • Brin, S., et al, “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” http://www7.scu.edu.au/programme/fullpapers/1921/com1921.htm, pp. 1-18, 1998.
  • Budzik, J., et al., User Interactions with Everyday Applications as Context for Just-in-time Information Access, Intelligent Information Laboratory, Northwestern University. pp. 1-8, no date.
  • DEVONthink, http://www.devon-techonologies.com/products/devonthink.php, printed Mar. 16, 2004.
  • dtSearch®—http://www.dtsearch.com/, printed Mar. 15, 2004.
  • Dumais. S., et al, “Stuff I've Seen: A System for Personal Information Retrieval and Re-Use,” Microsoft Research, SIGIR'03, Jul. 28-Aug, 1, 2003, pp. 1-8.
  • Enfish, http://www.enfish.com printed Mar. 16, 2004.
  • Fast Search & Transfer—Home—Enterprise Search, http://solutions.altavista.com/en/news/pr—020402—desktop,shtmu, printed Mar. 16, 2004.
  • Fertig, S., et al., “Lifestream : An Alternative to the Desktop Metaphor,” http://www.acm.org/sigchi/chi98/proceedings/videos/Fertig/etf.htm, pp. 1-3, printed Mar. 16, 2004.
  • Geisler, G., “Enriched Links: A Framework for Improving Web Navigation Using Pop-Up Views,” pp. 1-14, 2000.
  • ISYS Search Software—ISYS: desktop, http://www.isysusa.com/products/desktop/index.html, printed Mar. 16, 2004.
  • Joachims, T., et al., “WebWatcher: A Tour Guide for the World Wide Web,” 1996.
  • Markoff, J., “Google Moves Toward Clash with Microsoft,” The New York Times, May 19, 2004. http://www.nytimes.com/2004/5/19/technology/19google.html?ex=1085964389&ei=1&e.... pp. 1-4, printed May 19, 2004.
  • Naraine, R., “Future of Search Will Make You Dizzy,” Enterprise, May 20, 2004, http://www.internetnews.com/ent-news/article.php/3356831, pp. 1-4, printed May 21, 2004.
  • “Overview,” Stuff I've Seen—Home Page, http://www.research.Microsoft.com/adapt/sis/index.htm, pp. 1-2, printed May 26, 2004.
  • Rhodes, B., “Margin Notes Building a Contextually Aware Associative Memory,” The Proceedings of the International Conference on Intelligent User Interfaces (IUI'00). Jan. 9-12, 2000.
  • Rhodes, B., et al., “Just-in-time information retrieval agents,” Systems Journal, vol. 39, Nos. 3&4, 2000, pp. 685-704.
  • Rhodes, B., et al., “Remembrance Agent—A continuously running automated information retrieval system,” The Proceedings of the First International Conference on the Practical Application of Intelligent Agents and Multi Agent Technology (PAAM '96), pp. 487-495.
  • Rizzo, T., “WinFS 101: Introducing the New Windows File System,” Longhorn Developer Center Home: Headline Archive: WinFS 101: Introducing the New . . ., http://msdn.Microsoft.com/Longhorn/archive/default.aspx?pull+/library/en-us/dnwinfs/htm..., pp. 1-5, printed Apr. 21, 2004.
  • “Searching for the next Google—New trends are helping nimble startups elbow in to the plundered market,” Red Herring—The Business of Technology, Mar. 9, 2004, http://redherring.com/PrintArticle.aspx?a=4782§or=Capital, p. 1-5, printed Mar. 30, 2004.
  • “Selecting Task-Relevant Sources for Just-In-Time Retrieval” pp. 1-3, no date.
  • Sherman, C., “HotBot's New Desktop Search Toolbar,” www.searchenginewatch.com, http://searchenginewatch.com/searchday/print.php/34711—339921, pp. 1-3, printed Apr. 14, 2004.
  • “Standardization Priorities for the Directory—Directory Interoperability Forum White Paper,” The Open Group, Dec. 2001, pp. 1-21.
  • Sullivan, D., “Alta Vista Releases Search Software,” The Search Engine Report, Aug. 4, 1998, pp. 1-2.
  • WebWatcher Home Page “Welcome to the WebWatcher Project,” http://www-2.cs.cmu.edu/˜webwatcher/, printed Oct. 15, 2003.
  • “WhenU Just-In-Time Marketing,” http://www.whenu.com printed Mar. 19, 2004.
  • X1 instantly searches files & email. For outlook, Outlook, http://www.x1.com/, printed Mar. 15, 2004.
  • Zellweger, P., et al., “Fluid Links for Informed and Incremental Link Transitions,” Proceedings of Hypertext'98. Pittsburgh, PA, Jun. 20-24, 1998, pp. 50-57.
  • Berlin, J., et al., “Database Schema Matching Using Machine Learning with Feature Selection,” CAISE 2002, LNCS 2348, pp. 452-466, http://www.springerlink.com/contant/73u6cpt0qek8rqh0/.
  • Brill, E., “A Simple Rule-Based Part of Speech Tagger,” Department of Computer Science, University of Pennsylvania, 1992, pp. 1-5.
  • Claypool, M., et al., “Inferring User Interest,” IEEE Internet Computing, 2001, pp. 1-17, vol. 5, No. 6, located at http://web.cs.wpi.edu/˜claypool/papers/iui/iui.pdf.
  • Czerwinski, M., et al., “Visualizing Implicit Queries for Information Management and Retrieval,” ACM CHI '99, May 15-20, 1999, pp. 560-567.
  • Knezevic, P. et al., “The Architecture of the Obelix—An Improved Internet Search Engine,” Proceedings of the 33rd Annual Hawaii International Conference on System Sciences (HICSS) Jan. 4-7, 2000, Maui, HI, USA, pp. 2145-2155.
  • Li, W., et al., “Semantic Integration in Heterogeneous Databases Using Neural Networks,” Proceedings of the 20th International Conference on Very Large Data Bases, Sep. 12-15, 1994, pp. 1-12, Morgan Kaufmann Publishers, San Francisco, CA.
  • Li, W., et al., “Semint: A Tool for Identifying Attribute Correspondences in Heterogeneous Databases Using Neural Networks,” Data Knowl. Eng., Apr. 2000, pp. 484, vol. 33, No. 1, http://dx.doi.org/10.1016-/S0169-023X(99)00044-0.
  • Morita, M. et al., “Information Filtering Based on User Behavior Analysis and Best Match Text Retrieval,” Proceedings of the Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, Dublin, Jul. 3-6, 1994, pp. 272-281.
  • Phelps, A., “All You Can Seek,” Special Services, Jul. 1999, vol. 7, Iss. 7, [online] [Retrieved on Oct. 16, 2006] Retrieved from the Internet: http://www.smartcomputing.com/editorial/article.asp?article.articles/archive/g0707/26g07/26g07.asp.
  • Scha, R., et al., “An Augmented Context Free Grammar for Discourse,” Proceedings of the 12th Conference on Computational Linguistics—vol. 2, Computational Linguistics, Aug. 22-27, 1988, pp. 573-577, Morristown, NJ, http://dx.doi.org/10.3115/991719.991756.
  • International Search Report and Written Opinion, PCT/US2004/038562, Apr. 6, 2005, 12 pages.
  • Garofalakis, M., et al., “XTRACT: A System for Extracting Document Type Descriptors from XML Documents,” SIGMOD, ACM, Jun. 2000, pp. 165-176, vol. 29, No. 2.
  • Horvitz, E., et al., “The Lumiere project: Bayesian user modeling for inferring the goals and needs of software users”, Proceedings of the Fourteenth Conference on Uncertainty, 1998, pp. 256-265, Morgan Kaufmann: San Francisco.
  • Joho, H., et al., “A Study of User Interaction with a Concept-Based Interactive Query Expansion Support Tool,” Advances in Information Retrieval, A Study of User Interaction, Lecture Notes in Computer Science, Mar. 2, 2004, pp. 42-56, vol. 2997.
  • Jones, G., et al., “Context-Aware Retrieval for Ubiquitous Computing Environments,” Mobile and Ubiquitous Information Access, Lecture Notes in Computer Science, Jan. 27, 2004, pp. 227-243, vol. 2954.
  • Lieberman, H., “Letizia: An Agent That Assists Web Browsing,” AAAI Technical Report FS-95-03, 1995, pp. 97-102.
  • Shepherd, M., et al., “Browsing and Keyword-Based Profiles: A Cautionary Tale,” Proceedings of the 34th Hawaii International Conference on System Sciences, Jan. 3-6, 2001, pp. 1365-1373.
  • White, R., et al., “The Use of Implicit Evidence for Relevance Feedback in Web Retrieval,” Lecture Notes in Computer Science, Jan. 1, 2002, pp. 93-109, vol. 2291.
Patent History
Patent number: 8131754
Type: Grant
Filed: Jun 30, 2004
Date of Patent: Mar 6, 2012
Assignee: Google Inc. (Mountain View, CA)
Inventors: Stephen R. Lawrence (Mountain View, CA), Niniane Wang (Santa Clara, CA)
Primary Examiner: Usmaan Saeed
Assistant Examiner: Navneet K Ahluwalia
Attorney: Fenwick & West LLP
Application Number: 10/882,066
Classifications
Current U.S. Class: Analyzing Or Parsing Query To Determine Topic Or Category (707/771); Record, File, And Data Search And Comparisons (707/758)
International Classification: G06F 17/30 (20060101); G06F 7/00 (20060101);