System and Methods to use context graphs for targeting communications

Abstract

This disclosure provides a system and method to use context graphs for targeting communications to a user of an image rendering device. According to one exemplary embodiment, the method provides targeted communications generated as a function of one or more attributes associated with a user requested printed document. The attributes are provided by accessing a context graph including a plurality of links between a plurality of entities.

Description

This application claims the benefit of priority to U.S. Provisional Application No. 61/345,289, filed May 17, 2010, entitled “SYSTEM AND METHODS TO USE IN-PARTITE SCALE-FREE GRAPHS FOR INTERPRETING CONTEXTUAL INFORMATION AND TARGETING,” by Gnanasambandam et al., U.S. Provisional Application No. 61/345,340, filed May 17, 2010, entitled “OPTIMAL AUCTION MECHANISM FOR MULTI-LEVEL DEVICE CLICK-THROUGH (DCT) IN TARGETED PRINT COMMUNICATION,” by Lee et al., U.S. Provisional Application No. 61/345,301, filed May 17, 2010, entitled “SYSTEM AND METHOD TO PRODUCE AND CONTROL SUBSIDIZATION OF TARGETED MATERIALS AT POINT OF SALE,” by Gnanasambandam et al., and U.S. Provisional Application No. 61/345,377, filed May 17, 2010, entitled “METHOD FOR IDENTIFICATION, CATEGORIZATION AND SEARCH OF GRAPHICAL, AUDITORY AND COMPUTATIONAL PATTERN ENSEMBLES,” by Gnanasambandam et al., each of which is hereby incorporated by reference in their entirety.

CROSS REFERENCE TO RELATED PATENTS AND APPLICATIONS

• U.S. patent application Ser. No. 13/100,636, filed May 4, 2011, entitled “Method of Providing Targeted Communications to a User of a Printing System,” by Lee et al.
• U.S. patent application Ser. No. 13/104,136, filed May 10, 2011, entitled “System and Method to Produce and Control Subsidization of Targeted Materials at Point of Sale,” by Gnanasambandam et al.
• U.S. patent application Ser. No. ______, filed ______, entitled “Method for Identification, Categorization and Search of Graphical, Auditory and Computational Pattern Ensembles,” by Gnanasambandam et al., each of which is hereby incorporated by reference in their entirety.

BACKGROUND

This disclosure relates generally to a system and method for generating targeted advertisements, and more particularly to a system and method for generating targeted advertisements delivered via a printer assembly including at least one printer device. It is currently known for advertisers to use publicly available information for targeting potential and publicly available clients. Additionally, advertisers may use the publicly available information for advertising merchandise that is likely to be of interest to a particular person.

INCORPORATION BY REFERENCE

The following patent applications, Patent Application Publications and Non-patent references are incorporated herein by reference in their entirety:

• Gnanasambandam et al., U.S. patent application Ser. No. ______, filed ______, entitled “System and Method to Produce and Control Subsidization of Targeted Materials at Point of Sale”;
• Gnanasambandam et al., U.S. patent application Ser. No. ______, filed ______, entitled “Method for Identification, Categorization and Search of Graphical, Auditory and Computational Pattern Ensembles”;
• Lee et al., U.S. patent application Ser. No. 13/100,636; filed May 4, 2011, entitled “Method of Providing Targeted Communications to a User of a Printing System”;
• Gnanasambandam et al., U.S. patent application Ser. No. 12/780,543, filed May 14, 2010, entitled “System and Method to Prearrange Hyper-Local Value-Added Marketing Campaigns and Communication Along Consumer Trajectories”;
• Gnanasambandam, U.S. patent application Ser. No. 12/780,267, filed May 14, 2010, entitled “In-Situ Mobile Application Suggestions and Multi-Application Updates Through Context Specific Analytics”;
• Gnanasambandam et al., U.S. Patent Application Publication No. US 2010/0088178 A1, published Apr. 8, 2010, entitled “System and Method for Generating and Verifying Targeted Advertisements Delivered Via a Printer Device”;
• Gross, U.S. Patent Application Publication No. US 2010/0005486 A1, published Jan. 7, 2010, entitled “Apparatus and Method for Embedding Commercials”;
• Evevsky, U.S. Patent Application Publication No. US 2009/0313060 A1, published Dec. 17, 2009, entitled “System and Method for Personalized Printing and Facilitated Delivery of Personalized Campaign Items”;
• Chow et al., U.S. Patent Application Publication No. US 2009/0157650 A1, published Jun. 18, 2009, entitled “Outbound Content Filtering Via Automated Inference Detection”;
• Chow et al., U.S. Patent Application Publication No. US 2009/0150365 A1, published Jun. 11, 2009, entitled “Inbound Content Filtering Via Automated Inference Detection”;
• Gnanasambandam, U.S. patent application Ser. No. 12/761,985, filed Apr. 16, 2010, entitled “System and Method for Providing Feedback for Targeted Communications”;
• Liu, U.S. Patent Application Publication No. US 2011/0096354 A1, published Apr. 28, 2011, entitled “System and Method for Handling Print Requests from a Mobile Device”;
• Liu et al., U.S. Patent Application Publication No. US 2011/0040823 A1, published Feb. 17, 2011, entitled “System and Method for Communicating with a Network of Printers Using a Mobile Device”;
• Harrington, U.S. Patent Application Publication No. US 2011/0029952 A1, published Feb. 3, 2011, entitled “Method and System for Constructing a Document Redundancy Graph”;
• Gnanasambandam et al., U.S. Patent Application Publication No. US 2010/0325422 A1, published Dec. 23, 2010, entitled “System and Method for Policy-driven File Segmentation and Intercloud File Storage and Retrieval”;
• Partridge et al., U.S. Patent Application Publication No. US 2010/0309503 A1, published Dec. 9, 2010, entitled “Method and System for Printing Documents from a Portable Device”;
• Gnanasambandam et al., U.S. Patent Application Publication No, US 2010/0268591 A1, published Oct. 21, 2010, entitled “System and Method for Selectively Controlling the Use of Functionality in One or More Multifunction Devices and Subsidizing Their Use Through Advertisements”;
• Gnanasambandam et al., U.S. Patent Application Publication No. US 2010/0264214 A1, published Oct. 21, 2010, entitled “Method and System for Providing Contract-free ‘pay-as-you-go’ Options for Utilization of Multi-function Devices”;
• St. Jacques, Jr. et al., U.S. Patent Application Publication No. US 2010/0149572 A1, published Jun. 17, 2010, entitled “Method and System for Automatically Providing for Multi-point Document Storing, Access, and Retrieval”;
• Edelman, B. and M. Schwarz “Optimal Auction Design in a Multi-Unit Environment: The Case of Sponsored Search Auctions,” Unpublished manuscript, Harvard Business School 2006;
• Edelman, B., Ostrovsky, M., and Schwarz, M., “Internet Advertising and the Generalized Second-Price Auction: Selling Billions of Dollars Worth of Keywords,” American Economic Review, American Economic Association, Vol. 97(1), pages 242-259, March 2007; Google AdWords Select, 2002 https://www.google.co/accounts/ServiceLogin?service=adwords&hl;
• Myerson, R. B. “Optimal Auction Design,” Mathematics of Operations Research, 6(1), pages 58-73, 1981;
• Shazam Entertainment Ltd., 2002, http://www.shazam.com/music/web/pages/background.html,
• Talluri, K. and van Ryzin, G., “The Theory and Practice of Revenue Management,” Publisher Springer, 1^st Edition, Feb. 23, 2005;
• Ulku, L. “Optimal Combinatorial Mechanism Design,” Unpublished Manuscript, Rutgers University, 2006;
• Walsh, T., “Search on High Degree Graphs,” In Proceedings of IJCAI-2001, pages 266-271, 2001.

BRIEF DESCRIPTION

In one embodiment of this disclosure, described is a method of providing user requested printed material and one or more targeted communications to a user of a printing system, the method comprising the printing system acquiring material to be printed on a printing device to produce the user requested printed material, the printing device associated with the printing system; the printing system acquiring the one or more targeted communications associated with the user requested printed material; and, the printing system printing the user requested printed material utilizing the printing device and providing the acquired one or more targeted communications to the user, wherein the one or more targeted communications are generated as a function of one or more attributes associated with one or more of the user requested printed material the printing device and the user, the one or more attributes provided by a context graph including a plurality of links between a plurality of entities related to one or more of the user requested printed material, the printing device and the user.

In another embodiment, described is a printing system for providing user requested printed material and one or more targeted communications to a user of the printing system comprising a printing device; and, one or more servers operating connected to the printing device, wherein one or more of the printing device and the one or more servers are configured to execute a method comprising the printing system acquiring material to be printed on the printing device to produce the user requested printed material; the printing system acquiring the one or more targeted communications associated with the user requested printed material; and, the printing device printing the user requested printed material and the printing device providing the acquired one or more targeted communications to the user, wherein of one or more attributes associated with one or more of the user requested printed material, the printing device and the user, the attributes provided by a context graph including a plurality of links between a plurality of entities related one or more of the user requested printed material, the printing device and the user.

In still another embodiment of this disclosure, described is a method of providing a targeted communication to a first image rendering device operatively connected to one or more of a second image rendering device and comprising the image rendering device acquiring the target communication from one of the second image rendering device and the server; and the image rendering device rendering the target communication on the first image rendering device, wherein the targeted communication is generated as a function of one or more attributes associated with one or more of the first image rendering device, and a user associated with the first image rendering device, the one or more attributes provided by a context graph including a plurality of links between a plurality of entities related to one or more of the first image rendering device, the second image rendering device, and the user associated with the first image rendering device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1 and 2 are tripartite context graphs according to an exemplary embodiment of this disclosure.

FIG. 3 is a 5-partite context graph according to an exemplary embodiment of this disclosure.

FIG. 4 illustrates keyword interconnections associated with k1, k2, k3, k4, k5, k6, k7, k8, k9 and k10 as shown in the context graph of FIG. 3.

FIG. 5 is an exemplary embodiment of a context graph showing interactions among entities, for example, but not limited to, keywords and devices.

FIG. 6 shows a power-law in degree distribution of a context graph.

FIG. 7 shows un-weighted context graph A including a scaling exponent of 2.58.

FIG. 8 shows a weighted context graph A including a scaling exponent of 1.61.

FIG. 9 shows an un-weighted context graph B including a scaling exponent 2.6.

FIG. 10 shows a weighted context graph B including a scaling exponent 2.29.

FIG. 11 shows a sub-graph including a sequence of edges.

FIG. 12 shows a sub-graph including a loop.

FIG. 13 shows a sub-graph including a single-hop hub where k₃ is the origin.

FIG. 14 shows a sub-graph including a two hop hub where k₃ is the origin.

FIG. 15 shows a sub-graph including a 5-partite graph where k₃ is the origin.

FIG. 16 schematically illustrates a system for context graph based targeting using a MFD according to an exemplary embodiment of this disclosure.

FIG. 17 shows a targeted communication produced according to an exemplary embodiment of this disclosure, the targeted communication including text ads with graphics from two of the advertisers.

FIG. 18 shows another targeted communication produced according to an exemplary embodiment of this disclosure, the targeted communication including categorized coupons.

FIG. 19 schematically illustrates Transactional Print Workflow according to an exemplary embodiment of this disclosure.

DETAILED DESCRIPTION

This disclosure provides systems and methods of using n-partite graphs in determining contextual information that can be used to characterize a user. Such characterizations can be used for personalization and targeting of information, such as advertisements, to the user. Examples for the nodes of the graph include keywords extracted from the titles of documents being printed, the user who prints the document and the device on which it is printed. Graph edges are entered connecting the document key words to the user and the user to the printing device, but other edges such as a connection between key words that occur together in a title can be entered as well. The disclosed systems and methods maintain a large graph that captures all the activities (e.g. document printing), but can extract sub-graphs for a user and uses the information within the sub-graph to target advertisements to the user.

Conventional targeting techniques have been largely keyword based or spatio-temporal without specific attention to expressivity that an ordering or structure of events can bring to the effectiveness of targeting. Extracting an expressive sub-graph from a large graph (e.g. >10̂6 nodes and 10̂9 edges) from an n-partite graph of events has not been well studied.

Capturing multi-dimensional tuples of information and the interconnections between events is structurally complex. More so because a large number of users are causing or contributing to these users directly or through the usage of devices (mobile, print or otherwise). These can be captured coherently by graph theoretical structures.

More information can be gained if the structure of these complex interconnections (i.e. in the way contextual information gets related amongst users, devices, locations, time and other attributes) is analyzed dynamically. This disclosure also discloses processes to perform that analysis.

In this disclosure, a method is provided wherein graph theoretic structures in a large graph or sub-graph thereof are utilized to express certain correlated user activities and/or event correlations so that they can enhance targeted communications to the users. In other words, graphical structures such as sub-graphs—chains of related nodes, loops, paths etc. and statistics on top these structures—degrees, hop-distance, edge weight, edge direction etc. can capture certain inherent connectivity and association related properties in a more fine grained fashion. These structures or the properties that govern them can be used to relate how the users interact with their environments and/or social groups. Of particular interest, at least, are the following types of contextual information and their n-partite graphical representation when a person interacts with a device: the person, keywords, categories or topics that relate such an interaction, social group of the person, spatio-temporal aspects of the interaction, usage frequency and combinations thereof. The disclosed system and method capture and store these associations in real-time and represents them as n-partite graphs. Subsequently, sub-graphs are extracted dynamically according to a method described herein. The overall context graph resulting from the collection and storage of the aforementioned attributes is huge e.g. millions of vertices and edges. Also provided is a method to characterize the context graph and methods to use the graph with the aforementioned targeted communication application.

Complex graphs can capture interactions between events and contextual information that relate to a large user-population as and when the users interact with the environment and the multitude of devices contained in them, for example, but not limited to, phones, e-readers, printers, kiosks and the like. The scale of the interactions while huge can depict some underlying structure. As will be described below, provided are some examples of such interactions exhibiting specific graph theoretic properties, such as the presence of hubs, scale-free nature, etc. Such properties and other structural elements can be utilized for targeting users with information and marketing material of various kinds.

Examples of Targeting Context Derived from Devices and Processes

Mobile, Phone, E-reader Devices, Printing Devices, Interconnected Printing Device (or MFD) Workflows etc.: Meta-data associated with the use of a device or a multitude of devices can provide targeted communications. While some devices have access to global positioning system coordinates (latitude, longitude), and cell/base station, others have sensor readings (accelerometer, magnetometer etc), and context from documents/web applications (or apps). One or more subsets of these can be utilized for automatically providing meaningful information to the users of the devices and processes.

Printed documents: When users print to devices (or use other types of devices in myriad ways), there is a lot of unused context. In the print world, this context includes keywords in the document name, location, time, user_id, device name, device capabilities used and such. Furthermore, when a group of people utilize a group of devices, there is yet another social dimension to context—such as the user names of all people printing to a given device, document details and keywords contained in document names, preferred printing times of people and such.

Transactional Prints: Credit card statements are one example of transactional documents. These contain not just spatio-temporal information such as when and where the goods and services were purchased and used, but also the relative importance of such goods to the users. This is valuable information which can be characterized as certain kinds of purchases or user behavior being adjacent to each other in time or space.

The above applications and combinations thereof are areas where targeted communications fit in well. However, these are only provided as exemplary embodiments of the present disclosure and it is to be understood that the disclosed system and method is not limited to these applications.

Substantially provided herein, at least, is a (1) system and method to extract, rank and unify sub-graphs from large n-partite graphs; (2) system and method to utilize the extracted sub-graph for targeting; (3) controlling the scalability of the system by search termination strategies and graph properties; and (4) methods to search and aggregate multiple types of targeted communication by dynamically relating meta-data from context graph with a content repository.

Multi-Partite Context Graph

Multi-partite context graphs can represent a multitude of entities (explicitly or inferentially revealed)—including and not limited to events, people, devices, locations, date-time, keywords, categories, preferences etc. These connections between the entity types indicate actions performed by one or more users as dictated by a design choice. Furthermore, the weights of connections of entities indicate frequency of involvement of entities in an action.

For purposes of this disclosure, the meaning of a context graph is provided herein through an example in the document printing world, although the application of context graphs as provided herein is easily extended to other domains. In the printing world, context in the form of keywords, time-stamps, locations and various subject categories, such as automobiles, banks, investments etc., are generated collectively by a group of people by the act of printing, or using other rendering devices. The extracted keywords may be from job names or from document content. These attributes associated with the printed document can be used to illustrate the concept of a context graph as shown in FIG. 1.

With reference to FIG. 1, a tripartite graph is shown including a set of keywords K extracted from attributes associated with the printed document, such as the name of the document and other related information with or without extracting information from the document, a set of people P, and a set of devices D. Each link in the graph shows one of two things—either a person p_i used a keyword k_i or a person used a device d_i. Each unique p_i, k_i or d_i appears only once in the graph. Furthermore, the edge weights are utilized to denote the frequency of the aforementioned semantics relating to the usage.

Associating Meanings with Sub-Graphs and/or the Whole Graph

With reference to FIG. 2, shown is a graph including additional more semantics—if a person used two keywords in one usage of the device, such as one printout, one fax or scan etc., a link is provided between the two keywords as shown in the top layer. Note that FIG. 2 is not the transitive closure of FIG. 1. Notably, such relationships that can be derived with transitive closure are left as a matter of choice and something that may be done on the fly for relatively simple relationships or offline for relatively lengthier chains to which more semantics may be assigned.

This concept of tripartite context graphs is easily extended to n-partite graphs that can describe more semantics in addition to the above. For example, FIG. 3 shows the added set of C and N to FIG. 1, where c_i may indicate the type of usage such as printing or faxing and n_i indicates a feature corresponding to the devices. Graph-based models are chosen for the expressivity for the complex webs of information that can associated with keyword usage and context generation. But as for the term context graphs in the reminder of this disclosure, the disclosure does not differentiate between graphs that are n-partite, their transitive closures or something in between as shown in FIG. 2. FIG. 4 illustrates keyword interconnections associated with k1, k2, k3, k4, k5, k6, k7, k8, k9 and k10 as shown in the context graph of FIG. 3.

Characterizing Context Graph Structure

The structure and properties of a context graph are fundamental to the disclosed system and method, initially explained are some findings related to the graphs examined. Context generation has yielded several large graphs with heavy keyword interactions that are shown in FIG. 5, according to one example. Entities that are not associated with others by the users show up as isolated points. NOTE: Each “o” is a discrete entity.

The example of FIG. 5, as well as others not illustrated, from the device world of printers show the presence of hubs. These hubs show that there are some entities in an action that are heavily connected. For example, characterized are two examples. A first example with ˜2600 entities (˜3.38 million feasible edges) denoted as graph A, and a second example with ˜4200 nodes (˜8.82 million feasible edges) denoted as graph B. The edges further have weights based on the frequency with which pairs of entities, i.e. keywords, devices or both, were selected in combination. The graph as illustrated in FIG. 6, actually exhibits power-law characteristics for its degree distribution i.e. if X represents the random variable degree of a given node, P(X≧x) actually represents a power-law of the form x^−α. More precisely, the power law is believed to have an exponential tail in some cases that causes the behavior to taper off faster at higher degrees.

The power-law coefficients can be separately computed for the weighted (x^−α1) and un-weighted (x^−α2) graphs. A large difference may indicate that there is a significant positive impact that is created by factoring the repetitive social behavior i.e. clustering around common keywords, i.e. jargons, in an organization. For example, FIG. 7 shows un-weighted context graph A including a scaling exponent of 2.58; FIG. 8 shows a weighted context graph A including a scaling exponent of 1.61; FIG. 9 shows an un-weighted context graph B including a scaling exponent 2.6; and FIG. 10 shows a weighted context graph B including a scaling exponent 2.29.

In some cases, such as Graph B, the weighted context graph exhibits an exponential tail as shown in FIG. 10. In other words, P(X≧x) is proportional to x^−α e^−βx where the exponential term overwhelms the power law for a large x. During the analysis of contextual information in an organizational context, it is plausible that the mass usage of jargons and terms trail off more rapidly beyond a certain level of popularity, perhaps due to group size, seasonal effects etc. While it has been observed that such faster than power-law trail off phenomenon does occur, the exact causes are not the subject of this disclosure. This disclosure, and the system and method provided herein leverage the detected properties. One example of the property detected being power-law structure and faster than power-law trail off in regions of the graph.

Method for Sub-Graph Extraction from Context Graph

In this section, described are various ways sub-graphs can be extracted from a context graph to provide targeting. In addition, described are the generation of the context graph and how it is used in real-time.

Step 1: Graph Maintenance

Initially, a context tuple, e.g. a row of comma-separated context, is received by a targeting engine, and each of the entity types in the tuple is recognized by the context graph algorithm associated with the targeting engine. Either explicitly from data or as the result of an inference, the different entities associated with the context tuples are classified. The inference can be based on a machine learning algorithm, for example a known supervised classification technique such as those based on Support Vector Machines or Singular Value Decomposition. After the context tuple's various entities are recognized, they are assigned to one of the n layers of the n-partite graph and edges and weights are updated.

Further semantics may be assigned as an extension for each given tuple. The direction of each edge may be altered given the transaction's dependency relationships or ordering. It is the responsibility of the tuple provider to specify the directionality or ordering of the entities at the time the context is provided for producing a targeted message. Each side of edge has a directionality probability which is an aggregate of the set (or any reasonable sub-set) of users assigned to the graph. So an edge directionality probability is denoted e(p_a, p_b, p_ab) where p_a+p_b+p_ab=1, and {a,b} refers to the edge between node a and node b. The graph itself is stored on any scalable platform such as a distributed database. According to one exemplary embodiment, a non-relational distributed database, namely Hbase can be used.

Step 2: Retrieving a Sub-graph and Finding the Dominant Sub-graph

Various graphical structures such as sub-graphs—chains of related nodes, loops, paths, hubs etc. & statistics on top of these structures—degrees, hop-distance, edge weight, edge direction etc. can capture certain inherent connectivity and association related properties in a fine grained fashion. These structures or the properties that govern them relate to how the users interact with their environments and/or social groups, as it is the activities that are utilized to make the graph in the first place. Of particular interest are at-least the following types of contextual information and their n-partite graphical representation associated with a person interacting with a device: the person, keywords, categories or topics that relate such an interaction, social group of the person, spatio-temporal aspects of the interaction, usage frequency and combinations thereof.

Retrieving a sub-graph relates to extracting a relative smaller graph, i.e. sub-graph, where the keyword from the query is contained in a designated portion of the sub-graph e.g. center, 2^nd layer etc. Such a keyword is called the origin (w.r.t. to the sub-graph retrieval). According to one exemplary embodiment, the graphical substructures shown in FIGS. 10-15 are of particular interest without any limitations on other substructures that may be added.

Finding a dominant sub-graph relates to ranking the retrieved sub-graphs to find the most suitable sub-graph. Essentially a metric is computed with the properties of these sub-graphs to determine a quantitative ranking. The metric can be based on degree, edge weight, edge direction, number of nodes in a loop, hop count etc. Other constraints can be associated with a specific targeting purpose. Usually one of the keywords, denoted the origin, will be contained in the sub-graphs. If multiple keywords are part of a query, then multiple sub-graphs are retrieved iteratively. The resulting set of dominant sub-graphs are then merged based on common nodes. If there is no common node between a pair of origin keywords, the dominant sub-graphs are kept as disjoint pieces at this stage.

Step 3: Utilizing Structure of the Overall Context Graph and Sub-Graphs

For sub-graphs that do not have any overlapping entities, the structure of the overall graph is used. The overall context graph is known to exhibit hubs as previously discussed with regard to the power-law structure. Therefore, the strategy is to seek out hubs to connect the seemingly disjoint sub-graphs. The overall context graph is also known to exhibit other characteristics around the path lengths between nodes Notably, diameter and path lengths can be characterized as other relevant metrics.

Initially, the nearest hub associated with the sub-graph is determined. In most cases, this will be just a few hops away because of the scale-free structure. In other cases, the search is terminated owing to intractability. In this case, both sub-graphs are used to identify the nearest hub(s). The hubs may have some latent relationship with the sub-graphs. The revised unified sub-graph is the two sub-graphs joined by the hub-node. In this way, two seemingly dissimilar entities such as Malaria and CNN for example, can be connected, perhaps because of the airing of a Malaria program on CNN. Such connections become important when two sub-graphs seem very different. Notably, the above strategy is only done to discover a set of latent connections between sub-graphs.

Often owing to the scale of the context graph stopping criteria for search and traversal becomes important. The following are some conditions that may be used: hop-count from origin (most common); reversal in edge direction, hitting a hub, accrued edge weight along direction of traversal, number of layers of the graph traversed etc. Notably, sometimes Step 3 may be skipped without any loss of continuity to the method.

Step 4: Using Sub-Graph Outputs for Targeting

Once the sub-graphs are unified or kept disparate, a query is constructed and sent to an aggregator of information. The query may be broken into sub-queries to reflect the different ways to offer importance to the structure of the sub-graph(s) identified. The simplest way is to send the high-degree nodes from the sub-graph along with the most recent input that was received for targeting. Another way is to send all the nodes in the sub-graph in a sequence and to prioritize the results that contain the origin entities, e.g. keywords. The sub-graph will dictate the order of the sub-queries that are iteratively made to the aggregator.

Step 5: Retrieving Relevant Documents using Sub-graph Output

In the previous step, mentioned was sending a query, possibly composed of sub-queries, to an aggregator of information, e.g. coupons, advertisements, news feeds etc. Checking the relevance of retrieved documents is performed in this step. Relevance is performed by counting the number of entities that exist in the result and matching it with the sub-graph that is delineated in Steps 2-3. For result documents that are in text form the relevance is computed directly by keyword matches. For image/audio/video results the relevance is computed on meta-data, tags, transcriptions, translations etc. The filtered and ranked results are returned to the user in the format of choice. An alternate way is to use the redundancy observed in retrieved documents for the purpose of ranking. See U.S. patent application Ser. No. 12/533,901, by Harrington, entitled ‘method and system for constructing a document redundancy graph,” filed Jul. 31, 2009.

Implementation Details

In this section, briefly outlined is how to build the context graph and how to search and maintain it. Explained also is how an incoming job can be monetized with advertisements and credit is stored for various verification actions by the user. In the development of this system, utilized was the open source Hadoop/MapReduce/Hbase framework from the Apache Foundation to aid in computation and prototyping.

“Printing” is an overloaded term and can be meant to interpret sending a document or piece of information from one device to another.

Step 1: User prints (or scans faxes) to/from an MFD. Each MFD receives the metadata associated with the print (or other use). Each MFD is usually a data node of the Hadoop cluster. Upon receipt of metadata, the MFD calls contextual_ad_fetch( ) which is a remote call to the server component, typically residing with the name-node of the Hadoop file system. The MFD that receives the print job/stream caches the job temporarily so that the advertisements can be fetched.

Step 2: contextual_ad_fetch( ) first calls ContextGraph_Create_Maintain( ). As every user prints (or uses the network of MFDs), the keywords and context identifiers (time of day, location, username etc.) are extracted. Now is introduced an edge of vertex in the graph context graph G(E,V) where E is the set of edges and V is the set of vertices. Note that weights of the edges have to be recomputed as well.

a. For each context item that is extracted, update the n-partite graph. Generally speaking there may be n types of context. In the prototype, assumed was a tripartite graph with the constituent part being keywords, usernames and devices.

b. If any keyword is not present in the graph, it is added as a new vertex. Likewise for username and device.

c. If the keyword is already present, added is an edge between the username that printed it and the keyword printed. Also, an edge is added between the username and the device that the job was printed on. The above process is repeated for all context items that are extracted. Up to this step, nodes and vertices of the tripartite graphs have been created.

d. Next, a link is introduced between each pair of keywords associated with the current job. This means that the user associated those keywords together and attached some semantics to it in current domain of the users' operation. According to this example, only the job name was used for this step. Other similar metadata from the document may be extracted from the document, but only sparingly owing to tractability issues. This creates links in the keyword layer on the fly.

e. Then for each edge that got newly created, the weight is set to one. If there is an edge already between the two vertices under consideration then the weight is increased by one. The latter condition occurs when such a linkage was established by another user or the very same user as the current user in a prior print job. In this way, the graph is created, updated and maintained after every job.

f. From a storage perspective, the graph is stored as co-keywords, co-user-device and co-user-keywords where together with the edge weights, provided is a social context graph. The graph creation overhead is the same as update overhead as the graph creation is done as several iterative updates. Update overhead is estimated as follows. Complexity of finding an item in the ordered list is O(log n). Adding a new element that is not there is O(1). If the element is already there, computing or updating the weights requires another O(log n). This process has to be done for about 5 keywords on average. So the update complexity is 10*O(log n) where n is the total number of keywords.

Step 3: contextual_ad_fetch( ) then calls ad_fetch( ) based upon information in the graph, i.e. history, and those in the current context, i.e. current job. The aim of this step is to maximize the chance of advertisements and then getting the most relevant advertisements. The advertisements could be obtained by contacting an advertisement aggregator. Since it is not a human that is actively querying the advertisement aggregator, it is important to send a list of most relevant context.

Step 4: Once the ads are obtained from the advertisement aggregator, the available real-estate, both electronic and paper, is populated with the best advertisements. The advertisements that essentially have greater relevance, for example, computed relevance based on the occurrence of queried keywords in the advertisements, are essentially populated at more conspicuous areas such as the banner pages and the UI (User Interface) of the device. Once the ad-filled real-estate is prepared, the rest of document that was cached in step 1 is printed along with the ads and/or coupons. According to a prototype, it takes about 2-4 seconds to fetch the targeted ads. An additional couple of seconds may be required to meld the ads with the customer job in novel ways.

Step 5: Some advertisements may also be sent to the users' desktops through the print-driver so that they can be clicked on or used at a later point. Alternately, some advertisements may be cached at the device for use at a subsequent time. If an advertisement is clicked upon at the users' workstations, the advertisement identifier and user are captured and transmitted to the device. The device(s) track and store this information and from time-to-time transmit it to the advertisement aggregator. Printed coupons or other hard copies of advertisements that the users can carry with them to a store or vendor include a barcode, micro-text or glyph embedded in the image, so that a vendor can track the source of the advertisement. These methods enable the tracking of printed or non-printed targeted messages and provide verification to the advertiser or advertisement aggregator.

A diagrammatic representation of one system according to this disclosure is shown in FIG. 16.

Step 1. According to this embodiment, a user 150 prints a document. Printing a document includes using any device that may have a so called “print button” to invoke the print action. Information including name/login of person printing, credentials, location (GPS coordinates), print account details, preferences on the printer, preferences for the document to be printed including number of pages or page ranges, color/monochrome, document content (words and images with the document) are provided when invoking the print operation. The GUI may contain the print button and collect these pieces explicitly according to an opt-in model with respect to the information. Also the information may be stored in a profile page on a server. Content internal to the document that is being submitted for printing may be restricted from use by the customer that owns the print job. Printing can also mean printing to a driver such as, but not limited to, an Adobe PDF Printer which will perform the document conversion service by accepting the print job.

Step 2. A service (on the cloud, network or any other known service such as CUPS—Common Unix Printing Service) takes the printed job and other information from the user that is printing.

Step 3a. Automatic ad fetch and ranking service.

First context is extracted. The service will extract the context (such as the examples about the user, device, location, time, document above) from the print job. From the context it applies some business rules or algorithms to extract the query that will be used for fetching ads automatically. An optimal subset of the context is used for generating a query. This subset is a function of time, location, user, the device, the service provider (i.e. whether the printing service is in the airport vs. mall vs. office vs. home), past history of printing etc. Relevant context is extracted in the form of a subset of data. Many such candidate subset may be unearthed.

Step 3b. This context subset search is facilitated by the context graph that is stored in a database (the well known hBase in our case), where history of prior context are stored as a context graph explained before.

After the subset is extracted, it is sent to an ad-server that will provide some ads.

Step 4. Once the ads are received back by the ad-fetch and ranking service 154, the service performs relevance ranking according to settings and business rules within the ad fetch and ranking service. The results are called relevant advertisements or messages. These are passed back to (152) the print server.

Step 5. (152) This step merges the document to be printed (or converted to other forms if printing really means document conversion) with ads and messages and then sends them to another application. The other application can be, for example, a phone, an email server, an e-book reader or in our embodiment a printer or printer GUI.

Step 6. Depending on the device click-through is defined. Click-through are actions relating to clicks or click-like or view-like activity that is recorded and transmitted to the advertiser that sent the ads to the ads-fetch and ranking service (154). Click through only relates to those individual ads or messages that were acted upon by the user in some way.

List of locations and real-estate (paper and non-paper)

1. Banner page

2. Several Places within Document (margins, sides, interspersing etc.)

3. MFD UI−EIP+Screen

4. Via Print Driver to User Computer

5. Windows Application including browser

6. Mobile Destinations including e-reader, laptops etc.

FIG. 17 shows a targeted communication produced according to an exemplary embodiment of this disclosure, the targeted communication including text ads with graphics from two of the advertisers.

FIG. 18 shows another targeted communication produced according to an exemplary embodiment of this disclosure, the targeted communication including categorized coupons.

FIG. 18 refers to an one embodiment of a personalized ad or coupon listing in a categorized fashion that is again custom generated for the user or customer. FIG. 18 may be printed or may viewed on a GUI or viewing device, and provides coupons or ads as images relating to 4 categories.

(32) and (34)—Business A and Business B coupons 32 and 34 are in the clothes/shoes category.

Business C and Business D coupons 36 and 38 are in a different vacation category.

Both the categories chosen and the individual deals/ads chosen are personalized to the user view of this listing. Service 154 of FIG. 16 generated this listing.

The personalized ad/coupon listing also contains some images personalized to the user. The content within the picture may be enhanced to appeal to the user—such by including text or faces of people known to user or color preferences etc. Also included are tracking elements for paper or electronic forms such that an authorized device can use the tracking material (such as bar codes or QR codes) and know who utilized the personalized ad or coupon.

FIG. 19 shows another use-case in a production printing environment where the aforementioned method can be utilized. Alternately, the pdfs could be just viewed on screen as opposed to printing.

Described is a large workflow with targeted personalization inserted in to the workflow stream through the context graph technology.

180: Online or printed statement generation requests are initiated. This starts the process of providing credentials, transaction information (such as those in the statements) provided to the printer/electronic publisher.

182: The web (182) sends extracted context (context refers to content like that explained in FIG. 16 but extracted per user or per transaction for a user—usually one line in say a credit card statement) and send to (184) a targeting service (equivalent to ad-fetch and ranking service 154 in FIG. 16).

184: The process fetches the ads per user per transaction and ranks the relevance and applicability of the ads

186: The process merges ads and per user content into the stream. Stream is said here because in the workflow we have many statements corresponding to a multitude of users. 186 also applies image color manipulations or document conversions or device format conversions for rich look and appeal w.r.t device or document format.

188: Useful information, recommendations, coupons, ads or messages that are generated by 186 are transmitted by web to produce a new stream 190

190: A new print stream is augmented with extra content.

It will be appreciated that variants of the above-disclosed and other features and functions, or alternatives thereof, may be combined into many other different systems or applications. Various presently unforeseen or unanticipated alternatives, modifications, variations or improvements therein may be subsequently made by those skilled in the art.

Claims

1. A method of providing user requested printed material and one or more targeted communications to a user of a printing system, the method comprising:

a) the printing system acquiring material to be printed on a printing device to produce the user requested printed material, the printing device associated with the printing system;

b) the printing system acquiring the one or more targeted communications associated with the user requested printed material; and,

c) the printing system printing the user requested printed material utilizing the printing device and providing the acquired one or more targeted communications to the user,

wherein the one or more targeted communications are generated as a function of one or more attributes associated with one or more of the user requested printed material the printing device and the user, the one or more attributes provided by a context graph including a plurality of links between a plurality of entities related to one or more of the user requested printed material, the printing device and the user.

2. The method according to claim 1, wherein the context graph is a n-partite scale-free graph.

3. The method according to claim 2, wherein the context graph includes a historical representation of the plurality of links between the plurality of entities.

4. The method according to claim 3, the plurality of entities associated with one or more of events, people, devices, locations, date, time, keywords, categories, preferences.

5. The method according to claim 3, wherein eh context graph includes weights associated with the plurality of links, the weights related to a frequency of links between a pair of entities.

6. The method according to claim 3, wherein the context graph is dynamically updated.

7. The method according to claim 3, wherein the one or more attributes are provided by one or more sub-graphs extracted from the context graph.

8. The method according to claim 7, wherein the one or more sub-graphs extracted from the context graph are associated with a predefined structure.

9. The method according to claim 7, wherein the one or more attributes are provided by a dominant sub-graph associated with the one or more sub-graphs extracted from the context graph.

10. The method according to claim 3, wherein the one or more attributes are provided by two or more sub-graphs connected by one or more hubs associated with the context graph.

11. The method according to claim 1, wherein a verification action is performed by the user, the verification action indicating the user has read at least one target communication.

12. A printing system for providing user requested printed material and one or more targeted communications to a user of the printing system comprising:

a printing device; and,

one or more servers operating connected to the printing device,

wherein one or more of the printing device and the one or more servers are configured to execute a method comprising:

the printing system acquiring material to be printed on the printing device to produce the user requested printed material; a) the printing system acquiring the one or more targeted communications associated with the user requested printed material; and, b) the printing device printing the user requested printed material and the printing device providing the acquired one or more targeted communications to the user,

wherein of one or more attributes associated with one or more of the user requested printed material, the printing device and the user, the attributes provided by a context graph including a plurality of links between a plurality of entities related one or more of the user requested printed material, the printing device and the user.

13. The printing system according to claim 12, wherein the context graph is a n-Partite scale-free graph.

14. The printing system according to claim 13, wherein the context graph includes a historical representation of the plurality of link between the plurality of entities.

15. The printing system according to claim 14, the plurality of entities associated with one or more of events, people, devices, locations, date, time, keywords, categories, preferences.

16. The printing system according to claim 14, wherein the context graph includes weights associated with the plurality of links, the weights related to a frequency of links between a pair of entities.

17. The printing system according to claim 14, where the context graph is dynamically updated.

18. The printing system according to claim 14, wherein the one or more attributes are provided by one or more sub-graphs extracted from the context graph.

19. The printing system according to claim 18, wherein the one or more sub-graphs extracted from the context graph are associated with a predefined structure.

20. The printing system according to claim 18, wherein the one or more attributes are provided by a dominant sub-graph associated with the one or more sub-graphs extracted from the context graph.

21. The printing system according to claim 14, wherein the one or more attributes are provided by two or more sub-graphs connected by one or more hubs associated with the context graph.

22. The printing system according to claim 12, wherein a verification action is performed by the user, the verification action indicating the user has read at least one targeted communication.

23. A method of providing a targeted communication to a first image rendering device operatively connected to one or more of a second image rendering device and comprising:

a) the image rendering device acquiring the target communication from one of the second image rendering device and the server; and

b) the image rendering device rendering the target communication on the first image rendering device,

wherein the targeted communication is generated as a function of one or more attributes associated with one or more of the first image rendering device, and a user associated with the first image rendering device, the one or more attributes provided by a context graph including a plurality of links between a plurality of entities related to one or more of the first image rendering device, the second image rendering device, and the user associated with the first image rendering device.

24. The method according to claim 23, wherein the context graph is a n-Partite scale-free graph.

25. The method according to claim 23, wherein the context graph includes weights associated with the plurality of links, the weights related to a frequency of links between a pair of entities.