Personalized comment recommendation method based on link prediction model of graph bidirectional aggregation network

A personalized comment recommendation method based on a link prediction model of a graph bidirectional aggregation network. In a user-comment bipartite graph, comment features are aggregated into a user feature. A social network is used to fuse a neighbor feature of a user to obtain an embedding representation of the user. The embedding representation of the user is aggregated into a comment after an original feature of the user is removed, and the embedding representation of the user is adjusted based on a difference before and after comment aggregation. On this basis, a forwarding network is used to calculate a score of an edge based on an inner product of user node features at both ends of the edge, and finally make a recommendation based on the score. Furthermore, a recommendation system converts a comment recommendation task into a link prediction task between users in a small range.

Skip to: Description  ·  Claims  ·  References Cited  · Patent History  ·  Patent History
Description
CROSS REFERENCES OF THE RELATED APPLICATIONS

This application is a continuation-in-part application of PCT/CN2021/131595 filed on Nov. 19, 2021, which is based on and claims priority to Chinese patent application No. 202110683196.9 filed on Jun. 18, 2021, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to the technical field of personalized comment recommendations and, in particular, to a personalized comment recommendation method based on a link prediction model of a graph bidirectional aggregation network.

BACKGROUND

Internet “trolls” have existed for a long time. Relevant studies have found that the best strategy to deal with these “trolls” is to not respond to them. Although a comment itself has been screened using reverse chronological order, popularity, manual review, and the like, the problem of “trolls” still has not been resolved. The development of a recommendation system helps to resolve this problem, but the quality of a recommendation result greatly depends on the quality of an input feature. Most of the existing methods are intended to extract independent data of the same type. Although these methods have a certain expression capability, due to a lack of associated information, a feature obtained only from the same type of data can no longer meet the development demand of the recommendation system. Particularly, there are few methods for extracting different types of data commonly existing in the recommendation system.

SUMMARY

To overcome the above shortcomings in the prior art, the present disclosure provides a personalized comment recommendation method based on a link prediction model of a graph bidirectional aggregation network, which can process different types of data to make a final result more representative.

The technical solution used in the present disclosure to resolve the technical problem thereof is as follows:

A personalized comment recommendation method based on a link prediction model of a graph bidirectional aggregation network includes the following steps:

a) A user-comment bipartite graph, a social network, and a forwarding network are built based on specific comment content, a user concern relationship, and a comment forwarding relationship.

b) A comment made by a user is converted into an N-dimensional original comment feature hcom, and an original feature huser of the user is initialized.

c) A comment node is randomly deleted from the user-comment bipartite graph and an aggregated feature agghuseri of an ith user is calculated according to a formula agghuseri=SELU(Wagg·concat(huseri, aggregate({hcomj, ∀j∈Ncom(i)}))), where the aggregated feature of all users is represented by agghuser, SELU(·) represents an activation function, Wagg represents a weight of a feature extraction part, concat(·) represents a splicing function, huseri represents a user feature of an ith node, hcomj represents a comment feature of a jth node, Ncom(i) represents a number of comment nodes associated with an ith user node.

    • aggregate represents an aggregation function, namely,

aggregate = 1 N com ( i ) · j = 0 N com ( i ) Re LU ( w · h com j + b ) ,
where ReLU(·) represents an activation function, w represents a weight in the aggregate function, and b represents an offset in the aggregate function.

d) A fused neighbor node feature agghuseri, of the ith user is calculated according to a formula agghuseri=SELU(Σj=0N(i)αij·Wagg·huserj), where αij represents an attention coefficient of associating the ith user and a jth comment edge. and huseri represents a user feature of the jth node.

e) A new comment feature agghcomi of the ith user is calculated according to a formula agghcomi=SELU(Wagg·aggregate({agghcomj−huserj, ∇j∈Nuser(i)})) where new comment features of all the users are represented by agghcom, agghuseri represents the fused neighbor node feature of the jth user, and Nuser(i) represents a number of user nodes associated with an ith comment node.

f) A loss function value lossagg of a feature aggregation part is calculated according to a formula lossagg=smoothL1(hcom, agghcom), where smoothL1(·) represents a loss function, hcom represents comment features of all nodes, and a model of the feature extraction part is established.

g) Neighbor features are aggregated by using a GraphSAGE algorithm according to a formula preh=GraphSAGE(gretweet, agghuser) to obtain a user feature preh, where gretweet represents a graph structure of the forwarding network.

h) A score score of each edge in a positive-sample forwarding network in a form of an inner product is calculated according to a formula score=prehu·prehv, where prehu represents a user node feature on a left side of a positive-sample edge, and prehv represents a user node feature on a right side of the positive-sample edge. A score score′ of each edge in a negative-sample forwarding network in the form of an inner product is calculated according to a formula score′=prehu′·prehv′, where prehu′ represents a user node feature on a left side of a negative-sample edge, and prehv′ represents a node feature on a right side of the negative-sample edge.

i) A loss function value losspre of a link prediction part is calculated according to a formula losspre=max(0, M−score+score′), where M represents a boundary value, and M=1 is set to establish a model of the link prediction part.

j) A total loss Loss is calculated according to a formula Loss=lossagg×agg+losspre×pre, where agg represents the weight of the feature extraction part, and pre represents a weight of the link prediction part.

k) Steps c) to j) are repeated no less than N times to complete model training and optimization.

l) A recommendation list is generated in descending order based on the score score in step h), and a recommendation is made based on the list.

Preferably, in step a), a negative sample of the forwarding network is constructed by using a negative sampling algorithm, where K represents a parameter in negative sampling and K=5.

Further, in step b), the comment made by the user is converted into a 64-dimensional original comment feature hcom by using a Doc2Vec model.

Further, in step c), the comment node is randomly deleted from the user-comment bipartite graph based on a probability of 60%.

Further, in step d), αij is calculated by using an attention weight calculation method in a Graph Attention Network (GAT) algorithm.

Preferably, in step j), agg=1 and pre=2.

Preferably, in step k), a value of N is 100.

The present disclosure has the following beneficial effects: In a user-comment bipartite graph, comment features are aggregated into a user feature. A social network is used to fuse a neighbor feature of a user to obtain an embedding representation of the user. Subsequently, the embedding representation of the user is aggregated into a comment after an original feature of the user is removed, and the embedding representation of the user is adjusted based on a difference before and after comment aggregation. On this basis, a forwarding network is used to calculate a score of an edge based on an inner product of user node features at both ends of the edge, and finally make a recommendation based on the score. A recommendation system is used to help the user make no response and convert a comment recommendation task into a link prediction task between users in a small range. A feature extraction method capable of processing different types of data is provided, making the final result more representative.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of feature extraction according to the present disclosure.

FIG. 2 is a flowchart of link prediction according to the present disclosure.

FIG. 3 shows a social network and a forwarding network according to the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The present disclosure is further described below by referring to FIG. 1, FIG. 2, and FIG. 3.

A personalized comment recommendation method based on a link prediction model of a graph bidirectional aggregation network includes the following steps:

a) A user-comment bipartite graph, a social network, and a forwarding network are built based on specific comment content, a user concern relationship, and a comment forwarding relationship.

b) A comment made by a user is converted into an N-dimensional original comment feature hcom, and the original feature huser of the user is initialized.

c) In the user-comment bipartite graph, comment node features are aggregated into a user node. Specifically, a comment node is randomly deleted from the user-comment bipartite graph, and aggregated feature agghuseri of an ith user is calculated according to the formula agghuseri=SELU(Wagg·concat(huseri, aggregate({hcomj, ∀j∈Ncom(i)}))), where an aggregated feature of all users is represented by agghuser, SELU(·) represents an activation function, Wagg represents a weight of a feature extraction part, concat(·) represents a splicing function, huseri represents a user feature of an ith node, hcomj represents a comment feature of a jth node, Ncom(i) represents a number of comment nodes associated with an ith user node.

    • aggregate represents an aggregation function, namely,

aggregate = 1 N com ( i ) · j = 0 N com ( i ) Re LU ( w · h com j + b ) ,
where ReLU(·) represents an activation function, w represents a weight in the aggregate function, and b represents an offset in the aggregate function.

d) In the social network, a neighbor node feature is further fused to build a final embedding representation of the user. Specifically, the fused neighbor node feature agghuseri of the ith user is calculated according to the formula agghuseri=SELU(Σj=0N(i)αij·Wagg·huserj), where αij represents an attention coefficient of associating the ith user and a jth comment edge, and huserj represents a user feature of the jth node.

e) To ensure the effectiveness of a feature extraction result, after a user feature is removed from a final representation of the user, the user feature is fused to the comment node to generate a new comment feature agghcom. Specifically, the new comment feature agghcomi, of the ith user is calculated according to the formula agghcomi=SELU(Wagg·aggregate({agghuserj−huserj, ∀j∈Nuser(i)})), where new comment features of all the users are represented by agghcom, agghuserj represents the fused neighbor node feature of the jth user, and Nuser(i) represents a number of user nodes associated with an ith comment node.

F) An original comment feature is compared with the generated comment feature, and the loss function value lossagg of a feature aggregation part is calculated according to the formula lossagg=smoothL1(hcom, agghcom), where smoothL1(·) represents a loss function and hcom represents comment features of all nodes. So far, a model of the feature extraction part is established.

g) In the forwarding network, neighbor node features are aggregated to obtain the user feature. Specifically, the neighbor features are aggregated by using a GraphSAGE algorithm according to the formula preh=GraphSAGE(gretweet, agghuser) to obtain the user feature preh, where gretweet represents a graph structure of the forwarding network.

h) Score score of each edge in a positive-sample forwarding network is calculated in the form of an inner product according to the formula score=prehu·prehv, where prehu represents a user node feature on a left side of a positive-sample edge, and prehv represents a user node feature on a right side of the positive-sample edge. The score score′ of each edge in a negative-sample forwarding network is calculated in the form of an inner product according to the formula score′=prehu′·prehv′, where prehu′ represents a user node feature on a left side of a negative-sample edge, and prehv′ represents a node feature on a right side of the negative-sample edge.

i) Loss function value losspre of a link prediction part is calculated according to the formula losspre=max(0, M−score+score′), where M represents a boundary value, and M=1 is set to establish a model of the link prediction part.

j) Total loss Loss is calculated according to the formula Loss=lossagg×agg+losspre×pre, where agg represents the weight of the feature extraction part and pre represents a weight of the link prediction part.

k) Steps c) to j) are repeated no less than N times to complete model training and optimization.

l) A recommendation list is generated in descending order based on the score score in step h), and a recommendation is made based on the list.

In a user-comment bipartite graph, comment features are aggregated into a user feature. A social network is used to fuse a neighbor feature of a user to obtain an embedding representation of the user. Then, the embedding representation of the user is aggregated into a comment after an original feature of the user is removed, and the embedding representation of the user is adjusted based on the difference before and after comment aggregation. On this basis, a forwarding network is used to calculate a score of an edge based on an inner product of user node features at both ends of the edge, and finally make a recommendation based on the score. The present disclosure focuses on the problem of Internet “trolls” in comment recommendations. A recommendation system is used to help the user make no response and convert a comment recommendation task into a link prediction task between users in a small range. A feature extraction method capable of processing different types of data is provided, making the final result more representative.

Embodiment 1

In step a), a negative sample of the forwarding network is constructed by using a negative sampling algorithm, where K represents a parameter in negative sampling and K=5.

Embodiment 2

In step b), the comment made by the user is converted into a 64-dimensional original comment feature hcom by using a Doc2Vec model.

Embodiment 3

In step c), the comment node is randomly deleted from the user-comment bipartite graph based on a probability of 60%.

Embodiment 4

In step d), αij is calculated by using a calculation method of an attention weight in a GAT algorithm.

Embodiment 5

In step j), agg=1 and pre=2.

Embodiment 6

In step k), a value of N is 100.

Taking a comment in a microblog as an example, the following describes an implementation of the present disclosure:

A user-comment bipartite graph, a social network, and a forwarding network are built based on specific comment content in the microblog (Weibo), a user concern relationship corresponding to the specific comment content, and a corresponding comment forwarding relationship. In addition, for the forwarding network, a negative sample is constructed by using a negative sampling technology. The social network and the forwarding network are shown in FIG. 3. Comment content made by a user is converted into a 64-dimensional original comment feature hcom by using a Doc2Vec model, and the original feature huser of the user is initialized.

In the user-comment bipartite graph, comment node features are aggregated into a user node by taking the user as a central node. Feature information of the original comment hcom is accumulated and averaged to obtain aggregated information of a corresponding comment feature of an ith user. After that, the original feature of the user and an aggregated comment feature are added up to obtain the corresponding user feature agghuseri of the ith user by using the activation function SELU.

In the social network, the weight αij of an impact of a neighbor node on the central node is calculated to obtain a user feature containing a neighbor feature, and agghuseri is updated. In the user-comment bipartite graph, a comment is taken as the central node to calculate a difference between agghuser and huser, and a new comment feature agghcom is obtained according to the above mentioned aggregation process. Finally, agghcom is compared with hcom, and a loss function value of a feature aggregation part is calculated by using lossagg to continuously update the user feature agghuser in the user-comment bipartite graph.

In the forwarding network, a GraphSAGE algorithm is used to aggregate neighbor features of the user to obtain preh, and the score score of an edge between the user u and user v in the forwarding network is calculated in a form of an inner product. Positive and negative samples are compared, and a loss function value of a link prediction part is calculated by using losspre.

Finally, the feature aggregation part and the link prediction part are correlated through multi-task learning, such that the results of the two parts affect each other. Then a personalized comment recommendation is made based on the score score.

The method in the present disclosure is compared with traditional graph embedding models (DeepWalk, Node2Vec, SDNE, LINE) and graph neural network models (GCN, GraphSAGE, GAT, AGNN) to prove the effectiveness of the method in the present disclosure. Comparison results are shown in Table 1, where HR@K represents a hit rate of the first K articles and is used to measure the accuracy of a recommendation result, and NDCG@K represents the position of the recommendation result in a recommendation list and is used to measure the stability of the recommendation result.

The method in the present disclosure adopts an “aggregation-reverse aggregation” manner. After an association relationship is encoded, an attention mechanism is used to enhance the learning of a node feature by the model. Referring to microblog recommendations and evaluation indicators in Table 1, more comments meeting user preferences in the generated recommendation list are recommended. In addition, the quality of the recommendation result is related to the position of recommended content in the list. More content meeting the user preferences is recommended. However, because of the short content and a low behavioral cost of a comment recommendation, the requirement for whether a comment in the comment recommendation is in the most appropriate position of the list is not as high as that for a product recommendation or a music recommendation. Therefore, the method in the present disclosure allows some comments to be less prominent in the list but does not allow arranging content that users may like at the bottom of the list. Therefore, it can be seen by comparing the last line with other lines that although the value of the indicator NDCG@K is relatively small and a position of a recommended comment within a browsing range of the user is not stable, a large value of the indicator HR@K can reflect that a comment recommended by this model is more consistent with a preference of the user and easier to be seen by the user.

TABLE 1 Comparison results of models Model HR@3 NDCG@3 HR@5 NDCG@5 HR@10 NDCG@10 DeepWalk 0.677 0.065 0.773 0.066 0.831 0.068 Node2Vec 0.691 0.156 0.785 0.161 0.789 0.167 SDNE 0.639 0.109 0.662 0.113 0.742 0.119 LINE 0.663 0.166 0.709 0.173 0.703 0.169 GCN 0.429 0.637 0.487 0.638 0.477 0.612 GraphSAGE 0.512 0.684 0.515 0.721 0.532 0.759 GAT 0.421 0.577 0.472 0.579 0.462 0.583 AGNN 0.317 0.768 0.359 0.768 0.363 0.778 GBAN 0.848 0.503 0.864 0.507 0.866 0.514

Finally, it should be noted that the above descriptions are only preferred embodiments of the present disclosure and are not intended to limit the present disclosure. Although the present disclosure is described in detail by referring to the foregoing embodiments, a person skilled in the art can still make modifications to the technical solutions described in the foregoing embodiments, or make equivalent replacements of some technical features therein. Any modifications, equivalent substitutions, improvements, and the like made within the spirit and principle of the present disclosure should be included within the protection scope of the present disclosure.

Claims

1. A personalized comment recommendation method based on a link prediction model of a graph bidirectional aggregation network, comprising the following steps: aggregate = 1 N com ( i ) · ∑ j = 0 N com ( i ) ⁢ Re ⁢ LU ⁡ ( w · h com j + b ), wherein ReLU(·) represents an activation function, w represents a weight in the aggregate function, and b represents an offset in the aggregate function;

a) building a user-comment bipartite graph, a social network, and a forwarding network based on specific comment content, a user concern relationship, and a comment forwarding relationship;
b) converting a comment made by a user into an N-dimensional original comment feature hcom, and initializing an original feature huser of the user;
c) randomly deleting a comment node from the user-comment bipartite graph, and calculating an aggregated feature agghuseri of an ith user according to a formula agghuseri=SELU(Wagg·concat(huseri, aggregate({hcomi, ∀j∈Ncom(i)}))), wherein an aggregated feature of all users is represented by agghuser, SELU(·) represents an activation function, Wagg represents a weight of a feature extraction part, concat(·) represents a splicing function, huseri represents a user feature of an ith node, hcomj represents a comment feature of a jth node, Ncom(i) represents a quantity of comment nodes associated with an ith user node, and aggregate represents an aggregation function, namely,
d) calculating a fused neighbor node feature agghuseri, of the ith user according to a formula agghuseri=SELU(Σj=0N(i)αij·Wagg·huserj), wherein αij represents an attention coefficient of associating the ith user and a jth comment edge, and huserj represents a user feature of the jth node;
e) calculating a new comment feature agghcomi of the ith user according to a formula agghcomi=SELU(Wagg·aggregate({agghuserj−huserj, ∀j∈Nuser(i)})), wherein new comment features of all the users are represented by agghcom, agghuserj represents the fused neighbor node feature of the jth user, and Nuser(i) represents a quantity of user nodes associated with an ith comment node;
f) calculating a loss function value lossagg of a feature aggregation part according to a formula lossagg=smoothL1(hcom, agghcom), wherein smoothL1(·) represents a loss function, hcom represents comment features of all nodes; and establishing a model of the feature extraction part;
g) aggregating neighbor features by using a GraphSAGE algorithm according to a formula preh=GraphSAGE(gretweet, agghuser) to obtain a user feature preh, wherein gretweet represents a graph structure of the forwarding network;
h) calculating a score score of each edge in a positive-sample forwarding network in a form of an inner product according to a formula score=prehu·prehv, wherein prehu represents a user node feature on a left side of a positive-sample edge, and prehv represents a user node feature on a right side of the positive-sample edge; and calculating a score score′ of each edge in a negative-sample forwarding network in the form of an inner product according to a formula score′=prehu′·prehv′, wherein prehu′ represents a user node feature on a left side of a negative-sample edge, and prehv′ represents a node feature on a right side of the negative-sample edge;
i) calculating a loss function value losspre of a link prediction part according to a formula losspre=max(0, M−score+score′), wherein M represents a boundary value, and setting M=1 to establish a model of the link prediction part;
j) calculating a total loss Loss according to a formula Loss=lossagg×agg+losspre×pre, wherein agg represents the weight of the feature extraction part, and pre represents a weight of the link prediction part;
k) repeating steps c) to j) for no less than N times to complete model training and optimization; and
l) generating a recommendation list in descending order based on the score score in step h), and making a recommendation based on the list.

2. The personalized comment recommendation method based on the link prediction model of the graph bidirectional aggregation network according to claim 1, wherein in step a), a negative sample of the forwarding network is constructed by using a negative sampling algorithm, wherein K represents a parameter in negative sampling and K=5.

3. The personalized comment recommendation method based on the link prediction model of the graph bidirectional aggregation network according to claim 1, wherein in step b), the comment made by the user is converted into a 64-dimensional original comment feature hcom by using a Doc2Vec model.

4. The personalized comment recommendation method based on the link prediction model of the graph bidirectional aggregation network according to claim 1, wherein in step c), the comment node is randomly deleted from the user-comment bipartite graph based on a probability of 60%.

5. The personalized comment recommendation method based on the link prediction model of the graph bidirectional aggregation network according to claim 1, wherein in step d), αij is calculated by using an attention weight calculation method in a Graph Attention Network (GAT) algorithm.

6. The personalized comment recommendation method based on the link prediction model of the graph bidirectional aggregation network according to claim 1, wherein in step j), agg=1, and pre=2.

7. The personalized comment recommendation method based on the link prediction model of the graph bidirectional aggregation network according to claim 1, wherein in step k), a value of N is 100.

Referenced Cited
U.S. Patent Documents
20070192350 August 16, 2007 Gao et al.
20150242967 August 27, 2015 Shsh
20200160188 May 21, 2020 Zhao et al.
20210304281 September 30, 2021 Goshen
20220256002 August 11, 2022 Cetintas
20220318715 October 6, 2022 Goel
Foreign Patent Documents
103116639 May 2013 CN
103309972 September 2013 CN
112069402 December 2020 CN
113434756 September 2021 CN
Patent History
Patent number: 11748426
Type: Grant
Filed: Feb 6, 2023
Date of Patent: Sep 5, 2023
Patent Publication Number: 20230185865
Inventors: Minglei Shu (Jinan), Muchen Wang (Jinan), Yinglong Wang (Jinan), Zhao Li (Jinan), Tianlei Gao (Jinan)
Primary Examiner: Kris E Mackes
Application Number: 18/105,891
Classifications
Current U.S. Class: Social Networking (705/319)
International Classification: G06F 16/9535 (20190101); G06Q 50/00 (20120101);