Abstract: Graph data processing methods and system are disclosed. One example method comprises obtaining, by a master node, graph data, wherein the graph data comprises M vertexes and a plurality of directional edges, each edge connects two vertexes, a direction of each edge is from a source to a destination vertex in the two vertexes, and M is an integer greater than two. The node divides the graph data into P non-overlapping shards, where each shard comprises at least one incoming edge directed to at least one vertex in the corresponding shard. The node schedules at least two edge sets comprised in a first shard of the P shards and an associate edge set comprised in a second shard of the P shards for processing by at least two worker nodes.

Abstract: An entity search method, a related device, and a computer storage medium are provided. The method includes: determining a first classifier and a second classifier that are included in query information; determining, based on a first entity library and the first classifier and the second classifier that are included in the query information, s correlations corresponding to each of w candidate entities, wherein information about the candidate entity includes a third classifier and a fourth classifier, and the correlation indicates a correlation between the classifiers in the query information and the classifiers in the candidate entity; and determining, based on the s correlations corresponding to each of the w candidate entities, information about a target entity corresponding to the query information, wherein the target entity is an entity in the w candidate entities.

Abstract: Graph data processing methods and system are disclosed. One example method comprises obtaining, by a master node, graph data, wherein the graph data comprises M vertexes and a plurality of directional edges, each edge connects two vertexes, a direction of each edge is from a source to a destination vertex in the two vertexes, and M is an integer greater than two. The node divides the graph data into P non-overlapping shards, where each shard comprises at least one incoming edge directed to at least one vertex in the corresponding shard. The node schedules at least two edge sets comprised in a first shard of the P shards and an associate edge set comprised in a second shard of the P shards for processing by at least two worker nodes.

Abstract: A graph data processing method and a distributed system is disclosed. The distributed system includes a master node and a plurality of worker nodes. The master node obtains master node graph data, divides the graph data to obtain P shards, where the P shards include a first shard and a second shard. The master node determines at least two edge sets from each shard, schedules at least two edge sets included in the first shard to at least two worker nodes for processing, and schedules an associate edge set included in the second shard to the at least two worker nodes for processing, where the associate edge set is an edge set that includes an outgoing edge of a target vertex corresponding to the first shard.

Abstract: A distributed computing system is provided. Both a first computing node and a second computing node in the distributed computing system store information about a name, a size, and a communication peer side identifier of a first data flow graph parameter in a data flow graph. The first computing node stores the first data flow graph parameter, where the first computing node and the second computing node generate respective triplets based on same interface parameter generation algorithms and information about the first data flow graph parameter that are stored in the respective nodes. The triplet is used as an interface parameter of a message passing interface (MPI) primitive that is used to transmit the first data flow graph parameter between the first computing node and the second computing node.

Abstract: An entity search method, a related device, and a computer storage medium are provided. The method includes: determining a first classifier and a second classifier that are included in query information; determining, based on a first entity library and the first classifier and the second classifier that are included in the query information, s correlations corresponding to each of w candidate entities, wherein information about the candidate entity includes a third classifier and a fourth classifier, and the correlation indicates a correlation between the classifiers in the query information and the classifiers in the candidate entity; and determining, based on the s correlations corresponding to each of the w candidate entities, information about a target entity corresponding to the query information, wherein the target entity is an entity in the w candidate entities.

Abstract: A streaming application upgrading method includes obtaining an updated logical model of a streaming application, and determining a to-be-adjusted stream by comparing the updated logical model with an initial logical model; generating an upgrading instruction according to the to-be-adjusted stream; and delivering the generated upgrading instruction to a worker node, so that the worker node adjusts, according to an indication of the upgrading instruction, a stream between PEs distributed on the worker node.

Abstract: A streaming application upgrading method and a stream computing system, where the method includes obtaining a updated logical model of a streaming application, determining a to-be-adjusted stream by comparing the updated logical model with an initial logical model, generating an upgrading instruction according to the to-be-adjusted stream, and delivering the generated upgrading instruction to a worker node such that the worker node adjusts, according to an indication of the upgrading instruction, a stream between process elements (PEs) distributed on the worker node. The method provided in the present disclosure can upgrade the streaming application online without interrupting a service.

Abstract: A distributed computing system is provided. Both a first computing node and a second computing node in the distributed computing system store information about a name, a size, and a communication peer side identifier of a first data flow graph parameter in a data flow graph. The first computing node stores the first data flow graph parameter, where the first computing node and the second computing node generate respective triplets based on same interface parameter generation algorithms and information about the first data flow graph parameter that are stored in the respective nodes. The triplet is used as an interface parameter of a message passing interface (MPI) primitive that is used to transmit the first data flow graph parameter between the first computing node and the second computing node.

Abstract: A stream computer system and a method for processing a data stream in a stream computing system are disclosed. The method includes a first working node invokes at least one execution unit to process a data stream according to an initial parallelism degree, a control node collects information reflecting data traffic between the first working node and a second working node, and information reflecting data processing speed of the first working node, determines an optimized parallelism degree for the first working node according to the collected information, and adjusts the parallelism degree of the first working node to be consistent with the optimized parallelism degree.

Abstract: A streaming graph optimization method and apparatus are disclosed, relating to the stream processing field. A stream application streaming graph provided by a user is received and the streaming graph is parsed and a streaming graph described by an operator node and a data stream side is constructed. Additionally the streaming graph is disassembled according to a maximum atom division principle, so as to obtain at least one streaming subgraph and adjacency operator combination is performed on the at least one streaming subgraph according to a combination algorithm, so as to obtain an optimized streaming graph.

Abstract: A training method, apparatus, and chip for a neural network model includes determining a model training mode of each layer based on an estimated data volume in a model parameter set and an estimated data volume of output data of the layer, obtaining second output data that is obtained by m worker modules by training a (j?1)th layer, and directly obtaining by a worker module a global gradient of a model parameter by training the model parameter based on the second output data when a model parallel training mode is used for a jth layer.

Abstract: A graph data processing method and a distributed system is disclosed. The distributed system includes a master node and a plurality of worker nodes. The master node obtains master node graph data, divides the graph data to obtain P shards, where the P shards include a first shard and a second shard. The master node determines at least two edge sets from each shard, schedules at least two edge sets included in the first shard to at least two worker nodes for processing, and schedules an associate edge set included in the second shard to the at least two worker nodes for processing, where the associate edge set is an edge set that includes an outgoing edge of a target vertex corresponding to the first shard.

Abstract: The present invention disclose a method for processing a policy, including: obtaining at least one user policy, where the user policy is used to instruct a service processing unit to perform service processing; reading information of an application knowledge base corresponding to each user policy and performing rule conversion on the read information, so that the description language of the information is consistent with the rule description language of the user policy; combining a condition element and an action element of each user policy with information corresponding to the condition element and the action element, and compiling each result obtained through combination; and sending each compilation result to a service processing unit corresponding to each compilation result. Correspondingly, the embodiments of the present invention further disclose a device for processing a policy. The embodiments of the present invention can reduce system overhead.

Abstract: The present disclosure discloses a software defined network SDN-based data processing system, and the system includes: a source data node, configured to receive a first data packet, and send to a corresponding source control node; the source control node, configured to receive the first data packet, where the first data packet carries a destination address of the first data packet; and determine a destination control node; and the destination control node, configured to receive the first data packet, and generate a second data packet and a matching policy rule. According to a software defined network-based data processing system in an embodiment of the present disclosure, the collaboration capability between nodes is improved so as to reduce the redundancy of multi-node processing in a network device, thereby improving the service processing efficiency of the network. The present disclosure further discloses a software defined network-based data processing method and device.

Abstract: A stream computer system and a method for processing a data stream in a stream computing system are disclosed. The method includes a first working node invokes at least one execution unit to process a data stream according to an initial parallelism degree, a control node collects information reflecting data traffic between the first working node and a second working node, and information reflecting data processing speed of the first working node, determines an optimized parallelism degree for the first working node according to the collected information, and adjusts the parallelism degree of the first working node to be consistent with the optimized parallelism degree.

Abstract: A stream computer system and a method for processing a data stream in a stream computing system are disclosed. In an embodiment, the method includes collecting data traffic information between each working node and other working nodes and processing speed information for each working node, determining an optimized parallelism degree for each working node according to the collected data traffic information and processing speed information and adjusting a parallelism degree of the working node according to the optimized parallelism degree of the working node.

Abstract: The present disclosure discloses a software defined network SDN-based data processing system, and the system includes: a source data node, configured to receive a first data packet, and send to a corresponding source control node; the source control node, configured to receive the first data packet, where the first data packet carries a destination address of the first data packet; and determine a destination control node; and the destination control node, configured to receive the first data packet, and generate a second data packet and a matching policy rule. According to a software defined network-based data processing system in an embodiment of the present disclosure, the collaboration capability between nodes is improved so as to reduce the redundancy of multi-node processing in a network device, thereby improving the service processing efficiency of the network. The present disclosure further discloses a software defined network-based data processing method and device.

Abstract: A streaming graph optimization method and apparatus are disclosed, relating to the stream processing field. A stream application streaming graph provided by a user is received and the streaming graph is parsed and a streaming graph described by an operator node and a data stream side is constructed. Additionally the streaming graph is disassembled according to a maximum atom division principle, so as to obtain at least one streaming subgraph and adjacency operator combination is performed on the at least one streaming subgraph according to a combination algorithm, so as to obtain an optimized streaming graph.

Abstract: The present disclosure discloses a software defined network SDN-based data processing system, and the system includes: a source data node, configured to receive a first data packet, and send to a corresponding source control node; the source control node, configured to receive the first data packet, where the first data packet carries a destination address of the first data packet; and determine a destination control node; and the destination control node, configured to receive the first data packet, and generate a second data packet and a matching policy rule. According to a software defined network-based data processing system in an embodiment of the present disclosure, the collaboration capability between nodes is improved so as to reduce the redundancy of multi-node processing in a network device, thereby improving the service processing efficiency of the network. The present disclosure further discloses a software defined network-based data processing method and device.