Abstract: Disclosed are a network card communication method and apparatus for an AI training platform, a device, and a medium. The method includes: building a switch network topology on the basis of a spine-leaf network, and configuring a preset number of virtual local area networks for each leaf switch in the switch network topology; virtualizing a physical network card to obtain virtual network cards, allocating the virtual network cards to corresponding job-containers according to a preset allocation rule, and allocating, to each virtual network card in the job-containers, different sub-networks corresponding to the virtual local area networks; and adding a corresponding sub-network communication policy routing rule to a pod where each job-container is located, whereby a virtual network card in the job-container sends training data to the remaining virtual network cards on a basis of the sub-network communication policy routing rule.

Abstract: The present disclosure provides a configuration method for virtual network interface card resource, including: integrating a plurality of virtual network interface cards in a node where a container group is located, so as to obtain a virtual network interface card set including a plurality of virtual network interface card groups; assigning a target virtual network interface card group for the container group from the virtual network interface card set; analyzing the target virtual network interface card group to obtain address information of target virtual network interface cards in the target virtual network interface card group; and configuring a virtual network interface card resource for the container group according to the address information.

Abstract: A resource management method for a node in Kubernetes, a computer device, and a readable storage medium. The method includes following steps: initializing a component to start a main process; defining, by means of the main process, subprocesses for managing resources respectively; in response to a detection from the main process that a plurality of actual resources is present in a node among the resources, starting a subprocess corresponding to each of the actual resources, and starting gRPC service for each started subprocess; and registering the resources corresponding to a subprocesses where the gRPC service is successfully started to Kubernetes by using the main process enabling the subprocesses where the gRPC service is successfully started to manage the actual resources.

Abstract: The present application discloses a task allocation method and system based on a resource management platform. The method comprises: receiving an artificial intelligence model training and/or testing task and a name of data set required for processing the task; acquiring data set distribution information of a plurality of nodes; judging if the node has the required data sets according to names of the data sets in the node; and selecting a node with the size of the required data set meeting preset requirements for task allocation according to the size of the required data set in the node if the node has the required data set.

Abstract: The present application discloses Graphics Processing Unit (GPU)-shared dispatching and single-machine multi-card methods, systems, and devices, applied to a dispatcher of a central control host. In the present application, a resource occupation marker and an update marker are added to a pod, the resource occupation marker is used to prove that the pod is running, and the update marker is used to indicate that resource usage information in an annotation of the pod is consistent with actual resource usage information, whereby during dispatching, a resource usage situation of each pod may be determined. The resource usage information, i.e., GPU information of an actually used GPU, in the annotation of the pod is updated before dispatching, to ensure that the GPU information in the annotation is consistent with GPU information actually recorded in an environment variable of the pod.

Abstract: Disclosed is a data set and node cache-based scheduling method, which includes: obtaining storage resource information of each host node; in response to receiving a training task, obtaining operation information of the training task, and according to the operation information and the storage resource information, screening host nodes that satisfy a space required by the training task; in response to no host node satisfying the space required by the training task, scoring each host node according to the storage resource information; according to scoring results, selecting, from among all of the host nodes, a host node to be executed that is used to execute the training task; and obtaining and deleting an obsolete data set cache in the host node to be executed, and executing the training task in the host node to be executed.

Abstract: Disclosed is a data set and node cache-based scheduling method, which includes: obtaining storage resource information of each host node; in response to receiving a training task, obtaining operation information of the training task, and according to the operation information and the storage resource information, screening host nodes that satisfy a space required by the training task; in response to no host node satisfying the space required by the training task, scoring each host node according to the storage resource information; according to scoring results, selecting, from among all of the host nodes, a host node to be executed that is used to execute the training task; and obtaining and deleting an obsolete data set cache in the host node to be executed, and executing the training task in the host node to be executed.

Abstract: The present application discloses Graphics Processing Unit (GPU)-shared dispatching and single-machine multi-card methods, systems, and devices, applied to a dispatcher of a central control host. In the present application, a resource occupation marker and an update marker are added to a pod, the resource occupation marker is used to prove that the pod is running, and the update marker is used to indicate that resource usage information in an annotation of the pod is consistent with actual resource usage information, whereby during dispatching, a resource usage situation of each pod may be determined. The resource usage information, i.e., GPU information of an actually used GPU, in the annotation of the pod is updated before dispatching, to ensure that the GPU information in the annotation is consistent with GPU information actually recorded in an environment variable of the pod.

Abstract: A resource management method for a node in Kubernetes, a computer device, and a readable storage medium. The method includes following steps: initializing a component to start a main process; defining, by means of the main process, subprocesses for managing resources respectively; in response to a detection from the main process that a plurality of actual resources is present in a node among the resources, starting a subprocess corresponding to each of the actual resources, and starting gRPC service for each started subprocess; and registering the resources corresponding to a subprocesses where the gRPC service is successfully started to Kubernetes by using the main process enabling the subprocesses where the gRPC service is successfully started to manage the actual resources.

Abstract: The present application discloses a task allocation method and system based on a resource management platform. The method comprises: receiving an artificial intelligence model training and/or testing task and a name of data set required for processing the task; acquiring data set distribution information of a plurality of nodes; judging if the node has the required data sets according to names of the data sets in the node; and selecting a node with the size of the required data set meeting preset requirements for task allocation according to the size of the required data set in the node if the node has the required data set.