Abstract: A multi-node cluster-based inference method through GPU separation allocation of a pre-trained layer and a fine-tuning layer of multiple deep learning models. The method includes: receiving an input value from a client; distributing the received input value, and transmitting the first input value to a first computation node including a container in which a neural network bundle of a first stage is loaded; performing, by a first container of the first computation nodes, an operation through a neural network layer of a GPU by using the received first input value as an input, and generating a first output value; and selecting a container in which the neural network bundle of the next stage is loaded, and transmitting the second output value to the computation node that includes the container in which the next stage is to be executed or the container in which to execute the next stage.

Abstract: The present disclosure relates to a method for automatically scaling a multi-container-based computational session on a cluster. The method includes: receiving, by a node controller, event information and statistical information about a container from a container engine; forwarding, by the node controller, the sensed event information and statistical information to a cluster controller; determining, by the cluster controller, whether to add or delete the container for a specific session based on the received event information and statistical information; forwarding, by the cluster controller, a container addition or deletion command to the node controller of the specific session according to a determination result; and determining, by the cluster controller, whether to add or delete the node by determining whether it is impossible to add the container only with an existing node or when it is necessary to delete the container.

Abstract: A GPU virtualization method based on a container comprises the steps of: transmitting, if the container is created, a configuration file including GPU resource constraint information and an API profile to the container, by a node controller; and implementing a virtual GPU, when the container is executed, by intercepting a library call and changing an argument related to a GPU resource amount by a library controller provided in the container, and by intercepting a system call and changing argument and return values by a system call controller.

Abstract: A method for providing a session container mounted with a plurality of libraries requested by a user. The method includes: in response to receiving a container execution request from the user, searching for a library container in a container catalog by a node controller; checking, by the node controller, whether a session container to be mounted with a library in the library container and the library container are compatible; and when the session container to be mounted with the library in the library container and the library container are compatible, generating, by a container engine, a session container mounted with the library in the library container.

Abstract: A method for providing a session container mounted with a plurality of libraries requested by a user. The method includes: in response to receiving a container execution request from the user, searching for a library container in a container catalog by a node controller; checking, by the node controller, whether a session container to be mounted with a library in the library container and the library container are compatible; and when the session container to be mounted with the library in the library container and the library container are compatible, generating, by a container engine, a session container mounted with the library in the library container.

Abstract: A GPU virtualization method based on a container comprises the steps of: transmitting, if the container is created, a configuration file including GPU resource constraint information and an API profile to the container, by a node controller; and implementing a virtual GPU, when the container is executed, by intercepting a library call and changing an argument related to a GPU resource amount by a library controller provided in the container, and by intercepting a system call and changing argument and return values by a system call controller.