Abstract: A sequencer and method for batching execution of artificial intelligence (AI) jobs comprising receiving, by an AI server, a plurality of AI jobs from a plurality of clients connected to an AI appliance over a network; dynamically selecting from the plurality of AI jobs a set of AI jobs to be batched, wherein the selection is based on at least one batching parameter and a list of AI jobs prohibited from batching; aggregating each of the selected AI jobs into a created batch; continuing aggregating newly received AI jobs related to the selected AI jobs in the created batch until at least one service parameter is met; and sending the batch of AI jobs to a compute engine dedicated to executing the batch.

Abstract: A sequencer and method for batching execution of artificial intelligence (AI) jobs. A method includes receiving, by a plurality of AI servers, a plurality of AI jobs from a plurality of clients connected to the plurality of AI servers over a network, determining a first group of AI jobs from the plurality of AI jobs that are candidates for batching, batching the first group of AI jobs based on least one service parameter, and sending, for execution, the batch of first group AI jobs to a plurality of first compute engines reside in different AI servers of the plurality of AI servers.

Abstract: A method and system for communicating artificial intelligence (AI) tasks between AI resources are provided. The method includes establishing a connection between a first AI resource and a second AI resource; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource; transporting the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol is different than the communication protocol; and using a credit-based flow control mechanism to transfer messages between the first AI resource and the second AI resource over the transport protocol, thereby avoiding congestion on compute resources.

Abstract: A method for use with an artificial intelligence (AI) sequencer that is at least partially circuitry implemented and is adapted to be coupled to a plurality of AI accelerators via electronic connection, the method comprising: dispatching by the sequencer, using the electronic connection, an initial stage of an AI job having a plurality of stages to at least one of the AI accelerators for execution of the initial stage, wherein the AI job includes multiple AI functions; upon completion of the initial stage, receiving the AI job back at the sequencer; and, thereafter, dispatching by the sequencer, using the electronic connection, a next stage of the AI job to at least one different one of the AI accelerators; wherein the sequencer dispatches each AI function of the AI job at computer speed over the electronic connection to minimize any possible idle time of the AI accelerators.

Abstract: A remote artificial intelligence (AI) appliance system is provided. The system includes an artificial intelligence accelerator (AIA) switch; and a plurality of network attached artificial intelligence accelerator (NA-AIA) engines connected to the AIA switch; wherein each of the plurality of NA-AIA engines is configured to execute at least one processing Al task in response to a request received from at least one Al application server; wherein the AIA appliance server and the at least one application server are connected over a network connection; and wherein the AIA switch is configured to allow direct communication between at least one application server and the plurality of NA-AIA engines.

Abstract: A remote artificial intelligence (AI) acceleration system is provided. The system includes a plurality of application servers, wherein each of the plurality of application server is configured to execute AI applications over an AI software framework; at least one artificial intelligence accelerator (AIA) appliance server configured to execute AI processing tasks in response to requests from the AI applications; and at least one switch configured to allow connectivity between the plurality of application servers and the at least one AIA appliance server, wherein the plurality of network attached artificial intelligence accelerator (NA-AIA) engines connected to the AIA switch, wherein each of the plurality of NA-AIA engines connected is configured to execute at least one processing AI task.

Abstract: A sequencer and method for batching execution of artificial intelligence (AI) jobs. A method includes receiving, by a plurality of AI servers, a plurality of AI jobs from a plurality of clients connected to the plurality of AI servers over a network, determining a first group of AI jobs from the plurality of AI jobs that are candidates for batching, batching the first group of AI jobs based on least one service parameter, and sending, for execution, the batch of first group AI jobs to a plurality of first compute engines reside in different AI servers of the plurality of AI servers.

Abstract: A method and system for communicating artificial intelligence (AI) tasks for a server chaining are presented. The method includes establishing a first connection between an AI client and a first AI server; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol; and transporting the at least one request data frame over a network using a transport protocol over the first connection to the first AI server, wherein the first AI server spans the AI task over at least one second AI server, wherein the transport protocol provisions transport characteristics of the AI task and the transport protocol is different than the communication protocol, wherein AI task includes processing of a single compute graph thereby allow spanning the processing of the compute graph over one more AI servers.

Abstract: A remote artificial intelligence (AI) acceleration system is provided. The system includes a plurality of application servers, wherein each of the plurality of application server is configured to execute AI applications over an AI software framework; at least one artificial intelligence accelerator (AIA) appliance server configured to execute AI processing tasks in response to requests from the AI applications; and at least one switch configured to allow connectivity between the plurality of application servers and the at least one AIA appliance server, wherein the plurality of network attached artificial intelligence accelerator (NA-AIA) engines connected to the AIA switch, wherein each of the plurality of NA-AIA engines connected is configured to execute at least one processing AI task.