Abstract: A sequencer and method for batching execution of artificial intelligence (AI) jobs comprising receiving, by an AI server, a plurality of AI jobs from a plurality of clients connected to an AI appliance over a network; dynamically selecting from the plurality of AI jobs a set of AI jobs to be batched, wherein the selection is based on at least one batching parameter and a list of AI jobs prohibited from batching; aggregating each of the selected AI jobs into a created batch; continuing aggregating newly received AI jobs related to the selected AI jobs in the created batch until at least one service parameter is met; and sending the batch of AI jobs to a compute engine dedicated to executing the batch.

Abstract: A sequencer and method for batching execution of artificial intelligence (AI) jobs. A method includes receiving, by a plurality of AI servers, a plurality of AI jobs from a plurality of clients connected to the plurality of AI servers over a network, determining a first group of AI jobs from the plurality of AI jobs that are candidates for batching, batching the first group of AI jobs based on least one service parameter, and sending, for execution, the batch of first group AI jobs to a plurality of first compute engines reside in different AI servers of the plurality of AI servers.

Abstract: A method and system for communicating artificial intelligence (AI) tasks for a server chaining are presented. The method includes establishing a first connection between an AI client and a first AI server; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol; and transporting the at least one request data frame over a network using a transport protocol over the first connection to the first AI server, wherein the first AI server spans the AI task over at least one second AI server, wherein the transport protocol provisions transport characteristics of the AI task and the transport protocol is different than the communication protocol, wherein AI task includes processing of a single compute graph thereby allow spanning the processing of the compute graph over one more AI servers.

Abstract: A method and system for communicating artificial intelligence (AI) tasks between AI resources are provided. The method includes establishing a connection between a first AI resource and a second AI resource; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource; transporting the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol is different than the communication protocol; and using a credit-based flow control mechanism to transfer messages between the first AI resource and the second AI resource over the transport protocol, thereby avoiding congestion on compute resources.

Abstract: A method for use with an artificial intelligence (AI) sequencer that is at least partially circuitry implemented and is adapted to be coupled to a plurality of AI accelerators via electronic connection, the method comprising: dispatching by the sequencer, using the electronic connection, an initial stage of an AI job having a plurality of stages to at least one of the AI accelerators for execution of the initial stage, wherein the AI job includes multiple AI functions; upon completion of the initial stage, receiving the AI job back at the sequencer; and, thereafter, dispatching by the sequencer, using the electronic connection, a next stage of the AI job to at least one different one of the AI accelerators; wherein the sequencer dispatches each AI function of the AI job at computer speed over the electronic connection to minimize any possible idle time of the AI accelerators.

Abstract: A remote artificial intelligence (AI) appliance system is provided. The system includes an artificial intelligence accelerator (AIA) switch; and a plurality of network attached artificial intelligence accelerator (NA-AIA) engines connected to the AIA switch; wherein each of the plurality of NA-AIA engines is configured to execute at least one processing Al task in response to a request received from at least one Al application server; wherein the AIA appliance server and the at least one application server are connected over a network connection; and wherein the AIA switch is configured to allow direct communication between at least one application server and the plurality of NA-AIA engines.

Abstract: A remote artificial intelligence (AI) acceleration system is provided. The system includes a plurality of application servers, wherein each of the plurality of application server is configured to execute AI applications over an AI software framework; at least one artificial intelligence accelerator (AIA) appliance server configured to execute AI processing tasks in response to requests from the AI applications; and at least one switch configured to allow connectivity between the plurality of application servers and the at least one AIA appliance server, wherein the plurality of network attached artificial intelligence accelerator (NA-AIA) engines connected to the AIA switch, wherein each of the plurality of NA-AIA engines connected is configured to execute at least one processing AI task.

Abstract: An artificial intelligence (AI) sequencer is provided. The Al sequencer includes a queue manager configured to manage a plurality of queues for maintaining data of AI jobs, wherein an AI job includes processing of one or more AI functions; a scheduler for scheduling execution of data maintained by the plurality of queues; a plurality of job processing units (JPUs), wherein each of the plurality JPUs is configured to at least generate an execution sequence for an AI job; and a plurality of dispatchers connected to a plurality of AI accelerators, wherein each of the plurality of dispatchers is configured to dispatch at least a function of the AI job to an AI accelerator, wherein a function is dispatched to an AI accelerator at an order determined by an execution sequence created for a respective AI job.

Abstract: A sequencer and method for batching execution of artificial intelligence (AI) jobs. A method includes receiving, by a plurality of AI servers, a plurality of AI jobs from a plurality of clients connected to the plurality of AI servers over a network, determining a first group of AI jobs from the plurality of AI jobs that are candidates for batching, batching the first group of AI jobs based on least one service parameter, and sending, for execution, the batch of first group AI jobs to a plurality of first compute engines reside in different AI servers of the plurality of AI servers.

Abstract: A method and server for batching execution of artificial inelegance (AI) jobs are provided. The method includes receiving, by an AI server, a plurality of AI jobs from a plurality of clients connected to an AI appliance over a network; for each AI job of the plurality of AI jobs: deriving at least one attribute of the received AI job; determining based on the at least one AI job attribute and at least one batching parameter if the received AI job is a candidate for batching; aggregating the received AI job into a created batch when the received AI job is determined to be a candidate for batching; continuing aggregating newly received AI jobs determined to be candidates for batching in the created batch until at least one service parameter is met; and sending the batch of AI jobs to a compute engine dedicated to executing the batch.

Abstract: A method and system for communicating artificial intelligence (AI) tasks for a server chaining are presented. The method includes establishing a first connection between an AI client and a first AI server; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol; and transporting the at least one request data frame over a network using a transport protocol over the first connection to the first AI server, wherein the first AI server spans the AI task over at least one second AI server, wherein the transport protocol provisions transport characteristics of the AI task and the transport protocol is different than the communication protocol, wherein AI task includes processing of a single compute graph thereby allow spanning the processing of the compute graph over one more AI servers.

Abstract: A system and method for communicating artificial intelligence (AI) tasks between AI resources are provided. The method comprises establishing a connection between a first AI resource and a second AI resource; encapsulating a request to process an AI task in at least one request data frame compliant with a communication protocol, wherein the at least one request data frame is encapsulated at the first AI resource; and transporting the at least one request data frame over a network using a transport protocol to the second AI resource, wherein the transport protocol provisions the transport characteristics of the AI task, and wherein the transport protocol is different than the communication protocol.

Abstract: A remote artificial intelligence (AI) acceleration system is provided. The system includes a plurality of application servers, wherein each of the plurality of application server is configured to execute AI applications over an AI software framework; at least one artificial intelligence accelerator (AIA) appliance server configured to execute AI processing tasks in response to requests from the AI applications; and at least one switch configured to allow connectivity between the plurality of application servers and the at least one AIA appliance server, wherein the plurality of network attached artificial intelligence accelerator (NA-AIA) engines connected to the AIA switch, wherein each of the plurality of NA-AIA engines connected is configured to execute at least one processing AI task.