Abstract: A compute server of a distributed cloud computing network receives an inference request that is directed to an AI model hosted at a destination external to the distributed cloud computing network. The compute server determines that the inference request satisfies security rules associated with the AI model. Upon determining that the inference request is not answerable from a cache, the compute server transmits the inference request to the AI model hosted at the external destination. The compute server receives an inference response from the AI model in response to the inference request, transmits the inference response, and stores the inference request and the inference response in cache.

Abstract: A compute server of a distributed cloud computing network receives an inference request that is directed to an AI model hosted at a destination external to the distributed cloud computing network. The compute server determines that the inference request satisfies security rules associated with the AI model. Upon determining that the inference request is not answerable from a cache, the compute server transmits the inference request to the AI model hosted at the external destination. The compute server receives an inference response from the AI model in response to the inference request, transmits the inference response, and stores the inference request and the inference response in cache.

Abstract: A compute server of a distributed cloud computing network receives an inference request that is directed to an AI model hosted at a destination external to the distributed cloud computing network. The compute server determines that the inference request satisfies security rules associated with the AI model. Upon determining that the inference request is not answerable from a cache, the compute server transmits the inference request to the AI model hosted at the external destination. The compute server receives an inference response from the AI model in response to the inference request, transmits the inference response, and stores the inference request and the inference response in cache.

Abstract: A first compute server of a plurality of compute servers of a distributed cloud computing network receives an inference request. The first compute server determines that the received inference request triggers execution of code at the distributed cloud computing network, where the code is related to an artificial intelligence (AI) application that interacts with the inference request and causes input of the inference request to be run through an AI model. If the AI model is not loaded at the first compute server but is loaded at a second compute server, the inference request is routed to the second compute server for performing the inference operation.