Abstract: Disclosed is an SLO-aware artificial intelligence inference scheduler technology in a heterogeneous processor-based edge system. A scheduling method for a machine learning (ML) inference task, which is performed by a scheduling system, may include receiving inference task requests of multiple ML models with respect to an edge system composed of heterogeneous processors and operating heterogeneous processor resources of the edge system based on a service-level objective (SLO)-aware-based scheduling policy in response to the received inference task requests.

Abstract: An electronic device includes a System on Chip (SoC) and a memory. The SoC includes a processor and a neural processing unit (NPU). The memory includes an enclave page cache (EPC), in which a validation table is stored, and at least one NPU enclave. The NPU enclave and the EPC have a trusted execution environment, which is isolated from an execution environment in which system software of the CPU is executed.

Abstract: A device transmits or receives a packet in a memory network including one or more processors and/or one or more memory devices. The device includes a key storage unit configured to store a one-time password (OTP) key that is shared with a target node, an encryption unit configured to encrypt a transmission packet with the OTP key stored in the key storage unit and to transmit the encrypted transmission packet to the target node, and a decryption unit configured to decrypt a receiving packet from the target node with the OTP key stored in the key storage unit. The device is a processor or a memory device in the memory network.

Abstract: A device transmits or receives a packet in a memory network including one or more processors and/or one or more memory devices. The device includes a key storage unit configured to store a one-time password (OTP) key that is shared with a target node, an encryption unit configured to encrypt a transmission packet with the OTP key stored in the key storage unit and to transmit the encrypted transmission packet to the target node, and a decryption unit configured to decrypt a receiving packet from the target node with the OTP key stored in the key storage unit. The device is a processor or a memory device in the memory network.

Abstract: The present invention proposes a novel cache residence prediction mechanism that predicts whether requested data of a cache miss can be found in another cache. The memory controller can use the prediction result to determine if it should immediately initiate a memory access, or initiate no memory access until a cache snoop response shows that the requested data cannot be supplied by a cache. The cache residence prediction mechanism can be implemented at the cache side, the memory side, or both. A cache-side prediction mechanism can predict that data requested by a cache miss can be found in another cache if the cache miss address matches an address tag of a cache line in the requesting cache and the cache line is in an invalid state. A memory-side prediction mechanism can make effective prediction based on observed memory and cache operations that are recorded in a prediction table.

Abstract: In shared-memory multiprocessor systems, cache interventions from different sourcing caches can result in different cache intervention costs. With location-aware cache coherence, when a cache receives a data request, the cache can determine whether sourcing the data from the cache will result in less cache intervention cost than sourcing the data from another cache. The decision can be made based on appropriate information maintained in the cache or collected from snoop responses from other caches. If the requested data is found in more than one cache, the cache that has or likely has the lowest cache intervention cost is generally responsible for supplying the data. The intervention cost can be measured by performance metrics that include, but are not limited to, communication latency, bandwidth consumption, load balance, and power consumption.

Abstract: The present invention proposes a novel cache residence prediction mechanism that predicts whether requested data of a cache miss can be found in another cache. The memory controller can use the prediction result to determine if it should immediately initiate a memory access, or initiate no memory access until a cache snoop response shows that the requested data cannot be supplied by a cache. The cache residence prediction mechanism can be implemented at the cache side, the memory side, or both. A cache-side prediction mechanism can predict that data requested by a cache miss can be found in another cache if the cache miss address matches an address tag of a cache line in the requesting cache and the cache line is in an invalid state. A memory-side prediction mechanism can make effective prediction based on observed memory and cache operations that are recorded in a prediction table.

Abstract: The present invention proposes a novel cache residence prediction mechanism that predicts whether requested data of a cache miss can be found in another cache. The memory controller can use the prediction result to determine if it should immediately initiate a memory access, or initiate no memory access until a cache snoop response shows that the requested data cannot be supplied by a cache. The cache residence prediction mechanism can be implemented at the cache side, the memory side, or both. A cache-side prediction mechanism can predict that data requested by a cache miss can be found in another cache if the cache miss address matches an address tag of a cache line in the requesting cache and the cache line is in an invalid state. A memory-side prediction mechanism can make effective prediction based on observed memory and cache operations that are recorded in a prediction table.

Abstract: In shared-memory multiprocessor systems, cache interventions from different sourcing caches can result in different cache intervention costs. With location-aware cache coherence, when a cache receives a data request, the cache can determine whether sourcing the data from the cache will result in less cache intervention cost than sourcing the data from another cache. The decision can be made based on appropriate information maintained in the cache or collected from snoop responses from other caches. If the requested data is found in more than one cache, the cache that has or likely has the lowest cache intervention cost is generally responsible for supplying the data. The intervention cost can be measured by performance metrics that include, but are not limited to, communication latency, bandwidth consumption, load balance, and power consumption.

Abstract: The present invention proposes a novel cache residence prediction mechanism that predicts whether requested data of a cache miss can be found in another cache. The memory controller can use the prediction result to determine if it should immediately initiate a memory access, or initiate no memory access until a cache snoop response shows that the requested data cannot be supplied by a cache. The cache residence prediction mechanism can be implemented at the cache side, the memory side, or both. A cache-side prediction mechanism can predict that data requested by a cache miss can be found in another cache if the cache miss address matches an address tag of a cache line in the requesting cache and the cache line is in an invalid state. A memory-side prediction mechanism can make effective prediction based on observed memory and cache operations that are recorded in a prediction table.