Abstract: A heterogeneous memory system is implemented using a low-latency near memory (NM) and a high-latency far memory (FM). Pages in the memory system include NM blocks stored in the NM and FM blocks stored in the FM. A page is assigned to a region in the memory system based on the proportion of NM blocks in the page. When accessing a block, the block address is used to determine a region of the memory system, and a block offset is used to determine whether the block is stored in NM or FM. The memory system may observe memory accesses to determine the access statistics of the page and the block. Based on a page's hotness and access density, the page may be migrated to a different region. Based on a block's hotness, the block may be migrated between NM and FM allocated to the page.

Abstract: A heterogeneous memory system is implemented using a low-latency near memory (NM) and a high-latency far memory (FM). Pages in the memory system include NM blocks stored in the NM and FM blocks stored in the FM. A page is assigned to a region in the memory system based on the proportion of NM blocks in the page. When accessing a block, the block address is used to determine a region of the memory system, and a block offset is used to determine whether the block is stored in NM or FM. The memory system may observe memory accesses to determine the access statistics of the page and the block. Based on a page's hotness and access density, the page may be migrated to a different region. Based on a block's hotness, the block may be migrated between NM and FM allocated to the page.

Abstract: A heterogeneous memory system is implemented using a low-latency near memory (NM) and a high-latency far memory (FM). Pages in the memory system include NM blocks stored in the NM and FM blocks stored in the FM. A page is assigned to a region in the memory system based on the proportion of NM blocks in the page. When accessing a block, the block address is used to determine a region of the memory system, and a block offset is used to determine whether the block is stored in NM or FM. The memory system may observe memory accesses to determine the access statistics of the page and the block. Based on a page's hotness and access density, the page may be migrated to a different region. Based on a block's hotness, the block may be migrated between NM and FM allocated to the page.

Abstract: The described embodiments include a computing device that caches data acquired from a main memory in a high-bandwidth memory (HBM), the computing device including channels for accessing data stored in corresponding portions of the HBM. During operation, the computing device sets each of the channels so that data blocks stored in the corresponding portions of the HBM include corresponding numbers of cache lines. Based on records of accesses of cache lines in the HBM that were acquired from pages in the main memory, the computing device sets a data block size for each of the pages, the data block size being a number of cache lines. The computing device stores, in the HBM, data blocks acquired from each of the pages in the main memory using a channel having a data block size corresponding to the data block size for each of the pages.

Abstract: The present disclosure is directed to techniques for migrating data between heterogeneous memories in a computing system. More specifically, the techniques involve migrating data between a memory having better access characteristics (e.g., lower latency but greater capacity) and a memory having worse access characteristics (e.g., higher latency but lower capacity). Migrations occur with a variable migration granularity. A migration granularity specifies a number of memory pages, having virtual addresses that are contiguous in virtual address space, that are migrated in a single migration operation. A history-based technique that adjusts migration granularity based on the history of memory utilization by an application is provided. A profiling-based technique that adjusts migration granularity based on a profiling operation is also provided.

Abstract: A processor architecture utilizing a L3 translation lookaside buffer (TLB) to reduce page walks. The processor includes multiple cores, where each core includes a L1 TLB and a L2 TLB. The processor further includes a L3 TLB that is shared across the processor cores, where the L3 TLB is implemented in off-chip or die-stack dynamic random-access memory. Furthermore, the processor includes a page table connected to the L3 TLB, where the page table stores a mapping between virtual addresses and physical addresses. In such an architecture, by having the L3 TLB with a very large capacity, performance may be improved, such as execution time, by eliminating page walks, which requires multiple data accesses.

Abstract: A processor architecture which partitions on-chip data caches to efficiently cache translation entries alongside data which reduces conflicts between virtual to physical address translation and data accesses. The architecture includes processor cores that include a first level translation lookaside buffer (TLB) and a second level TLB located either internally within each processor core or shared across the processor cores. Furthermore, the architecture includes a second level data cache (e.g., located either internally within each processor core or shared across the processor cores) partitioned to store both data and translation entries. Furthermore, the architecture includes a third level data cache connected to the processor cores, where the third level data cache is partitioned to store both data and translation entries. The third level data cache is shared across the processor cores. The processor architecture can also include a data stack distance profiler and a translation stack distance profiler.

Abstract: A processor architecture which partitions the on-chip data caches to efficiently cache translation entries alongside data which reduces the conflicts between virtual to physical address translation and data accesses. The architecture includes processor cores that include a first level translation lookaside buffer (TLB) and a second level TLB located either internally within each processor core or shared across the processor cores. Furthermore, the architecture includes a second level data cache (e.g., located either internally within each processor core or shared across the processor cores) partitioned to store both data and translation entries. Furthermore, the architecture includes a third level data cache connected to the processor cores, where the third level data cache is partitioned to store both data and translation entries. The third level data cache is shared across the processor cores.

Abstract: The present disclosure is directed to techniques for migrating data between heterogeneous memories in a computing system. More specifically, the techniques involve migrating data between a memory having better access characteristics (e.g., lower latency but greater capacity) and a memory having worse access characteristics (e.g., higher latency but lower capacity). Migrations occur with a variable migration granularity. A migration granularity specifies a number of memory pages, having virtual addresses that are contiguous in virtual address space, that are migrated in a single migration operation. A history-based technique that adjusts migration granularity based on the history of memory utilization by an application is provided. A profiling-based technique that adjusts migration granularity based on a profiling operation is also provided.

Abstract: A heterogeneous memory system is implemented using a low-latency near memory (NM) and a high-latency far memory (FM). Pages in the memory system include NM blocks stored in the NM and FM blocks stored in the FM. A page is assigned to a region in the memory system based on the proportion of NM blocks in the page. When accessing a block, the block address is used to determine a region of the memory system, and a block offset is used to determine whether the block is stored in NM or FM. The memory system may observe memory accesses to determine the access statistics of the page and the block. Based on a page's hotness and access density, the page may be migrated to a different region. Based on a block's hotness, the block may be migrated between NM and FM allocated to the page.

Abstract: A processor architecture utilizing a L3 translation lookaside buffer (TLB) to reduce page walks. The processor includes multiple cores, where each core includes a L1 TLB and a L2 TLB. The processor further includes a L3 TLB that is shared across the processor cores, where the L3 TLB is implemented in off-chip or die-stack dynamic random-access memory. Furthermore, the processor includes a page table connected to the L3 TLB, where the page table stores a mapping between virtual addresses and physical addresses. In such an architecture, by having the L3 TLB with a very large capacity, performance may be improved, such as execution time, by eliminating page walks, which requires multiple data accesses.

Abstract: The described embodiments include a computing device that caches data acquired from a main memory in a high-bandwidth memory (HBM), the computing device including channels for accessing data stored in corresponding portions of the HBM. During operation, the computing device sets each of the channels so that data blocks stored in the corresponding portions of the HBM include corresponding numbers of cache lines. Based on records of accesses of cache lines in the HBM that were acquired from pages in the main memory, the computing device sets a data block size for each of the pages, the data block size being a number of cache lines. The computing device stores, in the HBM, data blocks acquired from each of the pages in the main memory using a channel having a data block size corresponding to the data block size for each of the pages.

Abstract: A memory subsystem incorporating a die-stacked DRAM (DSDRAM) is disclosed. In one embodiment, a system include a processor implemented on a silicon interposer of an integrated circuit (IC) package, a DSDRAM coupled to the processor, the DSDRAM implemented on the silicon interposer of the IC package, and a DRAM implemented separately from the IC package. The DSDRAM and the DRAM form a main memory having a contiguous address space comprising a range of physical addresses. The physical addresses of the DSDRAM occupy a first contiguous portion of the address space, while the DRAM occupies a second contiguous portion of the address space. Each physical address of the contiguous address space is augmented with a first bit that, when set, indicates that a page is stored in the DRAM and the DSDRAM.

Abstract: A memory subsystem incorporating a die-stacked DRAM (DSDRAM) is disclosed. In one embodiment, a system include a processor implemented on a silicon interposer of an integrated circuit (IC) package, a DSDRAM coupled to the processor, the DSDRAM implemented on the silicon interposer of the IC package, and a DRAM implemented separately from the IC package. The DSDRAM and the DRAM form a main memory having a contiguous address space comprising a range of physical addresses. The physical addresses of the DSDRAM occupy a first contiguous portion of the address space, while the DRAM occupies a second contiguous portion of the address space. Each physical address of the contiguous address space is augmented with a first bit that, when set, indicates that a page is stored in the DRAM and the DSDRAM.