Abstract: Disclosed techniques relate to supplemental (e.g., PC-based Bayesian) learning for a pattern-based prefetcher. In some embodiments, prefetch circuitry is configured to store access map data based on address information corresponding to memory accesses and generate, based on the access map data and stored access patterns, one or more prefetch requests to prefetch data to a cache. In some embodiments, supplemental access tracking circuitry is configured to store initial access tracking information for offsets corresponding to a first access map evicted from the prefetch circuitry, update the access tracking information based on one or more additional access maps evicted from the prefetch circuitry that match the stored program counter information, and generate, based on the updated access tracking information, a starting access map for allocation in the prefetch circuitry.

Abstract: A prefetcher for a coprocessor is disclosed. An apparatus includes a processor and a coprocessor that are configured to execute processor and coprocessor instructions, respectively. The processor and coprocessor instructions appear together in code sequences fetched by the processor, with the coprocessor instructions being provided to the coprocessor by the processor. The apparatus further includes a coprocessor prefetcher configured to monitor a code sequence fetched by the processor and, in response to identifying a presence of coprocessor instructions in the code sequence, capture the memory addresses, generated by the processor, of operand data for coprocessor instructions. The coprocessor is further configured to issue, for a cache memory accessible to the coprocessor, prefetches for data associated with the memory addresses prior to execution of the coprocessor instructions by the coprocessor.

Abstract: Disclosed techniques relate to supplemental PC-based learning for a pattern-based prefetcher. In some embodiments, prefetch circuitry is configured to store access map data based on address information corresponding to memory accesses and generate, based on the access map data and stored access patterns, one or more prefetch requests to prefetch data to a cache. In some embodiments, supplemental access tracking circuitry is configured to store initial access tracking information for offsets corresponding to a first access map evicted from the prefetch circuitry, store program counter information associated with an initial memory access that touched the first access map, update the access tracking information, based on one or more additional access maps evicted from the prefetch circuitry that match the stored program counter information, and generate, in response to a hit and based on the updated access tracking information, a starting access map for allocation in the prefetch circuitry.

Abstract: Disclosed techniques relate to adaptive adjustments of prefetch constraints. In some embodiments, prefetch circuitry is configured to store access map data based on address information corresponding to memory accesses and generate, based on the access map data and stored access patterns, one or more prefetch requests to prefetch data to a cache. Control circuitry may determine a number of active maps in the prefetch circuitry. In response to the number of active maps meeting a threshold, the control circuitry may impose a first restriction on a prefetch parameter for at least a first category of access maps. In response to the number of active maps in the prefetch circuitry failing to meet the threshold, the control circuitry may reduce the first restriction on the parameter. This may advantageously allow deep prefetching for certain workloads (e.g., with a small number of long-running streams) such as large-language model (LLM) workloads.

Abstract: Disclosed techniques relate to supplemental PC-based learning for a pattern-based prefetcher. In some embodiments, prefetch circuitry is configured to store access map data based on address information corresponding to memory accesses and generate, based on the access map data and stored access patterns, one or more prefetch requests to prefetch data to a cache. In some embodiments, supplemental access tracking circuitry is configured to store initial access tracking information for offsets corresponding to a first access map evicted from the prefetch circuitry, store program counter information associated with an initial memory access that touched the first access map, update the access tracking information, based on one or more additional access maps evicted from the prefetch circuitry that match the stored program counter information, and generate, in response to a hit and based on the updated access tracking information, a starting access map for allocation in the prefetch circuitry.

Abstract: Disclosed techniques related to compressed storage for access maps demoted from a prefetch table. In some embodiments, prefetch circuitry stores access map data based on address information corresponding to memory accesses, where a given access map of a plurality of stored access maps records categories of prior accesses to different offsets in a defined address region. The prefetch circuit may generate, based on the access map data and stored access patterns, one or more prefetch requests to prefetch data to a cache. Compressed map storage circuitry may store first data from a first access map, where the first data is a proper subset of data stored in the first access map prior to demotion of the first access map from the prefetch table. At least a portion of the first data may be a compressed version of data from the first access map.

Abstract: Disclosed techniques related to compressed storage for access maps demoted from a prefetch table. In some embodiments, prefetch circuitry stores access map data based on address information corresponding to memory accesses, where a given access map of a plurality of stored access maps records categories of prior accesses to different offsets in a defined address region. The prefetch circuit may generate, based on the access map data and stored access patterns, one or more prefetch requests to prefetch data to a cache. Compressed map storage circuitry may store first data from a first access map, where the first data is a proper subset of data stored in the first access map prior to demotion of the first access map from the prefetch table. At least a portion of the first data may be a compressed version of data from the first access map.

Abstract: A cache may store critical cache lines and non-critical cache lines, and may attempt to retain critical cache lines in the cache by, for example, favoring the critical cache lines in replacement data updates. Multiple levels of criticality may be available for a given cache line and cache circuitry may adjust the criticality value of in response to a criticality event. One or more upper criticality levels may be masked when selecting a victim cache line for replacement.

Abstract: A prefetcher for a coprocessor is disclosed. An apparatus includes a processor and a coprocessor that are configured to execute processor and coprocessor instructions, respectively. The processor and coprocessor instructions appear together in code sequences fetched by the processor, with the coprocessor instructions being provided to the coprocessor by the processor. The apparatus further includes a coprocessor prefetcher configured to monitor a code sequence fetched by the processor and, in response to identifying a presence of coprocessor instructions in the code sequence, capture the memory addresses, generated by the processor, of operand data for coprocessor instructions. The coprocessor is further configured to issue, for a cache memory accessible to the coprocessor, prefetches for data associated with the memory addresses prior to execution of the coprocessor instructions by the coprocessor.

Abstract: A cache may store critical cache lines and non-critical cache lines, and may attempt to retain critical cache lines in the cache by, for example, favoring the critical cache lines in replacement data updates. Multiple levels of criticality may be available for a given cache line and cache circuitry may adjust the criticality value of in response to a criticality event. One or more upper criticality levels may be masked when selecting a victim cache line for replacement.

Abstract: A prefetcher for a coprocessor is disclosed. An apparatus includes a processor and a coprocessor that are configured to execute processor and coprocessor instructions, respectively. The processor and coprocessor instructions appear together in code sequences fetched by the processor, with the coprocessor instructions being provided to the coprocessor by the processor. The apparatus further includes a coprocessor prefetcher configured to monitor a code sequence fetched by the processor and, in response to identifying a presence of coprocessor instructions in the code sequence, capture the memory addresses, generated by the processor, of operand data for coprocessor instructions. The coprocessor is further configured to issue, for a cache memory accessible to the coprocessor, prefetches for data associated with the memory addresses prior to execution of the coprocessor instructions by the coprocessor.

Abstract: A cache may store critical cache lines and non-critical cache lines, and may attempt to retain critical cache lines in the cache by, for example, favoring the critical cache lines in replacement data updates. Multiple levels of criticality may be available for a given cache line and cache circuitry may adjust the criticality value of in response to a criticality event. One or more upper criticality levels may be masked when selecting a victim cache line for replacement.

Abstract: A prefetcher for a coprocessor is disclosed. An apparatus includes a processor and a coprocessor that are configured to execute processor and coprocessor instructions, respectively. The processor and coprocessor instructions appear together in code sequences fetched by the processor, with the coprocessor instructions being provided to the coprocessor by the processor. The apparatus further includes a coprocessor prefetcher configured to monitor a code sequence fetched by the processor and, in response to identifying a presence of coprocessor instructions in the code sequence, capture the memory addresses, generated by the processor, of operand data for coprocessor instructions. The coprocessor is further configured to issue, for a cache memory accessible to the coprocessor, prefetches for data associated with the memory addresses prior to execution of the coprocessor instructions by the coprocessor.

Abstract: A prefetcher for a coprocessor is disclosed. An apparatus includes a processor and a coprocessor that are configured to execute processor and coprocessor instructions, respectively. The processor and coprocessor instructions appear together in code sequences fetched by the processor, with the coprocessor instructions being provided to the coprocessor by the processor. The apparatus further includes a coprocessor prefetcher configured to monitor a code sequence fetched by the processor and, in response to identifying a presence of coprocessor instructions in the code sequence, capture the memory addresses, generated by the processor, of operand data for coprocessor instructions. The coprocessor is further configured to issue, for a cache memory accessible to the coprocessor, prefetches for data associated with the memory addresses prior to execution of the coprocessor instructions by the coprocessor.

Abstract: A prefetcher for a coprocessor is disclosed. An apparatus includes a processor and a coprocessor that are configured to execute processor and coprocessor instructions, respectively. The processor and coprocessor instructions appear together in code sequences fetched by the processor, with the coprocessor instructions being provided to the coprocessor by the processor. The apparatus further includes a coprocessor prefetcher configured to monitor a code sequence fetched by the processor and, in response to identifying a presence of coprocessor instructions in the code sequence, capture the memory addresses, generated by the processor, of operand data for coprocessor instructions. The coprocessor is further configured to issue, for a cache memory accessible to the coprocessor, prefetches for data associated with the memory addresses prior to execution of the coprocessor instructions by the coprocessor.