MULTITHREADED PROCESSOR CORE WITH HARDWARE-ASSISTED TASK SCHEDULING
Embodiments of apparatuses, methods, and systems for a multithreaded processor core with hardware-assisted task scheduling are described. In an embodiment, a processor includes a first hardware thread, a second hardware thread, and a task manager. The task manager is to issue a task to the first hardware thread. The task manager includes a hardware task queue in which to store a plurality of task descriptors. Each of the task descriptors is to represent one of a single task, a collection of iterative tasks, and a linked list of tasks.
This invention was made with Government support under contract number HR0011-17-3-004 awarded by the Department of Defense. The Government has certain rights in this invention.
FIELD OF INVENTIONThe field of invention relates generally to computer architecture, and, more specifically, but without limitation, to processor core design.
BACKGROUNDThe performance of a computer system may depend on the suitability of the processor architecture for the applications to be executed. For example, a single instruction single data (SISD) architecture may be appropriate for general purpose computing whereas a single instruction multiple data (SIMD) architecture may be more appropriate for processing of media data.
The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:
In the following description, numerous specific details, such as component and system configurations, may be set forth in order to provide a more thorough understanding of the present invention. It will be appreciated, however, by one skilled in the art, that the invention may be practiced without such specific details. Additionally, some well-known structures, circuits, and other features have not been shown in detail, to avoid unnecessarily obscuring the present invention.
References to “one embodiment,” “an embodiment,” “example embodiment,” “various embodiments,” etc., indicate that the embodiment(s) of the invention so described may include particular features, structures, or characteristics, but more than one embodiment may and not every embodiment necessarily does include the particular features, structures, or characteristics. Some embodiments may have some, all, or none of the features described for other embodiments. Moreover, such phrases are not necessarily referring to the same embodiment. When a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
As used in this description and the claims and unless otherwise specified, the use of the ordinal adjectives “first,” “second,” “third,” etc. to describe an element merely indicate that a particular instance of an element or different instances of like elements are being referred to, and is not intended to imply that the elements so described must be in a particular sequence, either temporally, spatially, in ranking, or in any other manner.
Also, as used in descriptions of embodiments of the invention, a “I” character between terms may mean that an embodiment may include or be implemented using, with, and/or according to the first term and/or the second term (and/or any other additional terms).
As discussed in the background section, performance of a computer system may depend on the suitability of the processor architecture for the applications to be executed. The use of embodiments of the present invention may be desired for graph applications and other execution flows that are divergent and/or chaotic, as opposed to bulk synchronous parallel flows.
Core 100 includes single-threaded pipeline 102 and multithreaded pipelines 104, 106, and 108. Although core 100 shows one single-threaded pipeline and three multithreaded pipelines, a core according to an embodiment of the invention may include two or more single-threaded pipelines and no multithreaded pipelines, any number of multithreaded pipelines and no single-threaded pipelines, or any number of single-threaded pipelines and any number of multithreaded pipelines. In embodiments, each pipeline may be constructed of a small number of states, compared to a complex SIMD architecture. Pipelines 102, 104, 106, and 108 are interconnected through an on-core fabric, cross-bar, or other interconnect 110, which may be connected to an off-core fabric, cross-bar or other interconnect 112, such that pipelines 102, 104, 106, and 108 share a common path to system memory 120 and have a single, coherent view of system memory, including data cache traffic as illustrated in block 122. DRAM 120 may be connected to off-core fabric 112 through memory controller 124. Off-core fabric 112 may also connect core 100 to a network 126.
Core 100 may also include thread engine 130 to allocate and de-allocate hardware resources of pipelines 102, 104, 106, and 108. The instruction set architecture (ISA) of each of pipelines 102, 104, 106, and 108 is the same or compatible. Therefore, thread engine 130 may assign any thread to any pipeline and may switch any thread from any pipeline to any other pipeline. Thread engine 130 may perform thread assignment and/or switching to improve performance of any one or more threads, improve overall performance, reduce power consumption, or for any other reason.
Core 100 may also include scratchpad memory 140, memory engine 142, collecting engine 144, query engine 146, and task manager 148. Memory engine 142 may be to manage memory operations, such as direct memory access (DMA), for accelerator, input/output (I/O), and other peripheral devices. Collecting engine 144 may be to manage scatter/gather and/or other data collecting operations. Queue engine 146 may be to manage hardware assistance for queue operations or other stack-like or non-atomic software operations.
Pipeline 200 includes register file 260 such that each of pipelines 102, 104, 106, and 108 has its own independent register file to support multiple hardware threads per pipeline. Thread engine 130 may schedule threads at the core level instead of at a pipeline level, with no required affinities, such that new threads may be scheduled on any pipeline regardless of whether they are generated from outside the core or within a pipeline. Thread scheduling may be based on a series of heuristics to optimize placement of threads onto pipeline based on any desired metric. For example, for improved performance, threads may be distributed across pipelines, or for power reduction, threads may be targeted to an active pipeline instead of waking an inactive pipeline. Thread engine 130 may also recognize one or more conditions under which migration of a thread is favorable, pause the thread, copy the thread state to another pipeline, kill the old thread, and restart it as a new thread on the other pipeline.
Pipeline 200 also includes instruction cache 270, data cache 280, and shared memory port 290. Threads of applications, such as graph applications, with data access patterns for which the benefits of caching are limited may frequently access off-core memory structures (e.g., through shared memory port 290) and spend significant amounts of time waiting for memory traffic. In various embodiments of the invention, the number of single-threaded pipelines per core, the number of multithreaded pipelines per core, and the number of hardware threads per multithreaded pipeline may be chosen to provide hardware resources for a high level of interleaving between threads to saturate memory bandwidth and optimize performance for these types of applications.
Task manager 300 includes a hardware queue, task queue 310, to store task descriptors that represent three different collections of tasks: single tasks, iterative tasks, and linked tasks. As hardware threads on a processor or processor core, such as core 100, complete execution of tasks, they issue read requests to task manager 300. If task manager 300 has been supplied tasks through the task queue 310, it issues a task to the requesting hardware thread. Otherwise, the hardware thread remains in a waiting state until task manager 300 has a task to be assigned.
The use of embodiments of the invention may be desired to provide hardware assistance to task assignment and management. In an embodiment, an iterative task that spawns many tasks may be created with a single addition to task queue 310. In an embodiment, tasks awaiting completion of a given operation may be linked together in a linked list, as a linked task, and added to task queue 310 by a single addition of the head element.
In embodiments, load/store queue 320, which represents a pipeline load/store queue such as load/store queue 242, may request a new thread for its pipeline by issuing a read request to task manager 300. Then, the requesting hardware thread may be blocked and enter a low-power mode until task manager 300 issues a new task to the requesting pipeline. Implementing a new thread request as a read request to task manager 300 allows the requesting pipeline to respond to the new thread more quickly than it would in a software-based approach using an interrupt or polling mechanism.
Task manager 300 includes task finite state machine (FSM) 330, arbitrator 332, and response buffer 334. Arbitrator 332 accepts request for work from multiple hardware threads and holds the requests in a pending queue, buffer, or other storage structure until a new task is ready for each request. For each pending request, FSM 330 examines its internal buffer state to determine if a task descriptor in the response buffer 334 can be used to assign a task to the requesting hardware thread. If no such task exists in the response buffer 334 to fulfill the request, task manager 300 checks the task queue 310 for available task descriptors. Once a task descriptor is available on the task queue 310, FSM 330 copies the task descriptor from the task queue 310 to the response buffer 334 and assigns one task to the requesting hardware thread based on the task descriptor.
Task descriptors may include the fields shown in Table 1.
In various embodiments, task descriptors may include any of the fields shown in Table 1 and/or any other fields, and each field may be of the size indicated in Table 1 or of any other size.
The value in the type field may indicate that the task is one of three types: single, iterative, and linked list. A descriptor for a single task includes a single value or a pointer to a structure containing multiple values to be used as the task's initial state. When a single task is encountered, it is immediately processed and then disposed of. A descriptor for an iterative task includes a pointer to a structure containing multiple values to be used as a task's initial state, along with an offset to an iteration parameter within the structure and a maximum value to be used as an iteration limit for the iteration parameter. Iterative task descriptors assign tasks with unique iteration parameters to waiting hardware threads until the iteration limit has been reached. A descriptor for a linked list task includes a pointer to a structure containing both the task's internal state and a pointer to the next segment of internal state. Linked list tasks continue, following the list, until a terminating value (e.g., a null pointer) is reached.
In block 410, any ISA-compatible hardware thread in a processor core (e.g., core 100) having single-threaded and/or multithreaded pipelines requests work by issuing a read request from a load/store queue (e.g., load/store queue 242) of its pipeline (e.g., pipeline 200) to a hardware task manager (e.g., task manager 300). In block 412, the request is inserted into a request queue, buffer, or other storage structure.
In block 414, the task manager determines whether a request is available in the request queue. If not, then method 400 continues to block 416, in which the task manager waits until a request becomes available in the request queue. If so, then method 400 continues to block 420, in which the task manager determines whether a task is currently being processed, for example, by checking whether a task descriptor is available in a response buffer (e.g., response buffer 334).
If, in block 420, the task manager determines that no task is currently being processed, then method 400 continues in block 422, in which the task manager determines whether a task descriptor is available on a task queue (e.g., task queue 310). If so, then, method 400 continues in block 426, in which the available task descriptor is copied to the response buffer. If not, then method 400 continues in block 424, in which the task manager and the hardware thread wait until a task becomes available in the task queue, then to block 426. From block 426, method 400 continues to block 430.
If, in block 420, the task manager determines that a task is currently being processed, then method 400 continues in block 430.
In block 430, the task manager determines, based on a value in a type field of the task descriptor, whether the task descriptor represents a single task, a collection of iterative tasks, or a linked list of tasks.
If, in block 430, it is determined that the task descriptor represents a single task, then: in block 432, the task manager issues the single task for the current request, and in block 462, the task manager marks the task as processed.
If, in block 430, it is determined that the task descriptor represents a collection of iterative tasks, then: in block 440, the task manager issues an iterative task for the current request; in block 442, the task manager updates the iteration parameter; and, in block 444, the task manager determines whether the iteration limit has been reached. If so, then in block 462, the task manager marks the task as processed. If not, then in block 460, the task manager recognizes that the task is not complete.
If, in block 430, it is determined that the task descriptor represents a linked list of tasks, then: in block 450, the task manager issues a linked task for the current request; in block 452, the task manager fetches the next linked task; and, in block 454, the task manager determines whether the termination value has been reached. If so, then in block 462, the task manager marks the task as processed. If not, then in block 460, the task manager recognizes that the task is not complete.
From block 460 and/or block 462, method 400 returns to block 414.
Exemplary Computer Systems and ArchitecturesReferring now to
The optional nature of additional processors 515 is denoted in
The memory 540 may be, for example, dynamic random-access memory (DRAM), phase change memory (PCM), or a combination of the two. For at least one embodiment, the controller hub 520 communicates with the processor(s) 510, 515 via a multi-drop bus, such as a frontside bus (FSB), point-to-point interface such as QuickPath Interconnect (QPI), or similar connection 595.
In one embodiment, the coprocessor 545 is a special-purpose processor, such as, for example, a high-throughput MIC processor, a network or communication processor, compression engine, graphics processor, GPGPU, embedded processor, or the like. In one embodiment, controller hub 520 may include an integrated graphics accelerator.
There can be a variety of differences between the physical resources 510, 515 in terms of a spectrum of metrics of merit including architectural, microarchitectural, thermal, power consumption characteristics, and the like.
In one embodiment, the processor 510 executes instructions that control data processing operations of a general type. Embedded within the instructions may be coprocessor instructions. The processor 510 recognizes these coprocessor instructions as being of a type that should be executed by the attached coprocessor 545. Accordingly, the processor 510 issues these coprocessor instructions (or control signals representing coprocessor instructions) on a coprocessor bus or other interconnect, to coprocessor 545. Coprocessor(s) 545 accept and execute the received coprocessor instructions.
Referring now to
Processors 670 and 680 are shown including integrated memory controller (IMC) units 672 and 682, respectively. Processor 670 also includes as part of its bus controller units point-to-point (P-P) interfaces 676 and 678; similarly, second processor 680 includes P-P interfaces 686 and 688. Processors 670, 680 may exchange information via a point-to-point (P-P) interface 650 using P-P interface circuits 678, 688. As shown in
Processors 670, 680 may each exchange information with a chipset 690 via individual P-P interfaces 652, 654 using point to point interface circuits 676, 694, 686, 698. Chipset 690 may optionally exchange information with the coprocessor 638 via a high-performance interface 692. In one embodiment, the coprocessor 638 is a special-purpose processor, such as, for example, a high-throughput MIC processor, a network or communication processor, compression engine, graphics processor, GPGPU, embedded processor, or the like.
A shared cache (not shown) may be included in either processor or outside of both processors, yet connected with the processors via P-P interconnect, such that either or both processors' local cache information may be stored in the shared cache if a processor is placed into a low power mode.
Chipset 690 may be coupled to a first bus 616 via an interface 696. In one embodiment, first bus 616 may be a Peripheral Component Interconnect (PCI) bus, or a bus such as a PCI Express bus or another third generation I/O interconnect bus, although the scope of the present invention is not so limited.
As shown in
Referring now to
Referring now to
Embodiments of the mechanisms disclosed herein may be implemented in hardware, software, firmware, or a combination of such implementation approaches. Embodiments of the invention may be implemented as computer programs or program code executing on programmable systems comprising at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device.
Program code, such as code 630 illustrated in
The program code may be implemented in a high level procedural or object-oriented programming language to communicate with a processing system. The program code may also be implemented in assembly or machine language, if desired. In fact, the mechanisms described herein are not limited in scope to any particular programming language. In any case, the language may be a compiled or interpreted language.
One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium which represents various logic within the processor, which when read by a machine causes the machine to fabricate logic to perform the techniques described herein. Such representations, known as “IP cores” may be stored on a tangible, machine readable medium and supplied to various customers or manufacturing facilities to load into the fabrication machines that actually make the logic or processor.
Such machine-readable storage media may include, without limitation, non-transitory, tangible arrangements of articles manufactured or formed by a machine or device, including storage media such as hard disks, any other type of disk including floppy disks, optical disks, compact disk read-only memories (CD-ROMs), compact disk rewritables (CD-RWs), and magneto-optical disks, semiconductor devices such as read-only memories (ROMs), random access memories (RAMS) such as dynamic random access memories (DRAMs), static random access memories (SRAMs), erasable programmable read-only memories (EPROMs), flash memories, electrically erasable programmable read-only memories (EEPROMs), phase change memory (PCM), magnetic or optical cards, or any other type of media suitable for storing electronic instructions.
Accordingly, embodiments of the invention also include non-transitory, tangible machine-readable media containing instructions or containing design data, such as Hardware Description Language (HDL), which defines structures, circuits, apparatuses, processors and/or system features described herein. Such embodiments may also be referred to as program products.
In an embodiment, a processor includes multiple hardware threads and a task manager. The task manager is to issue a task to a first hardware thread. The task manager includes a hardware task queue in which to store a plurality of task descriptors. Each of the task descriptors is to represent one of a single task, a collection of iterative tasks, and a linked list of tasks.
In various embodiments, any or any combination of the following may also apply. A hardware thread may include a load/store queue to request the task from the task manager by issuing a read request to the task manager. A task descriptor representing a collection of iterative tasks may include a count value to specify a number of iterations. A task descriptor of a linked list of tasks may include a pointer to a head of the linked list of tasks. An instruction set architecture of the first hardware thread and an instruction set architecture of the second hardware thread may be compatible. The processor may also include a thread engine to migrate a software thread from the first hardware thread to a second hardware thread. The thread engine may migrate the software thread based on unequal processing needs, for example, to improve performance of a graph application. The first hardware thread may be in a first single-threaded pipeline and the second hardware thread may be in a second single-threaded pipeline. The first hardware thread and the second hardware thread may be in a single multithreaded pipeline. The first hardware thread may be in a first multithreaded pipeline and the second hardware thread may be in a second multithreaded pipeline. Only one of the first hardware thread and the second hardware thread may be in a single-threaded pipeline and the other of the first hardware thread and the second hardware thread may be in a multithreaded pipeline. The single multithreaded pipeline may restrict software threads to one outstanding operation at a time. A task descriptor may represent a single task or multiple tasks. A task descriptor representing multiple tasks (a multi-task descriptor) may represent a collection of iterative tasks, a linked list of tasks, or any other collection, list, or group of tasks.
In an embodiment, a method may include requesting, by a hardware thread of multithreaded processor core, work from a task manager; reading, by the task manager, a task descriptor from a hardware queue; and issuing, by the task manager, a task described by the task descriptor.
In various method embodiments, any or any combination of the following may also apply. The method may also include determining, by the task manager, that no existing task is being processed before reading the task descriptor from the hardware queue. The hardware thread may request work from the task manager by sending, by a load/store queue, a read request to the task manager. The task descriptor may specify that the task is a single task type. The task descriptor may specify that the task is one of a collection of iterative tasks and may specify a count value to indicate a number of iterations. The task descriptor may specify that the task is one of a linked list of tasks and may specify a pointer to a head of the linked list of tasks. A task descriptor may represent a single task or multiple tasks. A task descriptor representing multiple tasks (a multi-task descriptor) may represent a collection of iterative tasks, a linked list of tasks, or any other collection, list, or group of tasks.
In embodiments, an apparatus may include means for performing any of the methods described above. In embodiments, a machine-readable tangible medium may store instructions, which, when executed by a machine, cause the machine to perform any of the methods described above.
In an embodiment, a system may include a processor having multiple hardware threads and a task manager. The task manager is to issue a task to a first hardware thread. The task manager includes a hardware task queue in which to store a plurality of task descriptors. Each of the task descriptors is to represent one of a single task, a collection of iterative tasks, and a linked list of tasks. The system may also include a system memory coupled to the processor, wherein the first hardware thread and the second hardware thread have a coherent view of the system memory.
In system embodiments, as in apparatus and other embodiments, any or any combination of the following may also apply. A hardware thread may include a load/store queue to request the task from the task manager by issuing a read request to the task manager. A task descriptor representing a collection of iterative tasks may include a count value to specify a number of iterations. A task descriptor of a linked list of tasks may include a pointer to a head of the linked list of tasks. An instruction set architecture of the first hardware thread and an instruction set architecture of the second hardware thread may be compatible. The processor may also include a thread engine to migrate a software thread from the first hardware thread to a second hardware thread. The thread engine may migrate the software thread based on unequal processing needs, for example, to improve performance of a graph application. The first hardware thread may be in a first single-threaded pipeline and the second hardware thread may be in a second single-threaded pipeline. The first hardware thread and the second hardware thread may be in a single multithreaded pipeline. The first hardware thread may be in a first multithreaded pipeline and the second hardware thread may be in a second multithreaded pipeline. Only one of the first hardware thread and the second hardware thread may be in a single-threaded pipeline and the other of the first hardware thread and the second hardware thread may be in a multithreaded pipeline. The single multithreaded pipeline may restrict software threads to one outstanding operation at a time. A task descriptor may represent a single task or multiple tasks. A task descriptor representing multiple tasks (a multi-task descriptor) may represent a collection of iterative tasks, a linked list of tasks, or any other collection, list, or group of tasks.
Claims
1. A processor comprising:
- a first hardware thread;
- a second hardware thread; and
- a task manager to issue a task to the first hardware thread, the task manager including a hardware task queue in which to store a plurality of task descriptors, each of the task descriptors to represent one of a single task, a collection of iterative tasks, and a linked list of tasks.
2. The processor of claim 1, wherein the first hardware thread includes a load/store queue to request the task from the task manager by issuing a read request to the task manager.
3. The processor of claim 1, wherein a task descriptor representing a collection of iterative tasks is to include a count value to specify a number of iterations.
4. The processor of claim 1, wherein a task descriptor of a linked list of tasks is to include a pointer to a head of the linked list of tasks.
5. The processor of claim 1, wherein an instruction set architecture of the first hardware thread and an instruction set architecture of the second hardware thread are compatible.
6. The processor of claim 5, further comprising a thread engine to migrate a software thread from the first hardware thread to the second hardware thread.
7. The processor of claim 6, wherein the thread engine is to migrate the software thread to improve performance of a graph application.
8. The processor of claim 6, wherein the first hardware thread is in a first single-threaded pipeline and the second hardware thread is in a second single-threaded pipeline.
9. The processor of claim 6, wherein the first hardware thread and the second hardware thread are in a single multithreaded pipeline.
10. The processor of claim 6, wherein the first hardware thread is in a first multithreaded pipeline and the second hardware thread is in a second multithreaded pipeline.
11. The processor of claim 6, wherein only one of the first hardware thread and the second hardware thread is in a single-threaded pipeline and the other of the first hardware thread and the second hardware thread is in a multithreaded pipeline.
12. The processor of claim 9, wherein the single multithreaded pipeline restricts software threads to one outstanding operation at a time.
13. A method comprising:
- requesting, by a hardware thread of a multithreaded processor core, work from a task manager;
- reading, by the task manager, a task descriptor from a hardware queue; and
- issuing, by the task manager, a task described by the task descriptor.
14. The method of claim 13, further comprising determining, by the task manager, that no existing task is being processed before reading the task descriptor from the hardware queue.
15. The method of claim 13, wherein the hardware thread requests work from the task manager by sending, by a load/store queue, a read request to the task manager.
16. The method of claim 13, wherein the task descriptor specifies that the task is a single task type.
17. The method of claim 13, wherein the task descriptor:
- specifies that the task is one of a collection of iterative tasks; and
- specifies a count value to indicate a number of iterations.
18. The method of claim 13, wherein the task descriptor:
- specifies that the task is one of a linked list of tasks; and
- specifies a pointer to a head of the linked list of tasks.
19. A system comprising:
- a processor including: a first hardware thread; a second hardware thread; and a task manager to issue a task to the first hardware thread, the task manager including a hardware task queue in which to store a plurality of task descriptors, each of the task descriptors to represent one of a single task, a collection of iterative tasks, and a linked list of tasks; and
- a system memory coupled to the processor, wherein the first hardware thread and the second hardware thread have a coherent view of the system memory.
20. The system of claim 19, wherein the processor also includes a thread engine to migrate a software thread from the first hardware thread to the second hardware thread to improve performance of a graph application.
Type: Application
Filed: Jun 29, 2018
Publication Date: Jan 2, 2020
Patent Grant number: 11360809
Inventors: Paul Griffin (Portland, OR), Joshua Fryman (Corvallis, OR), Jason Howard (Portland, OR), Sang Phill Park (Hillsboro, OR), Robert Pawlowski (Portland, OR), Michael Abbott (Hillsboro, OR), Scott Cline (Beaverton, OR), Samkit Jain (Hillsboro, OR), Ankit More (San Francisco, CA), Vincent Cave (Hillsboro, OR), Fabrizio Petrini (Palo Alto, CA), Ivan Ganev (Hillsboro, OR)
Application Number: 16/024,343