Speculative Throughput Computing
Systems, methods, and apparatuses including computer program products for speculative throughput computing are disclosed. Speculative throughput computing is used to translate a program to execute on a plurality of processors, processor cores, or threads.
Latest NEMA LABS AB Patents:
This application claims priority to U.S. Provisional Application Ser. No. 60/897,969, for “System, methods, and business ideas for speculative execution of program segments in multiprocessors,” filed on Jan. 30, 2007, which provisional patent application is incorporated by reference herein in its entirety.
TECHNICAL FIELDThis subject matter generally relates to throughput computing.
BACKGROUNDA program (e.g., computer application) can be partitioned into a plurality of program segments. For example, a program can be partitioned into program segments P1, P2, . . . , PN, where N is a number of program segments. Conventional computing systems can execute the program segments one after another in an enumerated order. For example, a single processor computing system can execute P1 before executing P2, P2 before executing P3, and PN-1 before executing PN. Executing program segments in this order respects sequential semantics (e.g., a program segment with a higher enumeration order x reads from a memory location before a program segment with a lower enumeration order y writes to the memory location, where x>y).
For example, a first program segment can have an enumeration order i, and a second program segment can have an enumeration order j, where i<j. The first program segment and the second program segment can be executed in parallel without violating sequential semantics if the program segments do not access the same memory locations. Furthermore, sequential semantics is not violated if the first program segment does not write to a memory location after the second program segment reads from the memory location.
Multiprocessor, multicore, or multithreading computing systems can execute program segments in parallel (e.g., executing program segments at substantially the same time) on a plurality of processors, processor cores, or threads. Executing program segments in parallel that were not originally designed to execute in parallel can be referred to as “speculative execution.”
Conventional compilers can partition a program into program segments by determining which program segments access the same memory locations. Due to limitations of conventional analysis methods, or because accessed memory locations are unknown at a time of compiling, many programs cannot be partitioned by conventional compilers to allow for parallel execution of the program segments.
For example, some conventional analysis methods execute write instructions of program segments at temporary memory locations. These conventional analysis methods create execution overhead associated with using the temporary memory locations (e.g., storing and moving data from the temporary memory locations). Other conventional analysis methods use centralized data structures to store original data of memory locations where write instructions of program segments write, so that the original data may be restored. Updating the centralized structure can cause excessive overhead, especially if the write log is implemented in software using a dedicated data structure. Furthermore, if a miss-speculation occurs (e.g., when a program segment with an enumeration order i has written to a location that a program segment with an enumeration order j has already read, where i<j), program segments with an enumeration order higher (e.g., greater) than j halt and redo their executions. Halting and redoing executions causes execution overhead that can make speculative execution inefficient.
Furthermore, typical hardware and software implementations of conventional analysis methods use complex mechanisms and are inefficient. For example, typical software implementations create execution overhead because they use extra instructions, and cause poor memory system performance by lowering the memory locality, which can result in cache misses.
Some implementations monitor fixed regions of memory (e.g., a fixed range of one or more consecutive memory locations) to track if the region has been modified. A region size that is too large can result in false determinations of violations of sequential semantics. For example, a program segment may be forced to halt and redo its execution if it accesses the same region as another program segment with a lower enumeration order, even if none of the program segments accesses the same location. Alternatively, a region size that is too small increases the overhead of monitoring read and write instructions.
In addition, conventional profiling methods (e.g., test executions of programs to determine properties of a program to predict gains from speculative execution of the programs) assume a single method for speculative execution. Methods for speculative execution are also referred to as “speculative methods” or “processes for speculative throughput computing.” Furthermore, conventional dependence analyzers often are not able to determine whether the program segments may be executed in parallel.
SUMMARYSystems, methods, and apparatuses including computer program products for speculative throughput computing are disclosed. Speculative throughput computing is used to translate a program to execute on a plurality of processors, processor cores, or threads.
Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages. An advantage of speculative throughput computing is that it reduces execution overheads by using speculative read and write computer instructions.
An additional advantage of speculative throughput computing is that it reduces execution overhead related to commit operations by assuming that a speculation is valid, and by restoring content of speculatively modified memory locations using a decentralized scheme.
An additional advantage of speculative throughput computing is that it determines speculative parallelism by executing program segments out of sequential order.
An additional advantage of speculative throughput computing is that it increases the accuracy of determining violations of sequential semantics and reduces execution overhead by maintaining a precise range of locations that have been speculatively accessed.
An additional advantage of speculative throughput computing is that it increases speedup gains from speculative execution by selecting one of a plurality of speculative methods.
An additional advantage of speculative throughput computing is that by increasing speedup gains, speculative throughput computing lowers an operating frequency of a system and reduces energy consumption.
In some implementations, additional hierarchies of memory can be used in the multiprocessor computing system 100. For example, the memory 110 can be a secondary cache that is coupled to additional memory. In some implementations, the cache memory includes local memory that can be accessed by a processor coupled to the local memory. For example, a read or write instruction that is executed by the processor 131 can access local memory coupled to the processor 132 by invoking a software routine that sends a signal to the processor 132. The signal can invoke a software routine executed by the processor 132, where the processor 132 accesses the local memory and returns a value to the processor 131 by sending a signal to the processor 132.
Coherence (e.g., cache coherence) is maintained in the cache memory. In some implementations, a write-invalidate cache coherence mechanism can be used to maintain cache coherence. The write-invalidate cache coherence mechanism can invalidate a block of memory (e.g., contiguous locations of memory) in a cache memory when a processor coupled to another cache memory writes to the block of memory. In some implementations, a write-update cache coherence mechanism can be used to maintain cache coherence. In particular, a block of memory is updated when a processor coupled to another cache modifies the block of memory. In some implementations, a distribution protocol of invalidate and update requests can be one-to-all (e.g., snoopy cache protocols). In some implementations, the distribution protocol of invalidate and update requests can be one-to-one (e.g., directory-based protocols).
In some implementations, one or more of the processors can include a plurality of independent processor cores (e.g., a multi-core processor). For example, a dual-core processor includes two processors cores, and a quad-core processor includes four processor cores. The processor cores can execute a plurality of threads (e.g., program segments created by forks or splits of a program, or threads of execution) in parallel.
Program Translation OverviewThe system analyzes 210 data dependence of program segments of a program. In particular, the system determines whether program segments of the program, that can be sequentially executed (e.g., one after another) on a single processor computing system, can be executed in parallel (e.g., on a plurality of processors in a multiprocessor computing system, on a plurality of processor cores on a single processor, on a plurality of threads).
The system determines one or more methods to speculatively execute read and write instructions of program segments in parallel for use in the translation. In particular, the system profiles 220 the program. During a profiling pass, the system executes the program to collect statistics. Some examples of statistics include but are not limited to: a number of instructions executed in a program segment, an average number of cycles needed by each instruction, and a number of speculative reads and writes in a program segment. The system then selects 230 one or more speculative methods to use for translation. In some implementations, the system selects a combination of speculative methods.
Speculative Throughput ComputingIn some implementations, the system generates a precise range of locations that speculative read or write instructions have accessed for each processor in a multiprocessor, processor core in a multi-core processor, or thread. Other implementations are possible.
If a speculative write instruction has accessed a first precise range of locations corresponding to a first program segment or a second precise range of locations corresponding to a second program segment, the system can compare 320 the first precise range of locations with the second precise range of locations. In particular, the system can determine if a speculative execution of the program segments in parallel respects sequential semantics. For example, the first program segment and the second program segment can be executed in parallel. The first program segment can have an enumeration order less than an enumeration order of the second program segment. If the second program segment reads from a memory location before the first program segment writes to the memory location, then sequential semantics has been violated.
Sequential semantics can be violated if the ranges of locations overlap. For example, if the first precise range of locations overlaps with the second precise range of locations, then the first program segment and second program segment may have accessed the same memory location. If the first precise range of locations overlaps with the second precise range of locations and a location in the first precise range or the second precise range has been modified, the system identifies 330 a miss-speculation (e.g., a speculation that may not conform to sequential semantics).
If a miss-speculation is identified, the system restores 340 the memory content of locations speculative write instructions have accessed. Implementations of the process 300 for speculative throughput computing will be described in further detail with respect to
In some implementations, the data structure 410 is a table with a number of entries. Each entry can include fields. Some examples of fields include but are not limited to: an instruction address field 411, a data validity field 412, a maximum data address field 413, a minimum data address field 414, and an indicator field 415. The instruction address field 411 (e.g., “TAG” field) can store an instruction address (e.g., a 64-bit address that identifies a location of an instruction in memory). The data validity field 412 (e.g., “V” field) can store, for example, a single bit that indicates whether the data stored in an entry is valid. The maximum data address field 413 (e.g., “MAX” field) and the minimum data address field 414 (e.g., “MIN” field) can store data addresses. In particular, a maximum data address is stored in the maximum data address field 413 and a minimum data address is stored in the minimum data address field 414. The maximum data address field 413 and minimum data address field 414 can be used to define a precise range of locations, as described above with reference to
In some implementations, the write log 420 can be a table with entries that can store addresses and data associated with write instructions. In some implementations, a register can be used to store the write log pointer 430. In particular, the register can store an index of a next free entry in the write log 420.
The data associated with the write instructions can include an old value (e.g., a value of a location before a write instruction is executed), a new value (e.g., a value a write instruction will write to the location), and one or more status bits. For example, if a write instruction is executed, a value (e.g., new value) is written to a memory location. The original value (e.g., value in the memory location before the write instruction was executed) is stored as the old value.
The size of the old values and new values (e.g., number of bits that are used to store the value) can be selected for different architectures. For example, a 32-bit architecture can include old values and new values that are 32-bit values. As another example, a 64-bit architecture can include old values and new values that are 64-bit values.
The status bits can indicate whether the data stored in the old value and new value is valid. The status bits can also be used when deriving the old value of one or more memory locations. The number of status bits can depend on the size of the values (e.g., old values, and new values). For example, each entry in the write log can include one status bit per addressable unit (e.g., a status bit for each 8-bit value, or byte). In some implementations, unused bits or bytes in a log entry are filled with placeholder bits or bytes. For example, placeholders bits or bytes can be the same bit or byte values used to fill the unused bits or bytes in both the old value and the new value.
The system can distinguish between two types of memory instructions: regular read and write instructions (e.g., read and write instructions that are not speculative) and speculative read and write instructions. The system determines 501 if a read or write instruction is speculative. If the read or write instruction is not speculative (“No” branch of step 501), the read or write instruction is executed as usual (e.g., executed as a regular read or write instruction). Then, the system executes 513 a next instruction.
If the read or write instruction is speculative (“Yes” branch of step 501), a speculative read or write instruction is executed. The system locates 502 the speculative read or write instruction in the data structure (e.g., data structure 410 of
If the execution of the speculative read or write instruction is a first execution of the speculative read or write instruction (e.g., a match is not located in the data structure; “No” branch of step 503), then the system stores 504 the speculative read or write instruction. In particular, the system compares the instruction address of the speculative read or write instruction to the TAG fields of all of the entries in the data structure. If the execution of the speculative read or write instruction is a first execution of the speculative read or write instruction (e.g., the instruction address does not match any of the TAG fields), then the speculative read or write instruction is stored in the data structure. The system can allocate a new entry or use an entry that was evicted earlier.
If the execution of the speculative read or write instruction is not the first execution (e.g., a match is located in the data structure; “Yes” branch of step 503), the system compares an effective address of the speculative read or write instruction (e.g., a memory address that the speculative read or write instruction will access) with a maximum data address and a minimum data address. In particular, the system retrieves 505 values from the MAX field and MIN field in the data structure. If the effective address (e.g., ADDR of
The system determines whether or not the speculative read or write instruction is a speculative write instruction. If the speculative read or write instruction is not a speculative write instruction (“No” branch of step 510), then the system performs step 513. If the speculative read or write instruction is a speculative write instruction (“Yes” branch of step 510), then the system sets 511 an indicator (e.g., a bit in the “RW” field) to identify a speculative write instruction. In addition, the system stores 512 the write, or the effective address and the data associated with the write instruction, in the write log (e.g., write log 420). In particular, the effective address and the data associated with the write log can be stored in a write log entry pointed to by the write log pointer (e.g., write log pointer 430 of
Referring to
In addition, the system determines if there are entries in a second data structure that have not been compared to the entry determined in step 602. If there are not more entries in the second data structure (“No” branch of step 604), the system returns to step 602. If there are more entries (“Yes” branch of step 604), the system retrieves 605 a maximum data address (e.g., N.MAX), a minimum data address (e.g., N.MIN), and an indicator (e.g., N.RW) in the data structure for a second processor, processor core, or thread. The system compares the maximum data address and the minimum data address from the first processor, processor core, or thread with the maximum data address and minimum data address from the second processor, processor core, or thread.
In particular, if the indicator in the data structure for the first processor, processor core, or thread is set, or the indicator in the data structure for a second processor, processor core, or thread is set (“Yes” branch of step 606); then a speculative write instruction has accessed the first precise range of locations corresponding to a first program segment, or the second precise range of locations corresponding to a second program segment, respectively. If the indicators are not set (“No” branch of step 606), then the system returns to step 604.
If the maximum data address from the first processor is less than the maximum data address from the second processor, processor core, or thread; and the maximum data address from the first processor is greater than the minimum data address from the second processor (“Yes” branch of step 607); then the first precise range of locations overlaps with the second precise range of locations, and the system identifies 609 a miss-speculation. Otherwise (“No” branch of step 607), the system performs step 608.
If the minimum data address from the first process is greater than the minimum data address from the second processor, processor core, or thread; and the minimum data address from the first processor is less than the maximum data address from the second processor (“Yes” branch of step 608); then the first precise range of locations overlaps with the second precise range of locations, and the system identifies 609 a miss-speculation. Otherwise (“No” branch of step 608), the system returns to step 604. The system compares each entry for a processor, processor core, or thread with each entry in all of the other processors, processor cores, or threads, in this manner.
If a miss-speculation is identified, the system restores memory content of locations speculative write instructions have accessed.
The system determines a processor, processor core, or thread that wrote a final value to a location in memory. The system stores a value of the location and an address of the location. For example, the system can store 720 the value of the location in an “Actual” register and the address of the location (e.g., current address) in a “Current address” register. The value in the Actual register is compared 730 with all of the write log entries in all of the write logs. If the current address is contained in a write log entry, a new value corresponding to the current address is equal to the value of the location (e.g., value in the Actual register), and the new value is not equal to a corresponding old value, then the system determines (“Yes” branch of step 740) a matching entry. The system stores the matching entry in a data structure to book-keep the matching entries (e.g., back-track stack 800 of
Referring to
Returning to
An identifier of the particular processor, processor core, or thread (e.g., processor number one) is stored 790 in the Path field. The matching write log entry for the particular processor, processor core, or thread can be called the current entry. In some implementations, a checked bit for the current address in the current entry is marked 790.
The system determines a next value. The next value is an old value stored in the current entry in the write log(e.g., Actual.old). The next value is stored 795 in the Actual register. If there are matching entries remaining (e.g., the system determines another matching entry; “Yes” branch of step 740), the system returns to step 780.
If there is not another matching entry (“No” branch of step 740), the system determines if all entries have been checked. In particular, the system checks whether all write log entries containing the current address have the checked bit set, or if the old values and new values for the current address are equal. If all entries have been checked (“Yes” branch of step 750), then the system restores 760 a memory value and terminates 710. In particular, the system stores the Actual value in the location, thereby restoring the original value of the location.
If all entries have not been checked (e.g., determining that a write log entry containing the current address does not have a checked bit set, and the old values and new values for the current address are not equal; “No” branch of step 750), the system returns, or back-tracks, 770 to the previous entry in the back-track stack that included multiple matching entries (e.g., more than one of the processor, processor core, or thread fields are non-empty). The last entry used is pointed to by a pointer (e.g., pointer 840 of
The process for speculative throughput computing can be implemented in software. For example, the access matrix table can be implemented as a data structure in virtual memory that can be accessed by regular read and write instructions. The speculative read and write instructions can be emulated using a sequence of regular computer instructions that access the data structure in virtual memory (e.g., global variables, and pointer structures).
Each element can include but is not limited to: a maximum data address (e.g., MAX entity), a minimum data address (e.g., MIN entity), and an indicator (e.g., W entity). The MAX entity can store a maximum data address accessed by a corresponding program segment. The MIN entity can store a minimum data address accessed by a corresponding program segment. The W entity can identify whether a location between the maximum data address and the minimum data address has been modified (e.g., accessed by a write instruction) by the corresponding program segment.
The software implementation can also generate a write log and write log pointer in virtual memory that is analogous to the write log 420 and write log pointer 430 of
In a software implementation, a speculative read or write instruction can be emulated using regular read or write instructions (e.g., load or store instructions). For example, the speculative read or write instructions can be regular read or write instructions that are augmented with a sequence of ordinary instructions (e.g., checking code). The checking code can be used to determine 1020 if a read or write instruction is speculative. If the read or write instruction is not speculative (“No” branch of step 1020), then the read or write instruction is executed as usual (e.g., executed as a regular read or write instruction) and the system executes 1010 a next instruction.
If the read or write instruction is speculative (“Yes” branch of step 1020), then the system compares an effective address (e.g., ADDR of
The system determines whether or not the speculative read or write instruction is a speculative write instruction. If the speculative read or write instruction is not a speculative write instruction (“No” branch of step 1070), then the system executes 1010 a next instruction. If the speculative read or write instruction is a speculative write instruction (“Yes” branch of step 1070), then the system sets 1080 an indicator (e.g., a “W” bit) to identify a speculative write instruction. In addition, the system stores 1090 the write, or the effective address and the data associated with the write instruction, in a write log (e.g., a write log analogous to write log 420, but implemented in virtual memory). In particular, the effective address and the data associated with the write log can be stored in a write log entry pointed to by a write log pointer, and the write log pointer is incremented to point to a next free write log entry. Then, the system returns to step 1010.
Referring to
The system determines if there is a harmful overlap. In particular, if the indicator in the first element is set, or the indicator in the next element is set; then a speculative write instruction has accessed the first precise range of locations corresponding to the first program segment, or the second precise range of locations corresponding to the second program segment, respectively. If the maximum data address from the first element is less than the maximum data address from the next element; and the maximum data address from the first element is greater than the minimum data address from the next element; then the first precise range of locations overlaps with the second precise range of locations. The system determines (“Yes” branch of step 1120) a harmful overlap, and the system identifies 1140 a miss-speculation. If the minimum data address from the first element is greater than the minimum data address from the next element; and the minimum data address from the first element is less than the maximum data address from the next element; then the first precise range of locations overlaps with the second precise range of locations. The system determines a harmful overlap (“Yes” branch of step 1120), and the system identifies 1140 a miss-speculation.
If the system does not determine a harmful overlap (“No” branch of step 1120), the system determines if there are more entries for the pointer. The system compares each element associated with a pointer with all of the other elements associated with the pointer, in this manner. For example, assume that a row of a current pointer includes elements A, B, and C. The system compares pairs A with B, A with C, and B with C. If there are more entries for the pointer (“Yes” branch of step 1130), then the system returns to step 1115.
If there are not more entries for the pointer (“No” branch of step 1130), then the system determines and stores 1135 the values (e.g., MAX entity, MIN entity, and W entity) in the SUM element for the row. In particular, the system determines the maximum of all the MAX entities (e.g., highest maximum data address) and the minimum of all the MIN entities (e.g., lowest minimum data address) in the row. For example, assume that a row includes elements A, B, and C. The determined maximum is the highest MAX value of A, B, and C; and the determined minimum will be the lowest MIN value of A, B, and C. The determined maximum and minimum values are stored in the MAX and MIN entities of the SUM element of the row.
The system computes the W entity as the logical OR operation of the W entities of all the elements in the row. For example, assume that a row includes elements A, B, and C. The W entity of the SUM element is computed using the expression:
W(A) OR W(B) OR W(C), where
W(x) represents the W entity of element x.
If there are more pointers to compare (“Yes” branch of step 1105), then the system returns to step 1110. If there are not more pointers to compare (“No” branch of step 1105), then the system compares the SUM elements. In particular, the system selects 1145 a first SUM element, and retrieves MAX, MIN, and W entities corresponding to the first SUM element. The system selects 1150 a next SUM element, and retrieves MAX, MIN, and W entities corresponding to the next SUM element. The system uses these entities to compare a first precise range of locations with a second precise range of locations to determine whether a range of locations accessed by a processor, processor core, or thread overlaps with the ranges of locations accessed by other processors, processor cores, or threads.
The system determines miss-speculations using a process (e.g., steps 1155 and 1165) analogous to the process described previously with reference to steps 1120 and 1130. If there are no more entries for SUM (“No” branch of step 1065), then the system stops. In particular, if the system does not identify a harmful overlap (“No” branch of step 1155), the system determines if the system has compared all SUM elements. If the system has compared all SUM elements (“Yes” branch of step 1165), the system stops 1170. Otherwise (“No” branch of step 1165), the system returns to step 1150. The SUM elements are compared in a manner analogous to the comparison of the row elements described previously.
If a miss-speculation is identified, the system restores the memory content of locations speculative write instructions have accessed in a manner analogous to that described above with reference to
The example hardware and software implementations of the process for speculative throughput computing, described previously, illustrated processes to identify miss-speculations. In some implementations, miss-speculations can also be reduced in the process for speculative throughput computing.
In each iteration of the loop, a matrix element with index variables i and j (e.g., A[i,j]) is the sum of four neighbor matrix elements (e.g., A[i−1,j]+A[i+1,j]+A[i,j−1]+A[i,j+1]). If the iterations are enumerated in the order that they are executed (e.g., a first iteration is represented by state 1, and a second iteration by state 2), the resulting dependencies between iterations are derived, as illustrated in
The program segments to be executed in parallel can be formed using each of the iterations. Because there are dependencies between almost all of the consecutive iterations (e.g., 4 depends from 3, 2, and 1), uncovering parallelism can be difficult. The data dependence graph allows a system to uncover parallelism, for example, in the loop. For example, program segments with enumeration order 4, 7, 10, and 13 can be executed in parallel.
In order to derive the data dependence graph, the system augments the program with trigger points that delimit program segments. In some implementations, the trigger points increment a variable that enumerates the enumeration order of the program segment. For example, the program loop (code illustrated above) can be augmented with a trigger point (e.g., “trigger;”) as in the following example code:
Each iteration in the program loop includes a call to the trigger point. The trigger point can increment a variable that enumerates the enumeration order of the program segment. The system also augments the program with markings of read and write instructions that potentially cause dependencies. The markings identify the instructions as speculative instructions.
If the program executes sequentially, a trap is generated each time a speculative read or write instruction is executed. A software routine (e.g., a trap handler) records the address and type (e.g., read or write) of speculative instruction in a file. The enumeration order of the program segment is also recorded in the file. Post-processing of the file is used to derive 1210 the data dependence graph, and the system determines 1220 an execution order of the program segments from the data dependence graph.
The system executes 1230 the program segments according to the execution order. In particular, the system can use a programming construct to execute program segments in parallel. For example, a construct “parallel_for (i,j)=(x1,y1; x2,y2; . . . ; xn,yn)” can be interpreted so that program segments with index variables (x1,y1), (x2,y2), . . . , and (xn,yn) execute speculatively in parallel. As a further example, an example program loop that uses the construct includes the code:
The program loop includes seven “parallel_for” constructs. The system uses the construct to execute 1230 the program segments according to the execution order. For example, the second “parallel_for” construct allows the system to execute a set of program segments (2,1) and (1,2) in parallel. Alternatively, consecutive “parallel_for” constructs are executed sequentially. For example, the third “parallel_for” construct allows the system to execute program segments (3,1), (2,2) and (1,3) in parallel, after the system executes program segments (2,1) and (1,2) in parallel.
The system compares 1240 the program segment executions to identify dependencies.
Determining whether or not a program segment in a later “parallel_for” construct with a higher enumeration order causes a dependency with a program segment in an earlier “parallel_for” construct with a lower enumeration order may be difficult. In some implementations, the system compares 1240 the program segment executions to identify dependencies, after all of the program segments have been executed (e.g., at a “commit” point in the program).
Program Translation and Speculative ExecutionReturning to
The system predicts a speedup gain from speculative execution. In particular, the system determines an execution time of a sequential execution of the program (Tsequential). The system derives the speedup gain by applying an analytical model. For example, the analytical model can include execution times represented by an equation:
Texec(K)=Max[Tsegment-start(K)+Ni·CPIi+Ri·Rcost(K)+Wi·Wcost(K)+Proll-back·Roll-backcost(K)+(1−Proll-back)·Ccost(K)], where:
K identifies one of a plurality of speculative methods; Max is a maximum function; Tsegment-start models a start-up cost of initiating a speculative thread; Rcost models a cost of a speculative read instruction; Wcost models a cost of a speculative write instruction; Proll-back is a probability of a miss-speculation; Roll-backcost models a cost of a roll-back; and Ccost models a cost of a commit.
The expression inside the maximum function (Max) is an estimated execution time for program segment i. The maximum function determines an execution time of a slowest program segment for a speculative method K. Therefore, the analytical model includes execution times of a plurality of speculative methods, and the execution times of the plurality of speculative methods equal the execution times of a slowest program segment. Applying Texec(K) from the analytical model, the system can predict a speedup gain using an equation:
Speedup(K)=Tsequential/Texec(K).
The system then selects 230 one or more of a plurality of speculative methods (e.g., processes described with reference to
In some implementations, the system selects a combination of speculative methods. For example, the system can use a combination of parts of one or more speculative methods of the plurality of speculative methods. As another example, the system can use a combination of one or more speculative methods of the plurality of speculative methods executed sequentially. As yet another example, the system can use a combination of one or more speculative methods of the plurality of speculative methods executed in parallel.
In some implementations, the process 200 can be included in a module that can be integrated with a compiler. In some implementations, a combination of one or more steps of process 200 (e.g., combinations of steps 210, 220, and 230) can be included in a module. In some implementations, the modules can be integrated with compilers. For example, a first compiler can include a module that can analyze data dependence of the program (e.g., step 210). A second compiler can include a module that can perform the other steps of process 200 (e.g., steps 220 and 230). In some implementations, an apparatus (e.g., multiprocessor computing system 100, multicore computing system, or multi-threaded computing system) can perform speculative execution (e.g., according to some or all of the processes described with reference to
Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a tangible program carrier for execution by, or to control the operation of, data processing apparatus. The tangible program carrier can be a propagated signal or a computer-readable medium. The propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a computer. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them.
The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).
Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, to name just a few.
Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.
While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
Particular embodiments of the subject matter described in this specification have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.
Claims
1. A method comprising:
- translating a program to execute on a plurality of processors, processor cores, or threads including: analyzing data dependence of program segments of the program; collecting statistics that include at least one of: a number of instructions executed in a program segment, an average number of cycles used by an instruction, and a number of speculative reads and writes in a program segment; and predicting a speedup gain from speculative execution including: determining an execution time of a sequential execution of the program, and deriving the speedup gain by applying an analytical model.
2. The method of claim 1, where the analytical model includes execution times of a plurality of speculative methods, and the execution times of the plurality of speculative methods equal the execution times of a slowest program segment.
3. The method of claim 2, where translating the program to execute on a plurality of processors, processor cores, or threads further comprises:
- selecting a selected speculative method from the plurality of speculative methods using the speedup gain.
4. The method of claim 3, further comprising:
- executing the program using the selected speculative method.
5. The method of claim 2, where the execution times are represented by the equation Texec(K)=Max[Tsegment-start(K)+Ni·CPIi+Ri·Rcost(K)+Wi·Wcost(K)+Proll-back·Roll-backcost(K)+(1−Proll-back)·Ccost(K)], where:
- K identifies one of a plurality of speculative methods;
- Max is a maximum function;
- Tsegment-start models a start-up cost of initiating a speculative thread;
- Rcost models a cost of a speculative read instruction;
- Wcost models a cost of a speculative write instruction;
- Proll-back is a probability of a miss-speculation;
- Roll-backcost models a cost of a roll-back;
- Ccost models a cost of a commit;
- Ni is a number of instructions executed in program segment i;
- CPIi is an average number of cycles used by an instruction;
- Ri is a number of speculative read instructions in program segment i; and
- Wi is a number of speculative write instructions in program segment i.
6. The method of claim 5, where the speedup gain is represented by the equation Speedup(K)=Tsequential/Texec(K), and Tsequential is the execution time of the sequential execution of the program.
7. The method of claim 3, where the plurality of speculative methods comprises speculative methods that include:
- a combination of parts of one or more speculative methods of the plurality of speculative methods.
8. The method of claim 3, where the plurality of speculative methods comprises speculative methods that include:
- a combination of one or more speculative methods of the plurality of speculative methods executed sequentially.
9. The method of claim 3, where the plurality of speculative methods comprises speculative methods that include:
- a combination of one or more speculative methods of the plurality of speculative methods executed in parallel.
10. A computer program product, encoded on a computer-readable medium, operable to cause a data processing apparatus to:
- translate a program to execute on a plurality of processors, processor cores, or threads including: analyzing data dependence of program segments of the program; collecting statistics that include at least one of: a number of instructions executed in a program segment, an average number of cycles used by an instruction, and a number of speculative reads and writes in a program segment; and predicting a speedup gain from speculative execution including: determining an execution time of a sequential execution of the program, and deriving the speedup gain by applying an analytical model.
11. The computer program product of claim 10, where the analytical model includes execution times of a plurality of speculative methods, and the execution times of the plurality of speculative methods equal the execution times of a slowest program segment.
12. The computer program product of claim 11, where translating the program to execute on a plurality of processors, processor cores, or threads further comprises:
- selecting a selected speculative method from the plurality of speculative methods using the speedup gain.
13. The computer program product of claim 12, further comprising:
- executing the program using the selected speculative method.
14. A system comprising:
- one or more processors or processor cores;
- a computer-readable medium coupled to the one or more processors or processor cores and having instructions contained thereon, which, when executed by the one or more processors or processor cores, causes the one or more processors or processor cores to perform the operations of: translating a program to execute on a plurality of processors, processor cores, or threads including: analyzing data dependence of program segments of the program; collecting statistics that include at least one of: a number of instructions executed in a program segment, an average number of cycles used by an instruction, and a number of speculative reads and writes in a program segment; and predicting a speedup gain from speculative execution including: determining an execution time of a sequential execution of the program, and deriving the speedup gain by applying an analytical model.
15. The system of claim 14, where the analytical model includes execution times of a plurality of speculative methods, and the execution times of the plurality of speculative methods equal the execution times of a slowest program segment.
16. The system of claim 15, where translating the program to execute on a plurality of processors, processor cores, or threads further comprises:
- selecting a selected speculative method from the plurality of speculative methods using the speedup gain.
17. The system of claim 16, further comprising:
- executing the program using the selected speculative method.
Type: Application
Filed: Jan 30, 2008
Publication Date: Jul 31, 2008
Applicant: NEMA LABS AB (Gothenburg)
Inventors: Alexander Busck (Goteborg), Mikael Engbom (Goteborg), Per Stenstrom (Torslanda), Fredrik Warg (Boras)
Application Number: 12/022,814
International Classification: G06F 9/30 (20060101);