Abstract: Methods and systems for shuffling data to generate a dataset are described. A first map module may generate first pair data, and a second map module may generate second pair data, from source data. The first map module may insert the first pair data into a first local tuple space accessible to the first map module. The second map module may insert the second pair data into a second local tuple space accessible to the second map module. A shuffle module may request pair data that includes a particular key. The first and second pair data may be inserted into a global tuple space accessible by the first and second map modules. The shuffle module may identify the requested pair data in the global tuple space, and may fetch the identified pair data from a memory. The shuffle module may shuffle the fetched pair data to generate the dataset.

Abstract: Methods and systems for shuffling data are described. A processor may generate pair data from source data. The processor may insert the pair data into local tuple spaces. In response to a request for a particular key, the processor may determine a presence of the requested key in a global tuple space. The processor may, in response to a presence of the requested key in the global tuple space, update the global tuple space. The update may be based on the pair data among the local tuple spaces including the existing key. The processor may, in response to an absence of the requested key in the global tuple space, insert pair data including the missing key from the local tuple spaces into the global tuple space. The processor may fetch the requested pair data, and may shuffle the fetched data to generate a dataset.

Abstract: Aspects of the present invention disclose a method, computer program product, and system for scheduling an application. The method includes one or more processors receiving a task, the task includes instructions indicating desired nodes to perform the task through programs. The method further includes one or more processors identifying application characteristic information and node characteristic information associated with nodes within a data center composed of nodes. The application characteristic information includes resource utilization information for applications on nodes within the data center. The method further includes one or more processors determining that the nodes reach a threshold level of power consumption. The threshold level is a pre-set maximum amount of power utilized by a node within the data center. The method further includes one or more processors determining a node consuming an amount of power that is below a threshold level of power consumption in the data center.

Abstract: A data-centric reduction method, system, and computer program product include configuring a similarity threshold and a correlation threshold for an entire data set from at least two back-end nodes, reducing the entire data set to a reduced data set from the at least two back-end nodes sent to a front-end node by removing data based on the similarity threshold and the correlation threshold, and after the front-end receives the reduced data set, reconstructing the entire data set from the reduced data set using the similarity threshold and correlation threshold.

Abstract: Sub-components assembled into a computer are selected based on sub-component power efficiency levels (for example, low, medium, high) and/or anticipated usage of the computer. Multiple units of each type of sub-component (for example, a CPU) are tested to determine a power efficiency level of each unit. Computers in which sub-component efficiency levels are desired to match an overall computer efficiency level, receive sub-component units of corresponding efficiency level. Computers anticipated to run applications that make intensive use of a given type of sub-component receive the given units having a higher efficiency level. Computers anticipated to run applications that make little use of a given type of sub-component receive a physical unit having a lower efficiency level. Computers anticipated to run a wide variety of applications of no particular usage intensity for a given type of sub-component, receive a unit having an average efficiency level.

Abstract: A method for providing selective memory error protection responsive to a predictable failure notification associated with at least one portion of a memory in a computing system includes: obtaining an active error correcting code (ECC) configuration corresponding to the portion of the memory; determining whether the active ECC configuration is sufficient to correct at least one error in the portion of the memory affected by the predictable failure notification; when the active ECC configuration is insufficient to correct the error, determining whether data corruption can be tolerated by an application running on the computing system; when data corruption cannot be tolerated by the application, determining whether a stronger ECC level is available and, if a stronger ECC level is available, increasing a strength of the active ECC configuration; and when data corruption can be tolerated, performing page reassignment and aggregation of non-critical data.

Abstract: A method for selective duplication of subtasks in a high-performance computing system includes: monitoring a health status of one or more nodes in a high-performance computing system, where one or more subtasks of a parallel task execute on the one or more nodes; identifying one or more nodes as having a likelihood of failure which exceeds a first prescribed threshold; selectively duplicating the one or more subtasks that execute on the one or more nodes having a likelihood of failure which exceeds the first prescribed threshold; and notifying a messaging library that one or more subtasks were duplicated.

Abstract: A method for managing a network queue memory includes receiving sensor information about the network queue memory, predicting a memory failure in the network queue memory based on the sensor information, and outputting a notification through a plurality of nodes forming a network and using the network queue memory, the notification configuring communications between the nodes.

Abstract: Data processing techniques are provided to increase a computational speed of iterative computations that are performed over a domain of data points, such as stencil computations. For example, a method includes loading a set of domain data points into a cache memory; obtaining an iteration count T, and a base stencil operator having a first set of coefficients; generating a convolved stencil operator having a second set of coefficients, wherein the convolved stencil operator is generated by convolving the base stencil operator with itself at least one time; and iteratively processing the set of domain data points in the cache memory using the convolved stencil operator no more than T/2 iterations to obtain final processing results. The final processing results are similar to processing results that would be obtained by iteratively processing the set of domain data points using the base stencil operator for the iteration count T.

Abstract: Data processing techniques are provided to increase a computational speed of iterative computations that are performed over a domain of data points, such as stencil computations. For example, a method includes loading a set of domain data points into a cache memory; obtaining an iteration count T, and a base stencil operator having a first set of coefficients; generating a convolved stencil operator having a second set of coefficients, wherein the convolved stencil operator is generated by convolving the base stencil operator with itself at least one time; and iteratively processing the set of domain data points in the cache memory using the convolved stencil operator no more than T/2 iterations to obtain final processing results. The final processing results are similar to processing results that would be obtained by iteratively processing the set of domain data points using the base stencil operator for the iteration count T.

Abstract: A method for managing a network queue memory includes receiving sensor information about the network queue memory, predicting a memory failure in the network queue memory based on the sensor information, and outputting a notification through a plurality of nodes forming a network and using the network queue memory, the notification configuring communications between the nodes.

Abstract: A method, apparatus, and computer program product are provided for thermal- and spatial-aware task scheduling. The method may include monitoring a temperature for each core of a central processing unit having a plurality of cores; determining, from the monitoring, a set of hotspot cores from the plurality of cores determining temperature information and distance information for each hotspot core in the set of hotspot cores relative to each of the other cores on the central processing unit; calculating a placement metric for each core of the central processing unit based at least on the determined distance information and the determined temperature information; and scheduling a task by allocating the task to one or more cores of the central processing unit according to the placement metric.

Abstract: Responsive to a client request, a processing thread for handling the client request is assigned. Responsive to the client request, a server request is sent to a stream server configured to interact with a plurality of stream processing nodes. The processing thread is maintained in an idle state pending a write response message from the stream server. The processing thread is returned to an active state responsive to receiving the write response message including a stream processing result from the stream server. A client response including the stream processing result is sent to the client.

Abstract: A method, apparatus, and computer program product are provided for thermal- and spatial-aware task scheduling. The method may include monitoring a temperature for each core of a central processing unit having a plurality of cores; determining, from the monitoring, a set of hotspot cores from the plurality of cores determining temperature information and distance information for each hotspot core in the set of hotspot cores relative to each of the other cores on the central processing unit; calculating a placement metric for each core of the central processing unit based at least on the determined distance information and the determined temperature information; and scheduling a task by allocating the task to one or more cores of the central processing unit according to the placement metric.

Abstract: Data processing techniques are provided to increase a computational speed of iterative computations that are performed over a domain of data points, such as stencil computations. For example, a method includes loading a set of domain data points into a cache memory; obtaining an iteration count T, and a base stencil operator having a first set of coefficients; generating a convolved stencil operator having a second set of coefficients, wherein the convolved stencil operator is generated by convolving the base stencil operator with itself at least one time; and iteratively processing the set of domain data points in the cache memory using the convolved stencil operator no more than T/2 iterations to obtain final processing results. The final processing results are similar to processing results that would be obtained by iteratively processing the set of domain data points using the base stencil operator for the iteration count T.

Abstract: A method for providing notification of a predictable memory failure includes the steps of: obtaining information regarding at least one condition associated with a memory; calculating a memory failure probability as a function of the obtained information; calculating a failure probability threshold; and generating a signal when the memory failure probability exceeds the failure probability threshold, the signal being indicative of a predicted future memory failure.

Abstract: A method for providing selective memory error protection responsive to a predictable failure notification associated with at least one portion of a memory in a computing system includes: obtaining an active error correcting code (ECC) configuration corresponding to the portion of the memory; determining whether the active ECC configuration is sufficient to correct at least one error in the portion of the memory affected by the predictable failure notification; when the active ECC configuration is insufficient to correct the error, determining whether data corruption can be tolerated by an application running on the computing system; when data corruption cannot be tolerated by the application, determining whether a stronger ECC level is available and, if a stronger ECC level is available, increasing a strength of the active ECC configuration; and when data corruption can be tolerated, performing page reassignment and aggregation of non-critical data.

Abstract: A method for faulty memory utilization in a memory system includes: obtaining information regarding memory health status of at least one memory page in the memory system; determining an error tolerance of the memory page when the information regarding memory health status indicates that a failure is predicted to occur in an area of the memory system affecting the memory page; initiating a migration of data stored in the memory page when it is determined that the data stored in the memory page is non-error-tolerant; notifying at least one application regarding a predicted operating system failure and/or a predicted application failure when it is determined that data stored in the memory page is non-error-tolerant and cannot be migrated; and notifying at least one application regarding the memory failure predicted to occur when it is determined that data stored in the memory page is error-tolerant.

Abstract: A method for selective duplication of subtasks in a high-performance computing system includes: monitoring a health status of one or more nodes in a high-performance computing system, where one or more subtasks of a parallel task execute on the one or more nodes; identifying one or more nodes as having a likelihood of failure which exceeds a first prescribed threshold; selectively duplicating the one or more subtasks that execute on the one or more nodes having a likelihood of failure which exceeds the first prescribed threshold; and notifying a messaging library that one or more subtasks were duplicated.

Abstract: A method for selective duplication of subtasks in a high-performance computing system includes: monitoring a health status of one or more nodes in a high-performance computing system, where one or more subtasks of a parallel task execute on the one or more nodes; identifying one or more nodes as having a likelihood of failure which exceeds a first prescribed threshold; selectively duplicating the one or more subtasks that execute on the one or more nodes having a likelihood of failure which exceeds the first prescribed threshold; and notifying a messaging library that one or more subtasks were duplicated.