Abstract: In remote direct memory access transfers in a multinode data processing system in which the nodes communicate with one another through communication adapters coupled to a switch or network, failures in the nodes or in the communication adapters can produce the phenomenon known as trickle traffic, which is data that has been received from the switch or from the network that is stale but which may have all the signatures of a valid packet data. The present invention addresses the trickle traffic problem in two situations: node failure and adapter failure. In the node failure situation randomly generated keys are used to reestablish connections to the adapter while providing a mechanism for the recognition of stale packets. In the adapter failure situation, a round robin context allocation approach is used with adapter state contexts being provided with state information which helps to identify stale packets.

Abstract: In a multinode data processing system in which nodes exchange information over a network or through a switch, the mechanism which enables out-of-order data transfer via Remote Direct Memory Access (RDMA) also provides a corresponding ability to carry out broadcast operations, multicast operations, third party operations and conditional RDMA operations. In a broadcast operation a source node transfers data packets in RDMA fashion to a plurality of destination nodes. Multicast operation works similarly except that distribution is selective. In third party operations a single central node in a cluster or network manages the transfer of data in RDMA fashion between other nodes or creates a mechanism for allowing a directed distribution of data between nodes. In conditional operation mode the transfer of data is conditioned upon one or more events occurring in either the source node or in the destination node.

Abstract: An addressing model is provided where devices, including I/O devices, are addressed with internet protocol (IP) addresses, which are considered part of the virtual address space. A task, such as an application, may be assigned an effective address range, which corresponds to addresses in the virtual address space. The virtual address space is expanded to include Internet protocol addresses. Thus, the page frame tables are also modified to include entries for IP addresses and additional properties for devices and I/O. Thus, a processing element, such as an I/O adapter or even a printer, for example, may also be addressed using IP addresses without the need for library calls, device drivers, pinning memory, and so forth. This addressing model also provides full virtualization of resources across an IP interconnect, allowing a process to access an I/O device across a network.

Abstract: An addressing model is provided where all resources, including memory and devices, are addressed with internet protocol (IP) addresses. A task, such as an application, may be assigned a range of IP addresses rather than an effective address range. Thus, a processing element, such as an I/O adapter or even a printer, for example, may also be addressed using IP addresses without the need for library calls, device drivers, pinning memory, and so forth. This addressing model also provides full virtualization of resources across an IP interconnect, allowing a process to access an I/O device across a network.

Abstract: Disclosed are a method, a system and a computer program product for dynamically allocating and/or de-allocating resources and/or partitions that provide I/O and/or active storage access services in a supercomputing system. The supercomputing system can include multiple compute nodes, high performance computing (HPC) switches coupled to the compute nodes, and active non-volatile storage devices coupled to the compute nodes. Each of the compute nodes can be configured to communicate with another compute node through at least one of the HPC switches. In one or more embodiments, each of at least two compute nodes includes a storage controller and is configured to dynamically allocate and de-allocate a storage controller partition to provide storage services to the supercomputing system, and each of at least two compute nodes includes an I/O controller and is configured to dynamically allocate and de-allocate an I/O controller partition to provide I/O services to the supercomputing system.

Abstract: Disclosed are a method, a system and a computer program product for automatically allocating and de-allocating resources for jobs executed or processed by one or more supercomputer systems. In one or more embodiments, a supercomputing system can process multiple jobs with respective supercomputing resources. A global resource manager can automatically allocate additional resources to a first job and de-allocate resources from a second job. In one or more embodiments, the global resource manager can provide the de-allocated resources to the first job as additional supercomputing resources. In one or more embodiments, the first job can use the additional supercomputing resources to perform data analysis at a higher resolution, and the additional resources can compensate for an amount of time the higher resolution analysis would take using originally allocated supercomputing resources.

Abstract: Mechanisms are provided for processing streaming data at high sustained data rates. These mechanisms receive a plurality of data elements over a plurality of non-sequential communication channels and write the plurality of data elements directly to the file system of the data processing system in an unassembled manner. The mechanisms determining whether to perform a data scrubbing operation or not based on history information indicative of whether data elements in the plurality of data elements are being received in a substantially sequential manner. The mechanisms perform a data scrubbing operation, in response to a determination to perform data scrubbing, to identify any missing data elements in the plurality of data elements written to the file system and assemble the plurality of data elements into a plurality of data streams in response to results of the data scrubbing indicating that there are no missing data elements.

Abstract: Mechanisms for performing a backend operation in a file system are provided. A backend operation on a portion of the file system is initiated. At least one indirect transition table data structure is created for performing the backend operation. Metadata corresponding to the portion of the file system is linked to the at least one indirect transition table data structure. The backend operation is performed on data in a sub-portion of the portion of the file system and the at least one indirect transition table data structure is updated with pointers to new locations of the data in the sub-portion as transitions of the data are completed. At least one data access operation is performed to the portion of the file system at substantially a same time as performing the backend operation on the data in the sub-portion of the portion of the file system.

Abstract: In a multinode data processing system in which nodes exchange information over a network or through a switch, a structure and mechanism is provided within the realm of Remote Direct Memory Access (RDMA) operations in which DMA operations are present on one side of the transfer but not the other. On the side in which the transfer is not carried out in DMA fashion, transfer processing is carried out under program control; this is in contrast to the transfer on the DMA side which is characteristically carried out in hardware. Usage of these combination processes is useful in programming situations where RDMA is carried out to or from contiguous locations in memory on one side and where memory locations on the other side is noncontiguous. This split mode of transfer is provided both for read and for write operations.

Abstract: A synchronization optimized queuing method and device to minimize software/hardware interaction in network interface hardware during an end-of-initiative process, including network adapter queue implementations for network interface hardware for optimized communication in a computer system. An end-of-initiative procedure to ensure that the network interface hardware has received an interrupt enable and to recheck the interrupt queue is unnecessary in the present invention.

Abstract: A system and method to provide injection of important data directly into a processor's cache location when that processor has previously indicated interest in the data. The memory subsystem at a target processor will determine if the memory address of data to be written to a memory location associated with the target processor is found in a processor cache of the target processor. If it is determined that the memory address is found in a target processor's cache, the data will be directly written to that cache at the same time that the data is being provided to a location in main memory.

Abstract: A mechanism is provided for managing an application communication request. A first operating system passes a call from a first application in a first data processing system intended for a second application in a second data processing system to a first host fabric interface controller in the first data processing system without processing the call. The first host fabric interface processes the call using state information associated with the call to determine the second data processing system with which the call is associated. The first host fabric interface initiates a connection to a second host fabric interface in the second data processing system and transfers the call to a second operating system in the second data processing system via the connection to the second host fabric interface. The second data processing system then processes the call intended for the second application without assistance from the second application.

Abstract: Mechanisms are provided for processing streaming data at high sustained data rates. These mechanisms receive a plurality of data elements over a plurality of non-sequential communication channels and write the plurality of data elements directly to the file system of the data processing system in an unassembled manner. The mechanisms further perform a data scrubbing operation to determine if there are any missing data elements that are not present in the plurality of data elements written to the file system and assemble the plurality of data elements into a plurality of data streams associated with the plurality of non-sequential communication channels in response to results of the data scrubbing indicating that there are no missing data elements. In addition, the mechanisms release the assembled plurality of data streams for access via the file system.

Abstract: A mechanism is provided for managing a process-to-process inter-cluster communication request. A call from a first application is received in a first operating system in a first data processing system. The first operating system passes the call from the first operating system to a first host fabric interface controller in the first data processing system without processing the call. The first host fabric interface processes the call to determine a second data processing system in the plurality of data processing systems with which the call is associated, wherein the call is processed by the first host fabric interface without intervention by the first operating system. The first host fabric interface initiates an inter-cluster connection to a second host fabric interface in the second data processing. The call is then transferred to the second host fabric interface in the second data processing system via the inter-cluster connection.

Abstract: A mechanism is provided for managing an input/output device communication request. A first operating system passes a call from a first application intended for an input/output device in a second data processing system to a first host fabric interface controller in the first data processing system without processing the call. The first host fabric interface processes the call to determine the second data processing system with which the call is associated. The first host fabric interface initiates a connection to a second host fabric interface in the second data processing system and transfers the call to a second operating system associated with the input/output device in the second data processing system via the connection to the second host fabric interface. The second operating system then processes the call intended for the input/output device without assistance from any application running on the second data processing system.

Abstract: A mechanism is provided for managing a process-to-process intra-cluster communication request. A call from a first application is received in a first operating system in a first data processing system. The first operating system passes the call from the first operating system to a first host fabric interface controller in the first data processing system without processing the call. The first host fabric interface controller processes the call without intervention by the first operating system to determine a second data processing system in the plurality of data processing systems with which the call is associated. The first host fabric interface controller initiates an intra-cluster connection to a second host fabric interface controller in the second data processing system. The first host fabric interface controller then transfers the call to the second host fabric interface controller in the second data processing system via the intra-cluster connection.

Abstract: An addressing model is provided where all resources, including memory and devices, are addressed with internet protocol (IP) addresses. A task, such as an application, may be assigned a range of IP addresses rather than an effective address range. Thus, a processing element, such as an I/O adapter or even a printer, for example, may also be addressed using IP addresses without the need for library calls, device drivers, pinning memory, and so forth. This addressing model also provides full virtualization of resources across an IP interconnect, allowing a process to access an I/O device across a network.

Abstract: An addressing model is provided where devices, including I/O devices, are addressed with internet protocol (IP) addresses, which are considered part of the virtual address space. A task, such as an application, may be assigned an effective address range, which corresponds to addresses in the virtual address space. The virtual address space is expanded to include Internet protocol addresses. Thus, the page frame tables are also modified to include entries for IP addresses and additional properties for devices and I/O. Thus, a processing element, such as an I/O adapter or even a printer, for example, may also be addressed using IP addresses without the need for library calls, device drivers, pinning memory, and so forth. This addressing model also provides full virtualization of resources across an IP interconnect, allowing a process to access an I/O device across a network.

Abstract: In a multinode data processing system in which nodes exchange information over a network or through a switch, the mechanism which enables out-of-order data transfer via Remote Direct Memory Access (RDMA) also provides a corresponding ability to carry out broadcast operations, multicast operations, third party operations and conditional RDMA operations. In a broadcast operation a source node transfers data packets in RDMA fashion to a plurality of destination nodes. Multicast operation works similarly except that distribution is selective. In third party operations a single central node in a cluster or network manages the transfer of data in RDMA fashion between other nodes or creates a mechanism for allowing a directed distribution of data between nodes. In conditional operation mode the transfer of data is conditioned upon one or more events occurring in either the source node or in the destination node.

Abstract: In a multinode data processing system in which the nodes communicate with one another through communication adapters coupled to a switch or network, a method is provided for using the Internet Protocol (IP) for transmitting large broadcast data packets without incurring the overhead normally associated with packet fragmentation. By adding an Internet Protocol (IP) header as the first header in every packet fragment in the fragmentation process, fragmented packets are able to be assembled in the IP layer without intervention at the adapter interface layer.