Abstract: A mechanism for allowing a single physical IB node to virtualize a plurality of host channel adapters is provided. This includes providing the appearance of both a router and multiple virtual HCA's residing behind that router, to the external REAL subnet components. Each virtual host channel adapter will have unique access control levels. One or more InfiniBand subnets are virtualized in such a way that nodes residing both within the virtual subnets and in separate physical subnets are completely unaware of the virtualization. This virtualization of InfiniBand subnets significantly increases the horizontal scaling capabilities of a single InfiniBand physical component, while at the same time provides “native” network throughput for all the virtual hosts.

Abstract: A method in a network computing system for managing a plurality of subnet managers in the network computing system. An identification is received of a set of subnet managers within the plurality of subnet managers. The set of subnet managers is allowed to participate in a master election to select a master subnet manager. Subnet managers other than the set of subnet managers are placed in a dormant state. The master subnet manager is elected from the set of subnet managers through the master election, wherein other subnet managers within the number of subnet managers poll the master subnet manager to allow the other subnet managers to elect a new master subnet manager if the master subnet manager fails.

Abstract: A method and system for a queue identification (QI) system which significantly reduces resources required to identify a process associated with receiving an incoming packet is provided. A QI mechanism is used as a TCP option. During a TCP connection establishment process, QI numbers are exchanged for both a sending port and a receiving port of the connection. After the connection is established, the destination QI's number is inserted in a Transmission Control Protocol (TCP) header on outbound packets.

Abstract: A method and apparatus to perform network fabric management is provided. The method and apparatus provide a mechanism by which modifications to components of the network fabric may be made without tearing down existing connections. The apparatus and method facilitate such fabric management by placing send queues in a send queue drain state and suspending the send queues affected by changes to the network fabric while the modifications are being made. Once the modifications are complete, the send queues are place back into an operational state.

Abstract: An apparatus and method for distributed intersystem lock optimization are provided. With the apparatus and method of the present invention, the time required to obtain an uncontested lock, meaning a lock that no other program or process in the distributed system is simultaneously attempting to acquire, is minimized. The apparatus and method of the present invention increases the speed with which locks are acquired by splitting the process of obtaining a lock into two separate operations: a test for contention, and then if contention exists, a full lock operation. The test for contention is made fast by associating each lock with a memory location, and using an atomic operation or the like, to atomically set the memory location associated with the lock to a different value. If the lock is found to be contested, meaning that another program or process has already locked it, control is turned over to a slower operation than ensures that the lock-requesting program or process will eventually be granted the lock.

Abstract: A method in a node within network computing system for selecting a master network manager, wherein the first node is associated with a first priority. Requests are sent to the network computing system to discover other nodes within the network computing system. A second priority from the request is identified in response to receiving a response to one of the requests from another node within the network computing system. The first node shifts to a standby mode if it discovers a master subnet manager or the second priority is higher than the first priority. The first node shifts to a master mode if a response containing a priority higher than the first priority is absent in responses received by the first node and the first node has completed checking all other nodes in the network computing system.

Abstract: A method and apparatus for accessing a memory. Access rights for a memory operation are verified using a first data structure in response to receiving a request to perform the operation, wherein the request includes a virtual address for the operation. Responsive to access rights being verified for the memory operation, the virtual address translated into a real address using a second data structure.

Abstract: A method, apparatus, and computer implemented instructions for transferring data. A request is sent by a requester to a responder. The request includes an amount of available processing space at the requestor. When the request is received from the responder, data is identified using the request. The data is placed into a plurality of subsequences of data packets for transfer to the requester, wherein each packet within the set of subsequences hold data in amount less than or equal to the amount of available space. These subsequences are then sent to the requestor one subsequence at a time. A new subsequence is sent each time the available processing space at the requester becomes free to process data from another subsequence. The requestor receives a subsequence from the plurality of subsequences in response to the request each time the amount of available processing space is free, wherein data within each of the set of subsequences fits within the amount of available processing space.

Abstract: A method for enabling a Q_key that is tamper proof from applications on a distributed computer system to protect selected network operations is provided. Applications and an operating system (OS) execute on the end nodes and each may access various network resources. In the invention, the network resources are configured for selective access by particular applications or OS. In a preferred embodiment, a control bit of a Q_key, which allows applications to authenticate their use of particular communication resources, i.e., the send and receive queues, is reserved and utilized to signal whether a particular application is allowed access to the resources. Setting the control bit to 0 allows the Q_key to be set by an application directly. When the control bit is set to 1, the Q_key cannot be set by an application and can only be set using a privileged operation performed only by the OS.

Abstract: A method, apparatus, and computer implemented instructions for managing a plurality of caches of data, wherein the data processing system includes a plurality of independent computers. In response to initiating a read operation to read data on a data block, an indication is posted on a directory of data blocks identifying the computer that now holds a copy of that block and a location in the memory of that computer where a flag associated with that block is held. Then in response to initiating a write operation on that data block, messages are sent to all the computers holding that block which resets the said flag, thus informing each computer that the data in that block is no longer valid. These messages are sent using means that perform that flag reset without, in the preferred embodiment, any overhead of interruption of processing on the computers where the flags reside.

Abstract: An open format storage subsystem and method are provided. The storage subsystem and method include at least one host endnode, at least one processing unit endnode, and at least one storage endnode. These endnodes are partitioned according to partition tables assigned to the ports of the endnodes and partition keys assigned to queue pairs of the ports. Based on these partition keys, partitions in the storage subsystem are designated. In this way, certain endnodes may be designated as being able to communicate with only certain other ones of the endnodes. Because of the partitioning mechanism of the present invention, an open format storage subsystem is formulated such that the types of endnodes in the storage subsystem are not limited to vendor specific units. This enhances the ability to add and remove units from the storage subsystem by removing the limitations typically found in closed storage subsystems.

Abstract: A distributed computing system having (host and I/O) end nodes, switches, routers, and links interconnecting these components is provided. The end nodes use send and receive queue pairs to transmit and receive messages. The end nodes use completion queues to inform the end user when a message has been completely sent or received and whether an error occurred during the message transmission or reception process. A mechanism implements these queue pairs and completion queues in hardware. A mechanism for controlling the transfer of work requests from the consumer to the CA hardware and work completions from the CA hardware to the consumer using head and tail pointers that reference circular buffers is also provided. The QPs and CQs do not contain Work Queue Entries and Completion Queue Entries respectively, but instead contain references to these entries.

Abstract: A method and apparatus for pausing a send queue while preventing sympathy error from propagating through a SAN fabric system is provided. The method and apparatus place a send work queue in an error state, i.e. pauses the send work queue, when a reliable data error occurs in the send work queue but does not place any other work queues in an error state. In this way, the send queue experiencing the error is not able to send any further messages until error recovery is performed. However, other work queues continue to be able to send and/or receive messages. Once error recovery is performed, the send work queue that was placed in the error state is returned to a working state and is able to continue to send messages. In addition, the send queue that was in the error state will send the messages that it attempted to send at the time of the error. The messages sent will continue from a last known point at which the send work queue was operating properly.

Abstract: A method for managing allocation of network resources within the distributed computer system is provided. Specifically, the network traversal time and the end node response time for requests and/or packets being routed in a switch-connected system area network are utilized to determine the total round trip time for completion of the particular network operation. The sum of the timeout values for all switches that participate in routing the request from a requester (source) to the receptor node (target) is provided to the requester's channel adapter (CA). The time-out values are provided by the switch manufacturer and are sent to a network Subnet Manager (SM) via SM packets (SMP). The timeout values added together represent the SubnetTimeout. The time-out value of the target channel adapter (CA), the ResponseTime, is also provided to the requester. The requester then utilizes one of two timeout equations to calculate the overall response time required for the request to be completed.

Abstract: An apparatus and method for managing work and completion queues using head and tail circular pointers. With the apparatus and method, queue head and tail pointers are maintained in the channel interface and the host channel adapter. The head and tail pointers in the host channel adapter include a queue pointer table index and a queue page index for identifying a position within the queue. For work queues, the tail pointer in the channel interface is used to identify a next position where a work queue entry may be written. The head pointer in the channel interface is used only to determine whether the work queue is full or not. The head pointer in the host channel adapter is used to identify a next work queue entry for processing by the host channel adapter. The tail pointer in the host channel adapter is used by the host channel adapter to determine if the queue is empty. For completion queues, the head pointer in the channel interface is used to identify a next completion queue entry to be processed.

Abstract: An open format storage subsystem and method are provided. The storage subsystem and method include at least one host endnode, at least one processing unit endnode, and at least one storage endnode. These endnodes are partitioned according to partition tables assigned to the ports of the endnodes and partition keys assigned to queue pairs of the ports. Based on these partition keys, partitions in the storage subsystem are designated. In this way, certain endnodes may be designated as being able to communicate with only certain other ones of the endnodes. Because of the partitioning mechanism of the present invention, an open format storage subsystem is formulated such that the types of endnodes in the storage subsystem are not limited to vendor specific units. This enhances the ability to add and remove units from the storage subsystem by removing the limitations typically found in closed storage subsystems.

Abstract: An apparatus and method for managing reliable datagram work queues, and associated completion queues, using head and tail pointers with end-to-end context error cache are provided. With the apparatus and method, reliable datagram (RD) queue head and tail pointers are maintained in the channel interface and the host channel adapter. The head and tail pointers in the host channel adapter include a RD queue page table index and a RD queue page index for identifying a position within the RD queue. For RD work queues, the tail pointer in the channel interface is used to identify a next position where a work queue entry may be written. The head pointer in the channel interface is used only to determine whether the work queue is full or not. The head pointer in the host channel adapter is used to identify a next work queue entry for processing by the host channel adapter. The tail pointer in the host channel adapter is used by the host channel adapter to determine if the queue is empty.

Abstract: A distributed computing system is provided having (host and I/O) end nodes, switches, routers, and links interconnecting these components. The end nodes use send and receive queue pairs to transmit and receive messages. The end nodes use completion queues to inform the end user when a message has been completely sent or received and whether an error occurred during the message transmission or reception process. A mechanism may implement these queue pairs and completion queues in hardware. A mechanism controls the transfer of work requests from the consumer to the channel adapter hardware using only head pointers in the hardware is described, along with a mechanism for passing work completions from the channel adapter hardware to the consumer using only tail pointers in the hardware. With this scheme the channel adapter hardware can inform the CI that a work request has been completed and provide the work completion information with just a single write to system memory.

Abstract: A distributed computing system having (host and I/O) end nodes, switches, routers, and links interconnecting these components is provided. The end nodes use send and receive queue pairs to transmit and receive messages. The end nodes use completion queues to inform the end user when a message has been completely sent or received and whether an error occurred during the message transmission or reception process. A mechanism implements these queue pairs and completion queues in hardware. A mechanism for controlling the transfer of work requests from the consumer to the CA hardware and work completions from the CA hardware to the consumer using head and tail pointers that reference circular buffers is also provided. The QPs and CQs do not contain Work Queue Entries and Completion Queue Entries respectively, but instead contain references to these entries.

Abstract: A mechanism for allowing a single physical IB node to virtualize a plurality of host channel adapters is provided. This includes providing the appearance of both a router and multiple virtual HCA's residing behind that router, to the external REAL subnet components. Each virtual host channel adapter will have unique access control levels. One or more InfiniBand subnets are virtualized in such a way that nodes residing both within the virtual subnets and in separate physical subnets are completely unaware of the virtualization. This virtualization of InfiniBand subnets significantly increases the horizontal scaling capabilities of a single InfiniBand physical component, while at the same time provides “native” network throughput for all the virtual hosts.