Abstract: A method and apparatus is provided for routing error messages in a distributed computer system comprising multiple root nodes, and further comprising one or more PCI switches and one or more I/O adapters, wherein each root node includes one or more system images. In one useful embodiment, a method is provided for routing I/O error messages to root nodes respectively associated with the errors contained in the messages. The method includes detecting occurrence of an error at a specified one of the adapters, wherein the error affects one of the system images, and generating an error message at the specified adapter. The method further comprises routing the error message from the specified adapter to the particular root node that includes the affected system image. The error message is then selectively processed at the particular root node, in order to identify the affected system image.

Abstract: Method and system for address routing in a distributed computing system, such as a distributed computing system that uses PCI Express protocol to communicate over an I/O fabric. A destination identifier is provided to identify a physical or virtual host or end point. When a physical or virtual host or end point receives a PCI data packet it compares a list of source identifiers with destination identifiers to determine if a source identifier included in the transaction packet is associated with a destination identifier included in the transaction packet to determine if the transaction packet has a valid association. If the transaction packet has a valid association, it is routed to the target device. The present invention enables each host that attaches to PCI bridges or switches and shares a set of common PCI devices to have its own PCI 64-bit address space and enables the routing of PCI transaction packets between multiple hosts and adapters, through a PCI switched-fabric bus using a destination identifier.

Abstract: A method is provided for creating and managing tables for routing packets through an environment that includes multiple hosts and shared PCI switches and adapters. A Destination Identification (DID) field in the PBA is appended to a transaction packet dispatched through the PCI switches, wherein a particular DID is associated with a particular host or system image, and thus identifies the physical or virtual end point of its packet. In one embodiment, packets are routed through PCI switches in a distributed computer system comprising multiple root nodes, wherein each root node includes one or more hosts. The embodiment includes the step of creating a table or like data structure in a specified one of the switches. When a particular host of one of the root nodes becomes connected to the specified switch, a PCI Configuration Master (PCM), residing in one of the root nodes, is operated to enter a destination identifier or DID into the table.

Abstract: A computer-implemented method, apparatus, and computer program product are disclosed in a data processing environment that includes host computer systems that are coupled to adapters utilizing a switched fabric for routing packets between the host computer systems and the adapters. A unique destination identifier is assigned to one of the host computer systems. A portion of a standard format packet destination address is selected. Within a particular packet, the portion is set equal to the unique identifier that is assigned to the host computer system. The particular packet is then routed through the fabric to the host computer system using the unique destination identifier.

Abstract: A method is directed to use of a master root node, in a distributed computer system provided with multiple root nodes, to control the configuration of routings through an I/O switched-fabric. One of the root nodes is designated as the master root node or PCI Configuration Manager (PCM), and is operable to carry out the configuration while each of the other root nodes remains in a quiescent or inactive state. In one useful embodiment pertaining to a system of the above type, that includes multiple root nodes, PCI switches, and PCI adapters available for sharing by different root nodes, a method is provided wherein the master root node is operated to configure routings through the PCI switches. Respective routings are configured between respective root nodes and the PCI adapters, wherein each of the configured routings corresponds to only one of the root nodes.

Abstract: An apparatus and method for communicating with a network adapter using a queue data structure are provided. With the apparatus and method, a device driver invokes device driver services for initializing address translation and protection table (ATPT) entries in a root complex for the queue data structure. The device driver services return untranslated addresses to the device driver which are in turn provided to the network adapter. In response to retrieving a queue element from the queue data structure, the network adapter may request a translation of an untranslated address specified in the queue element and store the translated address in the network adapter prior to receiving a data packet targeting a buffer associated with the queue element.

Abstract: A method is directed to use of a master root node, in a distributed computer system provided with multiple root nodes, to control the configuration of routings through an I/O switched-fabric. One of the root nodes is designated as the master root node or PCI Configuration Manager (PCM), and is operable to carry out the configuration while each of the other root nodes remains in a quiescent or inactive state. In one useful embodiment pertaining to a system of the above type, that includes multiple root nodes, PCI switches, and PCI adapters available for sharing by different root nodes, a method is provided wherein the master root node is operated to configure routings through the PCI switches. Respective routings are configured between respective root nodes and the PCI adapters, wherein each of the configured routings corresponds to only one of the root nodes.

Abstract: A system and method for migration of a virtual endpoint from one virtual plane to another are provided. With the system and method, when a management application requests migration of a virtual endpoint (VE) from one virtual plane (VP) to another, a fabric manager provides an input/output virtualization intermediary (IOVI) with an interrupt to perform a stateless migration. The IOVI quiesces outstanding requests to the virtual functions (VFs) of the VE, causes a function level reset of the VFs, deconfigures addresses in intermediary switches corresponding to the VP, and informs the fabric manager that a destination migration is requested. The fabric manager sends an interrupt to the destination IOVI which performs a function level reset of the destination VFs and reprograms the intermediary switches with the addresses of the destination VP. The destination VFs may then be placed in an active state.

Abstract: A system and method for hot-plug/remove of a new component in a running communication fabric, such as a PCIe fabric, are provided. With the system and method, the addition of a new component in the fabric is detected and an event is sent to a multiple root fabric configuration manager. The multiple root fabric configuration manager gathers information about the new component and updates its I/O component tree structure in its configuration data structure to include the new component. The new component may then be utilized via the updated configuration data structure. When a component is to be removed, the multiple root fabric configuration manager receives an event indicating the component to be removed, determines which branches of the tree structure are affected by the removal, and updates its configuration data structure accordingly to remove the component and its associated components from the virtual plane of the removed component.

Abstract: A system and method for initializing shared memories for sharing endpoints across a plurality of root complexes is provided. In one illustrative embodiment, a multi-root PCIe manager (MR-PCIM) initializes the shared memory between root complexes and endpoints by discovering the PCIe switch fabric by traversing all the links accessible through the interconnected switches of the PCIe switch fabric. As the links are traversed, the MR-PCIM compares information obtained for each of the root complexes and endpoints to determine which endpoints and root complexes reside on the same blade. A virtual PCIe tree data structure is then generated that ties the endpoints available on the PCIe switch fabric to each root complex. The MR-PCIM, or a single-root PCIe manager (SR-PCIM), may then assign each endpoint and root complex a base and limit within the PCIe memory address space the endpoint belongs to.

Abstract: A system and method for migration of stateless virtual functions from one virtual plane to another are provided. When a migration of a source virtual function to a destination virtual function in another virtual plane is to be performed, a source single root PCI manager (SR-PCIM) is first interrupted by a multiple root PCI manager (MR-PCIM). Configuration information that defines the source virtual function is then redefined on the destination virtual function for this stateless migration. A function level reset may then be performed on the source virtual function. The destination SR-PCIM may be interrupted by the MR-PCIM with an interrupt for the destination virtual function. A function level reset may then be performed on the destination virtual function. The destination virtual function state may then be changed to an “active” state such that the migrated virtual function begins processing transactions.

Abstract: An apparatus and method for communicating with an I/O device or endpoint using a queue data structure and pre-translated addresses associated with the queue data structure are provided. With the apparatus and method, a device driver invokes device driver services for initializing address translation and protection table (ATPT) entries in a root complex for the queue data structure. The device driver services return untranslated addresses to the device driver which are in turn provided to the I/O device or endpoint. The I/O device or endpoint may then request a translation of these untranslated addresses and store them in the I/O device or endpoint prior to receiving an I/O operation targeting the queue data structure. The cached translation may be used to directly access the queue data structure from the I/O device or endpoint by bypassing the root complex's address translation facilities.

Abstract: A system and method for communication between host systems using a transaction protocol and shared memories are provided. Shared memories are initialized based on a discovery process in a communication fabric such that at least one endpoint has address ranges in shared memories of at least two host systems. A transaction oriented protocol may be established for using the shared memories of the host systems to communicate between root complexes and endpoints of the same or different host systems. The transaction oriented protocol specifies a series of transactions to be performed by the various elements, e.g., root complex or endpoint, to push or pull data. Various combinations of push and pull transactions may be utilized.

Abstract: A system and method for communication between host systems using a queuing system and shared memories are provided. Memory address spaces of the host systems are initialized such that endpoints may be accessible by root complexes across host systems. These memory address spaces may then be used to allow system images, and their corresponding applications, associated with these root complexes to communicate with the endpoints using a queuing system. Such a queuing system may comprise queue structures having doorbell structures for providing information about the queue entries in the queue structures. Queue elements may be generated and added to the queue structures, and the doorbell structure may be written to, in order to thereby inform an endpoint or root complex that queue elements are available for processing. DMA operations may be performed to retrieve the queue elements and the data corresponding to the queue elements.

Abstract: A system and method for registering combinations of physical and/or virtual functions for configuring an endpoint are provided. With the system and method, a mechanism informs a management component of a multifunction endpoint's functional combinations. The management component may then use this information to select the functional combinations that are to be made visible to each system image. The informing of the management component may be performed by writing values to various predefined fields in a configuration space for a physical or virtual function that provides information regarding the number of virtual functions supported, the combination of functions supported, a nominal combination of virtual functions to be used by the physical function, and a group identifier for identifying which functions are linked in a combination grouping.

Abstract: A system and method for communication between host systems using a socket connection and shared memories are provided. With such socket-based communication, a work queue in the host systems may be used to listen for incoming socket initialization requests. A first host system that wishes to establish a socket communication connection with a second host system may generate a socket initialization request work queue element in its work queue and may inform the second host system that the socket initialization request work queue element is available for processing. The second host system may then accept or deny the request. If the second host system accepts the request, it returns the second half of the socket's parameters for use by the first host system in performing socket based communications between the first and second host systems.

Abstract: A system and method for migration of single root stateless virtual functions are provided. A Single-Root PCI Configuration Manager (SR-PCIM) provides a system image (SI) with possible virtual function (VF) migration scenarios supported by the endpoint (EP). The SR-PCIM may be instructed that a stateless migration of a VF and its associated application(s) from one SI to another is required. Outstanding requests to the VF are completed and any applications associated with the VF are removed from the SI and the VF is detached from its associated physical function (PF). The SWI may then attach the VF to a target PF which may be in the same or a different EP. The SWI makes the VF available to the SI with which the VF is now associated and the SI configures the VF thereby making it available for use by associated applications.

Abstract: A computer-implemented method, apparatus, and computer program product are disclosed for bus/device/function (BDF) translation and routing of communications packets through a fabric that utilizes PCI switches. Identifiers are included in communications packets that are routed between a host and an I/O adapter using a PCI fabric to which the host and the I/O adapter are coupled. Destination identifiers that are included in first communications packets that are received by edge switches, which are connected directly to said host or directly connected to said I/O adapter, are translated before routing the communications packets out of the edge switches. Second communications packets that are received by internal switches, which are not directly connected to the host or directly connected to the I/O adapter, are routed without translating destination identifiers that are included in the second communications packets.

Abstract: A computer-implemented method, apparatus, and computer program product are disclosed for bus/device/function (BDF) translation and routing of communication packets through a fabric that utilizes a single multi-root PCI switch. A data processing environment includes multiple host computer systems that are coupled to and share I/O adapters using a PCI switched-fabric bus fabric. The processing environment includes an apparatus that receives a communications packet, from one of the host computer systems, that is intended to be delivered to a particular one of the adapters. The apparatus analyzes the packet to determine a non-unique host-assigned destination device functional identifier that is included in the packet. The apparatus translates the host-assigned destination device functional identifier into a unique virtual destination device functional identifier.

Abstract: System and method for managing routing of data in a distributed computing system, such as a distributed computing system that uses PCI Express protocol to communicate over an I/O fabric. A physical tree that is indicative of a physical configuration of the distributed computing system is determined, and a virtual tree is created from the physical tree. The virtual tree is then modified to change an association between at least one source device and at least one target device in the virtual tree. A validation mechanism validates the changed association between the at least one source device and the at least one target device to enable routing of data from the at least one source device to the at least one target device.