Abstract: Mechanisms for enabling both native and non-native input/output virtualization (IOV) in a single I/O adapter are provided. The mechanisms allow a system with a large number of logical partitions (LPARs) and system images to use IOV to share a native IOV enabled I/O adapter or endpoint that does not implement the necessary number of virtual functions (VFs) for each LPAR and system image. A number of VFs supported by the I/O adapter, less one, are assigned to LPARs and system images so that they may make use of native IOV using these VFs. The remaining VF is associated with a virtual intermediary (VI) which handles non-native IOV of the I/O adapter. Any remaining LPARs and system images share the I/O adapter using the non-native IOV via the VI. Thus, any number of LPARs and system images may share the same I/O adapter or endpoint.

Abstract: Mechanisms are provided for implementing a multi-root PCI manager (MR-PCIM) in a multi-root I/O virtualization management partition (MR-IMP) to control the shared functionality of an multi-root I/O virtualization (IOV) enabled switch fabric and multi-root IOV enabled I/O adapter (IOA) through the base functions (BF) of the switches and IOAs. A hypervisor provides device-independent facilities to the code running in the I/O Virtualization Management Partition (IMP), Multi-Root (MR)-IMP and client partitions. The MR-IMP may include device specific code without the hypervisor needing to sacrifice its size, robustness, and upgradeability. The hypervisor provides the virtual intermediary functionally for the sharing and control of the switch and IOA's control functions.

Abstract: Mechanisms for communicating with an I/O device or endpoint using a queue data structure and pre-translated addresses associated with the queue data structure are provided. With the mechanisms, a device driver invokes device driver services for initializing address translation and protection table (ATPT) entries in a root complex for the queue data structure. The device driver services return untranslated addresses to the device driver which are in turn provided to the I/O device or endpoint. The I/O device or endpoint may then request a translation of these untranslated addresses and store them in the I/O device or endpoint prior to receiving an I/O operation targeting the queue data structure. The cached translation may be used to directly access the queue data structure from the I/O device or endpoint by bypassing the root complex's address translation facilities.

Abstract: Mechanisms for communicating with a memory registration enabled adapter, such as an InfiniBand™ host channel adapter, are provided. With the mechanisms, device driver services may be invoked by a device driver for initializing address translation entries in an address translation data structure of a root complex. An address of a device driver data buffer data structure and registration modifiers may be passed by the device driver to the device driver services. The device driver services may create address translation data structure entries in the address translation data structure associated with the root complex and memory registration (MR) address translation entries in a MR address translation data structure of the adapter. The MR address translation data structure may then be used with I/O operations to bypass the address translation data structure associated with the root complex.

Abstract: A computer-implemented method, apparatus, and computer program product are disclosed for bus/device/function (BDF) translation and routing of communications packets through a fabric that utilizes PCI switches. Identifiers are included in communications packets that are routed between a host and an I/O adapter using a PCI fabric to which the host and the I/O adapter are coupled. Destination identifiers that are included in first communications packets that are received by edge switches, which are connected directly to said host or directly connected to said I/O adapter, are translated before routing the communications packets out of the edge switches. Second communications packets that are received by internal switches, which are not directly connected to the host or directly connected to the I/O adapter, are routed without translating destination identifiers that are included in the second communications packets.

Abstract: System and method for managing routing of data in a distributed computing system, such as a distributed computing system that uses PCI Express protocol to communicate over an I/O fabric. A physical tree that is indicative of a physical configuration of the distributed computing system is determined, and a virtual tree is created from the physical tree. The virtual tree is then modified to change an association between at least one source device and at least one target device in the virtual tree. A validation mechanism validates the changed association between the at least one source device and the at least one target device to enable routing of data from the at least one source device to the at least one target device.

Abstract: The system and method address the situation where an input/output (I/O) fabric is shared by more than one logical partition (LPAR) and where each LPAR can share with the other LPARs an I/O adapter (IOA). In particular, each LPAR is assigned its own separate address space to access a virtual function (VF) assigned to it such that each LPAR's perception is that it has its own independent IOA. Each VF may be shared across multiple LPARs. Facilities are provided for management of the shared resources of the IOA via a Physical Function (PF) of the IOA by assignment of that PF to an I/O Virtualization Management Partition (IMP). The code running in the IMP acts as a virtual intermediary to the VFs for fully managing the VF error handling, VF reset, and configuration operations. The IMP also acts as an interface to the PF for accessing common VF functionality.

Abstract: A PCI Express (PCIe) computer system utilizes address translation services to translate virtual addresses from I/O device adaptors to physical addresses of system memory. A combined memory controller and host bridge uses a translation agent to convert the I/O addresses via translation control entries (TCEs) in a TCE table (also known as an address translation and protection table). Some of the I/O device adaptors have address translation caches for local storage of TCEs. The TCE definition includes a new non-cacheable control bit which is set active in the TCE table when the TCE is in the process of being invalidated. The memory controller prevents further caching of the TCE while the non-cacheable control bit is active. A further implementation utilizes a change-in-progress control bit of the TCE to indicate that the TCE is in the process of being changed to allow simultaneous invalidation of the previously TCE information.

Abstract: The system and method address the situation where an input/output (I/O) fabric is shared by more than one logical partition (LPAR) and where each LPAR can share with the other LPARs an I/O adapter (IOA). In particular, each LPAR is assigned its own separate address space to access a virtual function (VF) assigned to it such that each LPAR's perception is that it has its own independent IOA. Each VF may be shared across multiple LPARs. Facilities are provided for management of the shared resources of the IOA via a Physical Function (PF) of the IOA by assignment of that PF to an I/O Virtualization Management Partition (IMP). The code running in the IMP acts as a virtual intermediary to the VFs for fully managing the VF error handling, VF reset, and configuration operations. The IMP also acts as an interface to the PF for accessing common VF functionality.

Abstract: A system and method which provide a mechanism for an I/O virtualization management partition (IMP) to control the shared functionality of an I/O virtualization (IOV) enabled I/O adapter (IOA) through a physical function (PF) of the IOA while the virtual functions (VFs) are assigned to client partitions for normal I/O operations directly. A hypervisor provides device-independent facilities to the code running in the IMP and client partitions. The IMP may include device specific code without the hypervisor needing to sacrifice its size, robustness, and upgradeability. The hypervisor provides the virtual intermediary functionally for the sharing and control of the IOA's control functions.

Abstract: A method, mechanism and computer usable medium is provided for distributing I/O fabric errors to the appropriate root nodes in a multi-root environment. The case where the I/O fabric is attached to more than one root node and where each root can potentially share with the other roots the I/O adapter (IOA) resources which are attached to the I/O is addressed. Additionally, a method, mechanism and computer usable medium is provided by which errors detected in an I/O fabric may be routed to all root nodes which may be affected by the error, while not being reported to the root nodes that will not be affected by those errors. In particular, distributed computing system which uses the PCI Express protocol to communicate over the I/O fabric is addressed.

Abstract: A system and method for registering combinations of physical and/or virtual functions for configuring an endpoint are provided. With the system and method, a mechanism informs a management component of a multifunction endpoint's functional combinations. The management component may then use this information to select the functional combinations that are to be made visible to each system image. The informing of the management component may be performed by writing values to various predefined fields in a configuration space for a physical or virtual function that provides information regarding the number of virtual functions supported, the combination of functions supported, a nominal combination of virtual functions to be used by the physical function, and a group identifier for identifying which functions are linked in a combination grouping.

Abstract: A computer-implemented method, apparatus, and computer usable program code are disclosed for migrating a virtual adapter from a source physical adapter to a destination physical adapter in a data processing system where multiple host computer systems share multiple adapters and communicate with those adapters through a PCI switched-fabric bus. The virtual adapter is first caused to stop processing transactions. All in-flight transactions that are associated with the virtual adapter are then captured. The configuration information that defines the virtual adapter is moved from the source physical adapter to the destination physical adapter. The in-flight transactions are then restored to their original locations on the destination virtual adapter. The virtual adapter is then restarted on the destination physical adapter such that the virtual adapter begins processing transactions.

Abstract: A method is directed to use of a master root node, in a distributed computer system provided with multiple root nodes, to control the configuration of routings through an I/O switched-fabric. One of the root nodes is designated as the master root node or PCI Configuration Manager (PCM), and is operable to carry out the configuration while each of the other root nodes remains in a quiescent or inactive state. In one useful embodiment pertaining to a system of the above type, that includes multiple root nodes, PCI switches, and PCI adapters available for sharing by different root nodes, a method is provided wherein the master root node is operated to configure routings through the PCI switches. Respective routings are configured between respective root nodes and the PCI adapters, wherein each of the configured routings corresponds to only one of the root nodes.

Abstract: A method for communicating with an input/output (I/O) adapter configured to communicate with a locally attached I/O device are provided using cached address translations. With the method, in response to receiving a storage transaction request, a queue element is created in a command queue specifying an untranslated buffer address. The queue element may be retrieved by the I/O adapter and a determination may be made as to whether the queue element contains a read operation command. If so, a translation request may be sent from the I/O adapter to a root complex at substantially a same time as the read operation command is sent to a locally attached external I/O device. The translated address corresponding to the untranslated address of the queue element may be returned and stored in the I/O adapter prior to receiving the data read from the external I/O device.

Abstract: A method, mechanism and computer usable medium is provided for distributing I/O fabric errors to the appropriate root nodes in a multi-root environment. The case where the I/O fabric is attached to more than one root node and where each root can potentially share with the other roots the I/O adapter (IOA) resources which are attached to the I/O is addressed. Additionally, a method, mechanism and computer usable medium is provided by which errors detected in an I/O fabric may be routed to all root nodes which may be affected by the error, while not being reported to the root nodes that will not be affected by those errors. In particular, distributed computing system which uses the PCI Express protocol to communicate over the I/O fabric is addressed.

Abstract: A mechanism, method and computer usable medium is provided for each root node of a multiple root node system and its own independent address space. This mechanism also allows multiple system images within the same root node to have their own independent address spaces. A mechanism is also provided for incorporating legacy root node and input/output adapters that are non-aware of the mechanisms introduced by this invention. Input/ output adapters which implement this invention may also have the number of functions that they support greatly expanded beyond the present eight functions per input/output adapter.

Abstract: A computer-implemented method, apparatus, and computer usable program code are disclosed for migrating a virtual adapter from a source physical adapter to a destination physical adapter in a data processing system where multiple host computer systems share multiple adapters and communicate with those adapters through a PCI switched-fabric bus. The virtual adapter is first caused to stop processing transactions. All in-flight transactions that are associated with the virtual adapter are then captured. The configuration information that defines the virtual adapter is moved from the source physical adapter to the destination physical adapter. The in-flight transactions are then restored to their original locations on the destination virtual adapter. The virtual adapter is then restarted on the destination physical adapter such that the virtual adapter begins processing transactions.

Abstract: A method and apparatus is provided for routing error messages in a distributed computer system comprising multiple root nodes, and further comprising one or more PCI switches and one or more I/O adapters, wherein each root node includes one or more system images. In one useful embodiment, a method is provided for routing I/O error messages to root nodes respectively associated with the errors contained in the messages. The method includes detecting occurrence of an error at a specified one of the adapters, wherein the error affects one of the system images, and generating an error message at the specified adapter. The method further comprises routing the error message from the specified adapter to the particular root node that includes the affected system image. The error message is then selectively processed at the particular root node, in order to identify the affected system image.

Abstract: Method and system for address routing in a distributed computing system, such as a distributed computing system that uses PCI Express protocol to communicate over an I/O fabric. A destination identifier is provided to identify a physical or virtual host or end point. When a physical or virtual host or end point receives a PCI data packet it compares a list of source identifiers with destination identifiers to determine if a source identifier included in the transaction packet is associated with a destination identifier included in the transaction packet to determine if the transaction packet has a valid association. If the transaction packet has a valid association, it is routed to the target device. The present invention enables each host that attaches to PCI bridges or switches and shares a set of common PCI devices to have its own PCI 64-bit address space and enables the routing of PCI transaction packets between multiple hosts and adapters, through a PCI switched-fabric bus using a destination identifier.