Abstract: A system and method for notifying a process about a creation or removal event of a named data element (NDE) in a coordination namespace distributed memory system. A controller runs methods to: generate a tuple corresponding to data generated by a requesting process, the tuple having a tuple name and data value; and generate a notification indicator in a pending notification list to indicate to one or more processes a notification of the creation or removal event associated with the corresponding tuple. Upon detecting the event performed on the tuple by a second process, the method further searches for NDEs in the distributed memory system having the same tuple name, and in response to determining an existence of an associated pending notification record in a pending notification list of records, notify each corresponding process of the one or more processes indicated in the list of the creation or removal event.

Abstract: A system and method for federating a tuple storage database across multiple coordinated namespace (CNS) extended memory storage systems allowing the sharing of tuples and tuple data across independent systems. The method provides a federation service for multiple coordination namespace systems. The method retrieves a tuple from connected independent CNS systems wherein a local CNS Controller sends a read request to the local gatekeeper to retrieve a first tuple and creates a local pending remote record. The local gatekeeper at a requesting node sends a broadcast query to a plurality of remote gatekeepers for the tuple and Remote gatekeepers at remote nodes query in its local CNS for the tuple. The Local gatekeeper process at the requesting node receives results from a plurality of remote gatekeepers for the said tuple and selects one remote gatekeeper to receive the requested tuple and broadcasts a read for tuple data with selected gatekeeper.

Abstract: A system and method of supporting point-to-point synchronization among processes/nodes implementing different hardware barriers in a tuple space/coordinated namespace (CNS) extended memory storage architecture. The system-wide CNS provides an efficient means for storing data, communications, and coordination within applications and workflows implementing barriers in a multi-tier, multi-nodal tree hierarchy. The system provides a hardware accelerated mechanism to support barriers between the participating processes. Also architected is a tree structure for a barrier processing method where processes are mapped to nodes of a tree, e.g., a tree of degree k, to provide an efficient way of scaling the number of processes in a tuple space/coordination namespace.

Abstract: A system and method for supporting tuple record interlinking in one or more tuple space/coordinated namespace (CNS) extended memory storage systems. A system-wide CNS provides for efficient storing and communicating of data generated by local processes running at the nodes, and coordinated to generate a union/intersection of multiple CNS where tuple records are interlinked in multiple CNS hashtables, and/or share tuple data between two sets of processes that are part of different CNSs. Local node processes further generate multi-key tuples where two or more tuple records are interlinked within the same CNS hash table, thereby permitting a look up of the tuple data by either tuple name/keys. A CNS controller further provides a tuple iterator for a key-value storage in a CNS system that adds more links between tuples enables creation of iterator structures such as linked list or trees etc. of “different” tuples in a tuple database.

Abstract: A system and method for supporting data MapReduce operations in a tuple space/coordinated namespace (CNS) extended memory storage architecture. The system-wide CNS provides an efficient means for storing and communicating data generated by local processes running at the nodes, and coordinated to provide MapReduce operations in a multi-nodal system. A hardware accelerated mechanism supports map reduce sorting/shuffle operations and reduce operations according to an aggregate function. Local processes running at a node generate a tuple corresponding to data generated by a process, each tuple having a tuple name and tuple data value corresponding to the generated data. Each tuple is processed and stored at the node or another node, dependent upon its tuple name. Tuple records associated with a tuple name are accumulated at one or more nodes according to a linked list structure at each that is accessible via a hash table index pointer at the node.

Abstract: A system and method of supporting point-to-point synchronization among processes/nodes implementing different hardware barriers in a tuple space/coordinated namespace (CNS) extended memory storage architecture. The system-wide CNS provides an efficient means for storing data, communications, and coordination within applications and workflows implementing barriers in a multi-tier, multi-nodal tree hierarchy. The system provides a hardware accelerated mechanism to support barriers between the participating processes. Also architected is a tree structure for a barrier processing method where processes are mapped to nodes of a tree, e.g., a tree of degree k, to provide an efficient way of scaling the number of processes in a tuple space/coordination namespace.

Abstract: Scatter gather operation(s) are performed by accessing a shared memory that is shared amongst nodes interconnected through network(s) and having a CNS shared amongst the nodes. Data is gathered from multiple processes at corresponding multiple nodes into location(s) in the CNS, and tuple(s) having a same tuple name are created in the CNS. The tuple(s) have information referencing the gathered data in the location(s). Alternatively, data that has been gathered using the same tuple name is scattered to multiple processes participating in the CNS. The scattering uses the tuple(s) in the CNS, and is performed from the location(s) into other location(s) at one or multiple nodes for one or multiple processes at the corresponding one or multiple nodes. Both the gathering data and the scattering data may also be performed.

Abstract: A system and method for federating a tuple storage database across multiple coordinated namespace (CNS) extended memory storage systems allowing the sharing of tuples and tuple data across independent systems. The method provides a federation service for multiple coordination namespace systems. The method retrieves a tuple from connected independent CNS systems wherein a local CNS Controller sends a read request to the local gatekeeper to retrieve a first tuple and creates a local pending remote record. The local gatekeeper at a requesting node sends a broadcast query to a plurality of remote gatekeepers for the tuple and Remote gatekeepers at remote nodes query in its local CNS for the tuple. The Local gatekeeper process at the requesting node receives results from a plurality of remote gatekeepers for the said tuple and selects one remote gatekeeper to receive the requested tuple and broadcasts a read for tuple data with selected gatekeeper.

Abstract: A system and method for notifying a process about a creation or removal event of a named data element (NDE) in a coordination namespace distributed memory system. A controller runs methods to: generate a tuple corresponding to data generated by a requesting process, the tuple having a tuple name and data value; and generate a notification indicator in a pending notification list to indicate to one or more processes a notification of the creation or removal event associated with the corresponding tuple. Upon detecting the event performed on the tuple by a second process, the method further searches for NDEs in the distributed memory system having the same tuple name, and in response to determining an existence of an associated pending notification record in a pending notification list of records, notify each corresponding process of the one or more processes indicated in the list of the creation or removal event.

Abstract: A system and method for supporting tuple record interlinking in one or more tuple space/coordinated namespace (CNS) extended memory storage systems. A system-wide CNS provides for efficient storing and communicating of data generated by local processes running at the nodes, and coordinated to generate a union/intersection of multiple CNS where tuple records are interlinked in multiple CNS hashtables, and/or share tuple data between two sets of processes that are part of different CNSs. Local node processes further generate multi-key tuples where two or more tuple records are interlinked within the same CNS hash table, thereby permitting a look up of the tuple data by either tuple name/keys. A CNS controller further provides a tuple iterator for a key-value storage in a CNS system that adds more links between tuples enables creation of iterator structures such as linked list or trees etc. of “different” tuples in a tuple database.

Abstract: A system and method of supporting point-to-point synchronization among processes/nodes implementing different hardware barriers in a tuple space/coordinated namespace (CNS) extended memory storage architecture. The system-wide CNS provides an efficient means for storing data, communications, and coordination within applications and workflows implementing barriers in a multi-tier, multi-nodal tree hierarchy. The system provides a hardware accelerated mechanism to support barriers between the participating processes. Also architected is a tree structure for a barrier processing method where processes are mapped to nodes of a tree, e.g., a tree of degree k to provide an efficient way of scaling the number of processes in a tuple space/coordination namespace.

Abstract: Methods and systems for shuffling data are described. A processor may generate pair data from source data. The processor may insert the pair data into local tuple spaces. In response to a request for a particular key, the processor may determine a presence of the requested key in a global tuple space. The processor may, in response to a presence of the requested key in the global tuple space, update the global tuple space. The update may be based on the pair data among the local tuple spaces including the existing key. The processor may, in response to an absence of the requested key in the global tuple space, insert pair data including the missing key from the local tuple spaces into the global tuple space. The processor may fetch the requested pair data, and may shuffle the fetched data to generate a dataset.

Abstract: Methods and systems for shuffling data to generate a dataset are described. A first map module may generate first pair data, and a second map module may generate second pair data, from source data. The first map module may insert the first pair data into a first local tuple space accessible to the first map module. The second map module may insert the second pair data into a second local tuple space accessible to the second map module. A shuffle module may request pair data that includes a particular key. The first and second pair data may be inserted into a global tuple space accessible by the first and second map modules. The shuffle module may identify the requested pair data in the global tuple space, and may fetch the identified pair data from a memory. The shuffle module may shuffle the fetched pair data to generate the dataset.

Abstract: A system and method for supporting data MapReduce operations in a tuple space/coordinated namespace (CNS) extended memory storage architecture. The system-wide CNS provides an efficient means for storing and communicating data generated by local processes running at the nodes, and coordinated to provide MapReduce operations in a multi-nodal system. A hardware accelerated mechanism supports map reduce sorting/shuffle operations and reduce operations according to an aggregate function. Local processes running at a node generate a tuple corresponding to data generated by a process, each tuple having a tuple name and tuple data value corresponding to the generated data. Each tuple is processed and stored at the node or another node, dependent upon its tuple name. Tuple records associated with a tuple name are accumulated at one or more nodes according to a linked list structure at each that is accessible via a hash table index pointer at the node.

Abstract: A system and method of supporting point-to-point synchronization among processes/nodes implementing different hardware barriers in a tuple space/coordinated namespace (CNS) extended memory storage architecture. The system-wide CNS provides an efficient means for storing data, communications, and coordination within applications and workflows implementing barriers in a multi-tier, multi-nodal tree hierarchy. The system provides a hardware accelerated mechanism to support barriers between the participating processes. Also architected is a tree structure for a barrier processing method where processes are mapped to nodes of a tree, e.g., a tree of degree k to provide an efficient way of scaling the number of processes in a tuple space/coordination namespace.

Abstract: Scatter gather operation(s) are performed by accessing a shared memory that is shared amongst nodes interconnected through network(s) and having a CNS shared amongst the nodes. Data is gathered from multiple processes at corresponding multiple nodes into location(s) in the CNS, and tuple(s) having a same tuple name are created in the CNS. The tuple(s) have information referencing the gathered data in the location(s). Alternatively, data that has been gathered using the same tuple name is scattered to multiple processes participating in the CNS. The scattering uses the tuple(s) in the CNS, and is performed from the location(s) into other location(s) at one or multiple nodes for one or multiple processes at the corresponding one or multiple nodes. Both the gathering data and the scattering data may also be performed.

Abstract: Methods and systems for shuffling data are described. A processor may generate pair data from source data. The processor may insert the pair data into local tuple spaces. In response to a request for a particular key, the processor may determine a presence of the requested key in a global tuple space. The processor may, in response to a presence of the requested key in the global tuple space, update the global tuple space. The update may be based on the pair data among the local tuple spaces including the existing key. The processor may, in response to an absence of the requested key in the global tuple space, insert pair data including the missing key from the local tuple spaces into the global tuple space. The processor may fetch the requested pair data, and may shuffle the fetched data to generate a dataset.

Abstract: Methods and systems for shuffling data to generate a dataset are described. A first map module may generate first pair data, and a second map module may generate second pair data, from source data. The first map module may insert the first pair data into a first local tuple space accessible to the first map module. The second map module may insert the second pair data into a second local tuple space accessible to the second map module. A shuffle module may request pair data that includes a particular key. The first and second pair data may be inserted into a global tuple space accessible by the first and second map modules. The shuffle module may identify the requested pair data in the global tuple space, and may fetch the identified pair data from a memory. The shuffle module may shuffle the fetched pair data to generate the dataset.

Abstract: The present invention provides a mechanism for a processor to write data to a cache or other fast memory, without also writing it to main memory. Further, the data is “locked” into the cache or other fast memory until it is loaded for use. Data remains in the locking cache until it is specifically overwritten under software control. The locking cache or other fast memory can be used as additional system memory. In an embodiment of the invention, the locking cache is one or more sets of ways, but not all of the sets or ways, of a multiple set associative cache.

Abstract: The present invention provides a mechanism of storing data transferred from an I/O device, a network, or a disk into a portion of a cache or other fast memory, without also writing it to main memory. Further, the data is “locked” into the cache or other fast memory until it is loaded for use. Data remains in the locking cache until it is specifically overwritten under software control. In an embodiment of the invention, a processor can write data to the cache or other fast memory without also writing it to main memory. The portion of the cache or other fast memory can be used as additional system memory.