Abstract: In some examples, a computing device may communicate with a plurality of network storage systems, such as a first network storage system provided by a first service provider employing a first storage protocol and a second network storage system provided by a second service provider employing a second storage protocol different from the first storage protocol. The computing device receives a first object, and determines, for the first object, a first remote bucket at the first network storage system and a second remote bucket at the second network storage system. The computing device may add a synchronization event to a queue for replicating the first object to the first remote bucket and the second remote bucket. Based on consuming the synchronization event from the queue, the computing device replicates data and metadata of the first object to the first remote bucket and the second remote bucket.

Abstract: In some examples, a computing device of a plurality of computing devices at a first geographic location may divide data into a plurality of partitions. For example, at least two computing devices at the first geographic location may maintain a copy of the data of a first partition of the plurality of partitions and may exchange periodic heartbeat communications related to the first partition. The computing device may determine that a computing device at the second geographic location has a lower frequency of access load than the computing devices at the first location that maintain the first partition. The computing device may migrate the data of the first partition to a computing device at the second geographic location to cause at least two computing devices at the second location to maintain the data of the first partition and exchange periodic heartbeat communications related to the first partition.

Abstract: In some examples, a computing device may communicate with a plurality of network storage systems, such as a first network storage system provided by a first service provider employing a first storage protocol and a second network storage system provided by a second service provider employing a second storage protocol different from the first storage protocol. The computing device receives a first object, and determines, for the first object, a first remote bucket at the first network storage system and a second remote bucket at the second network storage system. The computing device may add a synchronization event to a queue for replicating the first object to the first remote bucket and the second remote bucket. Based on consuming the synchronization event from the queue, the computing device replicates data and metadata of the first object to the first remote bucket and the second remote bucket.

Abstract: In some examples, a client device may send, to a service computing device over a network, a request for information related to a change to a file system associated with the client device. In response, the client device may receive, from the service computing device, a list of one or more changes to the file system. The client device may add and/or update metadata for a stub file on the client device based at least partially on the received list of one or more changes. In some cases, the stub file may include a stub data structure stored on the client device, while the file content of the file corresponding to the stub file may be stored at a storage over the network.

Abstract: In some examples, a system may include at least one class of storage that is configured for having freed storage space reclaimed to enable reuse of the freed storage space. For instance, the system may determine whether a volume corresponding to the at least one class of storage is used to store system data or user data. If the volume is used to store user data, then the system may determine whether any of the user data has been deleted from the volume. If data has been deleted from the volume, the system may determine whether an available capacity of the volume is less than a remaining capacity threshold before performing reclamation on the at least one storage device corresponding to the volume. Alternatively, if the volume is used to store system data, the system may perform reclamation based on an elapsed period of time since the last reclamation.

Abstract: In some examples, a client device may send, to a service computing device over a network, a request for information related to a change to a file system associated with the client device. In response, the client device may receive, from the service computing device, a list of one or more changes to the file system. The client device may add and/or update metadata for a stub file on the client device based at least partially on the received list of one or more changes. In some cases, the stub file may include a stub data structure stored on the client device, while the file content of the file corresponding to the stub file may be stored at a storage over the network.

Abstract: In some examples, a system may include at least one class of storage that is configured for having freed storage space reclaimed to enable reuse of the freed storage space. For instance, the system may determine whether a volume corresponding to the at least one class of storage is used to store system data or user data. If the volume is used to store user data, then the system may determine whether any of the user data has been deleted from the volume. If data has been deleted from the volume, the system may determine whether an available capacity of the volume is less than a remaining capacity threshold before performing reclamation on the at least one storage device corresponding to the volume. Alternatively, if the volume is used to store system data, the system may perform reclamation based on an elapsed period of time since the last reclamation.

Abstract: A storage system has a plurality of nodes which are grouped into a plurality of cluster systems each having multiple nodes, each cluster system being logically partitioned into a plurality of namespaces, each namespace including a collection of data objects, each cluster system having multiple tenants, each tenant being a grouping of namespaces, each cluster system having a plurality of capabilities, at least some of the capabilities being bound to the tenants. A node in the cluster system comprises: a memory, and a controller operable to bind each capability to one of a plurality of IP networks so that each capability is bound to only one of the IP networks and has a destination IP address of the IP network to which the capability is bound. It is permissible for one or more capabilities to be bound to the same IP network. Each IP network has one corresponding network interface.

Abstract: Apparatus and method for placing data based on the content of the data in random access memory such that indexing operations are not required. A strong (e.g., cryptographic) hash is applied to a data element resulting in a signature. A weaker hash function is then applied to the signature to generate a storage location in memory for the data element. The weaker hash function assigns multiple data elements to the same storage location while the signature comprises a unique identifier for locating a particular data element at this location. In one embodiment a plurality of weak hash functions are applied successively to increase storage space utilization. In other embodiments, the assigned storage location can be determined by one or more attributes of the data element and/or the storage technology, e.g, long-lived versus short-lived data and/or different regions of the memory having different performance (e.g., access latency memory lifetime) characteristics.

Abstract: Method and apparatus for storing records in non-uniform access memory. In various embodiments, the placement of records is localized in one or more regions of the memory. This can be accomplished utilizing different ordered lists of hash functions to preferentially map records to different regions of the memory to achieve one or more performance characteristics or to account for differences in the underlying memory technologies. For example, one ordered list of hash functions may localize the data for more rapid access. Another list of hash functions may localize the data that is expected to have a relatively short lifetime. Localizing such data may significantly improve the erasure performance and/or memory lifetime, e.g., by concentrating the obsolete data elements in one location. Thus, the two or more lists of ordered hash functions may improve one or more of access latency, memory lifetime, and/or operation rate.

Abstract: Methods and systems are provided for tracking object instances stored on a plurality of network nodes, which tracking enables a global determination of when an object has no references across the networked nodes and can be safely de-allocated. According to one aspect of the invention, each node has a local object store for tracking and optionally storing objects on the node, and the local object stores collectively share the locally stored instances of the objects across the network. One or more applications, e.g., a file system and/or a storage system, use the local object stores for storing all persistent data of the application as objects.

Abstract: Method and apparatus for storing records in non-uniform access memory. In various embodiments, the placement of records is localized in one or more regions of the memory. This can be accomplished utilizing different ordered lists of hash functions to preferentially map records to different regions of the memory to achieve one or more performance characteristics or to account for differences in the underlying memory technologies. For example, one ordered list of hash functions may localize the data for more rapid access. Another list of hash functions may localize the data that is expected to have a relatively short lifetime. Localizing such data may significantly improve the erasure performance and/or memory lifetime, e.g., by concentrating the obsolete data elements in one location. Thus, the two or more lists of ordered hash functions may improve one or more of access latency, memory lifetime, and/or operation rate.

Abstract: Methods and systems are provided for tracking object instances stored on a plurality of network nodes, which tracking enables a global determination of when an object has no references across the networked nodes and can be safely de-allocated. According to one aspect of the invention, each node has a local object store for tracking and optionally storing objects on the node, and the local object stores collectively share the locally stored instances of the objects across the network. One or more applications, e.g., a file system and/or a storage system, use the local object stores for storing all persistent data of the application as objects.

Abstract: Method and apparatus for storing records in non-uniform access memory. In various embodiments, the placement of records is localized in one or more regions of the memory. This can be accomplished utilizing different ordered lists of hash functions to preferentially map records to different regions of the memory to achieve one or more performance characteristics or to account for differences in the underlying memory technologies. For example, one ordered list of hash functions may localize the data for more rapid access. Another list of hash functions may localize the data that is expected to have a relatively short lifetime. Localizing such data may significantly improve the erasure performance and/or memory lifetime, e.g., by concentrating the obsolete data elements in one location. Thus, the two or more lists of ordered hash functions may improve one or more of access latency, memory lifetime, and/or operation rate.

Abstract: A storage system has a plurality of nodes which are grouped into a plurality of cluster systems each having multiple nodes, each cluster system being logically partitioned into a plurality of namespaces, each namespace including a collection of data objects, each cluster system having multiple tenants, each tenant being a grouping of namespaces, each cluster system having a plurality of capabilities, at least some of the capabilities being bound to the tenants. A node in the cluster system comprises: a memory, and a controller operable to bind each capability to one of a plurality of IP networks so that each capability is bound to only one of the IP networks and has a destination IP address of the IP network to which the capability is bound. It is permissible for one or more capabilities to be bound to the same IP network. Each IP network has one corresponding network interface.

Abstract: Apparatus and method for placing data based on the content of the data in random access memory such that indexing operations are not required. A strong (e.g., cryptographic) hash is applied to a data element resulting in a signature. A weaker hash function is then applied to the signature to generate a storage location in memory for the data element. The weaker hash function assigns multiple data elements to the same storage location while the signature comprises a unique identifier for locating a particular data element at this location. In one embodiment a plurality of weak hash functions are applied successively to increase storage space utilization. In other embodiments, the assigned storage location can be determined by one or more attributes of the data element and/or the storage technology, e.g, long-lived versus short-lived data and/or different regions of the memory having different performance (e.g., access latency memory lifetime) characteristics.

Abstract: Methods and systems are provided for tracking object instances stored on a plurality of network nodes, which tracking enables a global determination of when an object has no references across the networked nodes and can be safely de-allocated. According to one aspect of the invention, each node has a local object store for tracking and optionally storing objects on the node, and the local object stores collectively share the locally stored instances of the objects across the network. One or more applications, e.g., a file system and/or a storage system, use the local object stores for storing all persistent data of the application as objects.

Abstract: Apparatus and method for placing data based on the content of the data in random access memory such that indexing operations are not required. A strong (e.g., cryptographic) hash is applied to a data element resulting in a signature. A weaker hash function is then applied to the signature to generate a storage location in memory for the data element. The weaker hash function assigns multiple data elements to the same storage location while the signature comprises a unique identifier for locating a particular data element at this location. In one embodiment a plurality of weak hash functions are applied successively to increase storage space utilization. In other embodiments, the assigned storage location can be determined by one or more attributes of the data element and/or the storage technology, e.g, long-lived versus short-lived data and/or different regions of the memory having different performance (e.g., access latency memory lifetime) characteristics.

Abstract: Methods and systems are provided for tracking object instances stored on a plurality of network nodes, which tracking enables a global determination of when an object has no references across the networked nodes and can be safely de-allocated. According to one aspect of the invention, each node has a local object store for tracking and optionally storing objects on the node, and the local object stores collectively share the locally stored instances of the objects across the network. One or more applications, e.g., a file system and/or a storage system, use the local object stores for storing all persistent data of the application as objects.

Abstract: Methods and systems are provided for tracking object instances stored on a plurality of network nodes, which tracking enables a global determination of when an object has no references across the networked nodes and can be safely de-allocated. According to one aspect of the invention, each node has a local object store for tracking and optionally storing objects on the node, and the local object stores collectively share the locally stored instances of the objects across the network. One or more applications, e.g., a file system and/or a storage system, use the local object stores for storing all persistent data of the application as objects.