Abstract: The present disclosure provides an advantageous systems and related methods for scalable object storage. One embodiment relates to a system that includes at least a manifest tracking subsystem and a chunk payload subsystem. The manifest tracking subsystem stores version manifests and encodes forward-references from the version manifests to chunks. The chunk payload subsystem stores the chunks and encodes back-references to version manifests that reference the chunk. Other embodiments, aspects and features are also disclosed.

Abstract: The present disclosure provides advantageous methods and systems for input/output processing workflows in a heterogeneous data volume. One embodiment relates to a method of writing data to a heterogeneous data volume having multiple disk classes of storage. A class of storage tier for the data write is selected using operating modes for the tiers, where the operating mode for a tier instance depends on statistical measures of operating parameters for that tier. One operating mode is an elastic mode, where the chance that a tier instance is selected may vary depending on a statistical measure of an operating parameter in relation to lower and upper threshold values. Other embodiments, aspects and features are also disclosed.

Abstract: Embodiments disclosed herein provide advantageous methods and systems that use multicast communications via unreliable datagrams sent on a protected traffic class. These methods and systems provide effectively reliable multicast delivery while avoiding the overhead associated with point-to-point protocols. Rather than an exponential scaling of point-to-point connections (with expensive setup and teardown of the connections), the traffic from one server is bounded by linear scaling of multicast groups. In addition, the multicast rendezvous disclosed herein creates an edge-managed flow control that accounts for the dynamic state of the storage servers in the cluster, without needing centralized control, management or maintenance of state. This traffic shaping avoids the loss of data due to congestion during sustained oversubscription. Other embodiments, aspects and features are also disclosed.

Abstract: Embodiments disclosed herein provide advantageous methods and systems that use multicast communications via unreliable datagrams sent on a protected traffic class. These methods and systems provide effectively reliable multicast delivery while avoiding the overhead associated with point-to-point protocols. Rather than an exponential scaling of point-to-point connections (with expensive setup and teardown of the connections), the traffic from one server is bounded by linear scaling of multicast groups. In addition, the multicast rendezvous disclosed herein creates an edge-managed flow control that accounts for the dynamic state of the storage servers in the cluster, without needing centralized control, management or maintenance of state. This traffic shaping avoids the loss of data due to congestion during sustained oversubscription. Other embodiments, aspects and features are also disclosed.

Abstract: Embodiments disclosed herein provide advantageous methods and systems that use multicast communications via unreliable datagrams sent on a protected traffic class. These methods and systems provide effectively reliable multicast delivery while avoiding the overhead associated with point-to-point protocols. Rather than an exponential scaling of point-to-point connections (with expensive setup and teardown of the connections), the traffic from one server is bounded by linear scaling of multicast groups. In addition, the multicast rendezvous disclosed herein creates an edge-managed flow control that accounts for the dynamic state of the storage servers in the cluster, without needing centralized control, management or maintenance of state. This traffic shaping avoids the loss of data due to congestion during sustained oversubscription. Other embodiments, aspects and features are also disclosed.

Abstract: Embodiments disclosed herein provide advantageous methods and systems that use multicast communications via unreliable datagrams sent on a protected traffic class. These methods and systems provide effectively reliable multicast delivery while avoiding the overhead associated with point-to-point protocols. Rather than an exponential scaling of point-to-point connections (with expensive setup and teardown of the connections), the traffic from one server is bounded by linear scaling of multicast groups. In addition, the multicast rendezvous disclosed herein creates an edge-managed flow control that accounts for the dynamic state of the storage servers in the cluster, without needing centralized control, management or maintenance of state. This traffic shaping avoids the loss of data due to congestion during sustained oversubscription. Other embodiments, aspects and features are also disclosed.

Abstract: The present disclosure provides an advantageous systems and related methods for scalable object storage. One embodiment relates to a system that includes at least a manifest tracking subsystem and a chunk payload subsystem. The manifest tracking subsystem stores version manifests and encodes forward-references from the version manifests to chunks. The chunk payload subsystem stores the chunks and encodes back-references to version manifests that reference the chunk. Other embodiments, aspects and features are also disclosed.

Abstract: The present disclosure relates to an advantageous system and related methods for distributed deduplication of encrypted chunks. One embodiment relates to a method for storing encrypted chunks in which an encryption key is generated independently from a chunk payload. With this method, two encrypted chunks are identifiable as having identical chunk payloads even when the chunk payloads are encrypted with different encryption keys. Other embodiments, aspects and features are also disclosed.

Abstract: A method and system is disclosed for providing a cloud storage system supporting existing APIs and protocols. The method of storing cloud storage system (CSS) object metadata separates object metadata that describes each CSS object as a collection of named chunks with chunk locations specified as a separate part of the metadata. Chunks are identified using globally unique permanent identifiers that are never re-used to identify different chunk payload. While avoiding the bottleneck of a single metadata server, the disclosed system provides ordering guarantees to clients such as guaranteeing access to the most recent version of an object. The disclosed system also provides end-to-end data integrity protection, inline data deduplication, configurable replication, hierarchical storage management and location-aware optimization of chunk storage.

Abstract: The present disclosure provides an advantageous systems and related methods for scalable object storage. One embodiment relates to a system that includes at least a manifest tracking subsystem and a chunk payload subsystem. The manifest tracking subsystem stores version manifests and encodes forward-references from the version manifests to chunks. The chunk payload subsystem stores the chunks and encodes back-references to version manifests that reference the chunk. Other embodiments, aspects and features are also disclosed.

Abstract: A method and system for providing unified local storage support for file and cloud access is disclosed. The method comprises writing a chunk on a storage server, and replicating the chunk to other selected storage servers when necessary. The method and system further comprise writing a version manifest on the storage server; replicating the version manifest to other selected storage servers when necessary. Object puts or appends are implemented by first chunking the object, determining if the chunks are new, transferring the chunks if required, followed by creation of a new version manifest referencing the chunks. Finally, the method and system include providing concurrent file-oriented read and write access consistent with the stored version manifests and chunks.

Abstract: A method and system is disclosed for providing a cloud storage system supporting existing APIs and protocols. The method of storing cloud storage system (CSS) object metadata separates object metadata that describes each CSS object as a collection of named chunks with chunk locations specified as a separate part of the metadata. Chunks are identified using globally unique permanent identifiers that are never re-used to identify different chunk payload. While avoiding the bottleneck of a single metadata server, the disclosed system provides ordering guarantees to clients such as guaranteeing access to the most recent version of an object. The disclosed system also provides end-to-end data integrity protection, inline data deduplication, configurable replication, hierarchical storage management and location-aware optimization of chunk storage.

Abstract: An Upper Layer Protocol (ULP) offload engine system, method and associated data structure are provided for performing protocol offloads without requiring a Transmission Control Protocol (TCP) offload engine (TOE). In an embodiment, the ULP offload engine provides Internet Small Computer System Interface (iSCSI) offload services.

Abstract: A method and system is disclosed for providing storage services to Virtual Machines (VMs) and the Hypervisor Platforms that host them. NAS referrals are used to control selection of NAS proxy servers that are optimized for the support of VMs and the Hypervisor Platforms accessing directories that are exclusively used in conjunction of a single VM. Referrals are dynamically controlled in a manner specific to the target environment to provide an optimal combination of local and network storage in support of VMs. A method of using named attributes allows referrals to subsets of a directory. The overhead of using a NAS proxy is tailored to the smallest set of accesses required to gain beneficial result from the use of local storage, and using direct access to shared network storage when local storage is not allocated. Most of the new suggested techniques are transparent to the existing NAS clients; some techniques may require modification of the NAS client software.

Abstract: A method and system is disclosed for providing a heterogeneous data storage comprising a plurality of storage devices of different types with respect to device vendor, model, capacity, performance and/or function. The present invention employs data access mechanisms specific to the type of underlying storage and the type of data to be stored or retrieved, and provides for integrated remote mirroring, disaster recovery and hierarchical storage management (HSM), as well as improved I/O performance and life expectancy of storage disks. A method of writing to and reading from heterogeneous data volume is also disclosed.

Abstract: A method and system is disclosed for resolving a single server bottleneck. Logically associated data is typically collocated within a single filesystem or a single block device accessible via a single storage server. A single storage server can provide a limited I/O bandwidth, which creates a problem known as “single I/O node” bottleneck. The method and system provides techniques for spreading I/O workload over multiple I/O domains, both local and remote, while at the same time increasing operational mobility and data redundancy. Both file and block level I/O access are addressed.

Abstract: A method and system is disclosed for providing fault tolerant data storage with built-in backup capabilities, improved I/O performance, and improved utilization of storage devices.

Abstract: A ULP offload engine system, method and associated data structure are provided for performing protocol offloads without requiring a TCP offload engine (TOE). In an embodiment, the ULP offload engine provides iSCSI offload services.