Abstract: Disclosed is a storage system comprising: receiving a first data segment and first metadata associated with the first data segment to be stored in the storage system; storing the first data segment and the first metadata in a persistent storage device of the storage system; compressing the first data segment using a predetermined compression algorithm to generate a first compressed data segment; and storing the first metadata and the first compressed data segment in a solid state drive (SSD) cache device of the storage system, including aligning the first metadata and the first compressed data segment to a page boundary of the SSD device to reduce a number of input and output (IO) operations required for accessing the first metadata and the first compressed data segment from the SSD cache device.

Abstract: A method for assigning data to similarity groups (sim-groups). The method includes: partitioning a data stream, originating from a data source, into stream chunks; for at least a first stream chunk of the stream chunks: initializing a first chunk bin-map for the first stream chunk; partitioning the first stream chunk into first chunk segments; updating the first chunk bin-map based on a chunk segment processing of the first chunk segments; computing, to obtain a first similarity distance set, a similarity distance between the first chunk bin-map and each sim-group bin-map in a first set of sim-group bin-maps; identifying a first sim-group corresponding to a first similarity distance in the first similarity distance set; and assigning the first stream chunk to the first sim-group.

Abstract: Data segments and metadata segments to be stored in a storage system, where the data segments are deduplicated segments and each of the metadata segments includes a fingerprint for the corresponding data segment, for each of the metadata segments. It is determined that the metadata segment contains one or markers inserted by a client device of the storage system. The metadata segment is examined to determine whether the metadata segment satisfies a predetermined condition. In response to determining that the metadata satisfies the predetermined condition, the metadata segment is compressed using a predetermined compression algorithm. The compressed metadata segment is stored in the storage system, otherwise the metadata segment is stored in the storage system without compression. Thereafter, the data segments are stored in the storage system.

Abstract: Disclosed is a storage system comprising: receiving a first data segment and first metadata associated with the first data segment to be stored in the storage system; storing the first data segment and the first metadata in a persistent storage device of the storage system; compressing the first data segment using a predetermined compression algorithm to generate a first compressed data segment; and storing the first metadata and the first compressed data segment in a solid state drive (SSD) cache device of the storage system, including aligning the first metadata and the first compressed data segment to a page boundary of the SSD device to reduce a number of input and output (IO) operations required for accessing the first metadata and the first compressed data segment from the SSD cache device.

Abstract: Data segments and metadata segments to be stored in a storage system, where the data segments are deduplicated segments and each of the metadata segments includes a fingerprint for the corresponding data segment, for each of the metadata segments. It is determined that the metadata segment contains one or markers inserted by a client device of the storage system. The metadata segment is examined to determine whether the metadata segment satisfies a predetermined condition. In response to determining that the metadata satisfies the predetermined condition, the metadata segment is compressed using a predetermined compression algorithm. The compressed metadata segment is stored in the storage system, otherwise the metadata segment is stored in the storage system without compression. Thereafter, the data segments are stored in the storage system.

Abstract: A method, article of manufacture, and apparatus for analyzing a distributed file system is discussed. A metadata operation is captured in an audit log, wherein the metadata operation is on an object stored in the distributed file system. An event is pushed to a distributed commit log and a real-time analytics engine in response to capturing the metadata operation. The event is persistently stored in the distributed commit log.