Abstract: Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.

Abstract: A method, non-transitory computer readable medium, and device that assists with reducing memory fragmentation in solid state devices includes identifying an allocation area within an address range to write data from a cache. Next, the identified allocation area is determined for including previously stored data. The previously stored data is read from the identified allocation area when it is determined that the identified allocation area comprises previously stored data. Next, both the write data from the cache and the read previously stored data are written back into the identified allocation area sequentially through the address range.

Abstract: Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.

Abstract: A method, non-transitory computer readable medium, and device that assists with reducing memory fragmentation in solid state devices includes identifying an allocation area within an address range to write data from a cache. Next, the identified allocation area is determined for including previously stored data. The previously stored data is read from the identified allocation area when it is determined that the identified allocation area comprises previously stored data. Next, both the write data from the cache and the read previously stored data are written back into the identified allocation area sequentially through the address range.

Abstract: Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.

Abstract: Techniques are provided for selectively storing data into allocation areas using streams. A set of allocation areas (e.g., ranges of block numbers such as virtual block numbers) are defined for a storage device. Data having particular characteristics (e.g., user data, metadata, hot data, cold data, randomly accessed data, sequentially accessed data, etc.) will be sent to the storage device for selective storage in corresponding allocation areas. For example, when a file system receives a write stream of hot data, the hot data may be assigned to a stream. The stream will be tagged using a stream identifier that is used as an indicator to the storage device to process data of the stream using an allocation area defined for hot data. In this way, data having different characteristics will be stored/confined within particular allocation areas of the storage device to reduce fragmentation and write amplification.

Abstract: A method, non-transitory computer readable medium, and device that assists with reducing memory fragmentation in solid state devices includes identifying an allocation area within an address range to write data from a cache. Next, the identified allocation area is determined for including previously stored data. The previously stored data is read from the identified allocation area when it is determined that the identified allocation area comprises previously stored data. Next, both the write data from the cache and the read previously stored data are written back into the identified allocation area sequentially through the address range.

Abstract: Conventional storage filers utilize a data reading process that requires client read request messages to be suspended in the operating system while the data is retrieved from the physical storage system. Then, once the operating system retrieves the data from the physical storage system, the operating system must restart the suspended read message in order to forward the retrieved data to the client. Accordingly, the inventors have developed a system and method that allows the physical storage system's server to send the data directly to the client rather than routing back through the operating system.

Abstract: Conventional storage filers utilize a data reading process that requires client read request messages to be suspended in the operating system while the data is retrieved from the physical storage system. Then, once the operating system retrieves the data from the physical storage system, the operating system must restart the suspended read message in order to forward the retrieved data to the client. Accordingly, the inventors have developed a system and method that allows the physical storage system's server to send the data directly to the client rather than routing back through the operating system.

Abstract: Methods and systems for a storage server are provided. One method includes storing data at a first storage tier by a processor executable storage operating system; tracking the data stored at the first storage tier for moving the data to a second storage tier; transferring the data from the first storage tier to the second storage tier; and updating a data structure that tracks a transfer block number of a block that stores the data, where the transfer block number is based on a virtual identifier, a generation count and an offset value of a storage chunk that is a portion of a physical volume block number address space.

Abstract: Methods and systems for a storage server are provided. One method includes storing data at a first storage tier by a processor executable storage operating system; tracking the data stored at the first storage tier for moving the data to a second storage tier; transferring the data from the first storage tier to the second storage tier; and updating a data structure that tracks a transfer block number of a block that stores the data, where the transfer block number is based on a virtual identifier, a generation count and an offset value of a storage chunk that is a portion of a physical volume block number address space.

Abstract: A system and method for determining an optimal cache size of a computing system is provided. In some embodiments, the method comprises selecting a portion of an address space of a memory structure of the computing system. A workload of data transactions is monitored to identify a transaction of the workload directed to the portion of the address space. An effect of the transaction on a cache of the computing system is determined, and, based on the determined effect of the transaction, an optimal cache size satisfying a performance target is determined. In one such embodiment the determining of the effect of the transaction on a cache of the computing system includes determining whether the effect would include a cache hit for a first cache size and determining whether the effect would include a cache hit for a second cache size different from the first cache size.

Abstract: A system and method for determining an optimal cache size of a computing system is provided. In some embodiments, the method comprises selecting a portion of an address space of a memory structure of the computing system. A workload of data transactions is monitored to identify a transaction of the workload directed to the portion of the address space. An effect of the transaction on a cache of the computing system is determined, and, based on the determined effect of the transaction, an optimal cache size satisfying a performance target is determined. In one such embodiment the determining of the effect of the transaction on a cache of the computing system includes determining whether the effect would include a cache hit for a first cache size and determining whether the effect would include a cache hit for a second cache size different from the first cache size.

Abstract: Among other things, one or more techniques and/or systems are provided for storing data within a hybrid storage aggregate comprising a solid state storage tier and a non-solid state storage tier. In particular, frequently accessed data and/or randomly accessed data may be identified and stored within the solid state storage tier to mitigate mechanical latency that may arise from accessing such data from the non-solid state storage tier. Infrequently accessed data and/or sequentially accessed data may be identified and stored within the non-solid state storage tier to utilize cost effective storage space while mitigating undesirable mechanical latency associated with accessing such data.

Abstract: Methods and apparatuses for performing deduplication in a hybrid storage aggregate are provided. In one example, a method includes operating a hybrid storage aggregate that includes a plurality of tiers of different types of physical storage media. The method includes identifying a first storage block and a second storage block of the hybrid storage aggregate that contain identical data and identifying caching statuses of the first storage block and the second storage block. The method also includes deduplicating the first storage block and the second storage block based on the caching statuses of the first storage block and the second storage block.

Abstract: Methods and apparatuses for operating a hybrid storage aggregate are provided. In one example, such a method includes operating a first tier of physical storage of the hybrid storage aggregate as a cache for a second tier of physical storage of the hybrid storage aggregate. The first tier of physical storage includes a plurality of assigned blocks. The method also includes updating metadata of the assigned blocks in response to an event associated with at least one of the assigned blocks. The metadata includes block usage information tracking more than two possible usage states per assigned block. The method can further include processing the metadata to determine a caching characteristic of the assigned blocks.

Abstract: The techniques introduced here provide a storage server migration that is non-disruptive to client applications. The techniques described here allow the migration of data from a source storage server to a destination storage server without the need for planned downtime typically associated with storage server migration. Using the techniques described here, data migration operations are performed while client applications continue to run and access data from the source storage server. Once data migration operations are complete, all applications and clients are transitioned to the destination storage server without disruption to the client applications.

Abstract: Techniques are provided for securely delivering media data. A requestor is directed to a secure streaming portal after being authenticated. A requesting application contacts the secure streaming portal and requests media data. A configuration header having a key for decrypting the media data is sent to the requesting application using secure communications. The media data is dynamically encrypted using the key and streamed to the requesting application. The requesting application uses the key and decrypts the encrypted media data for consumption.

Abstract: Data-storage volumes, such as a file system on a storage system, may utilize a particular operating system format. Converting a data-storage volume to a different format may be desirable at times. However, this can be time consuming and resource intensive if data needs to be copied. Accordingly, substantially copying-free, in-place conversion of a data-storage volume from a first format to a second format is provided herein. In one embodiment, a data-storage volume may be converted from a first format to a second format while the data-storage volume is mounted and/or without copying data within the data-storage volume to a separate data-storage volume. Before conversion, features associated with the first format and the second format may be compared to detect incompatibilities that may result from conversion. During conversion, access to the data-storage volume may be blocked. Upon successful conversion, various post conversion operations may be performed.

Abstract: Techniques are provided for distributing data over a network. One or more source nodes distribute data to an intermediate node. The intermediate node accumulates the data, dynamically selects a first path for distributing the data to a first destination node, and dynamically selects a second path for distributing the data to a second destination node. In an embodiment, the data is simultaneously distributed over the first and second paths from the intermediate node.