Abstract: A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Data is written in segments and Metadata is stored in the segments indicating LBAs of data stored therein and offsets within the segments at which data for LBAs are stored. For write requests, index entries are stored in an index buffer for a segment and written to the segment when the buffer is full. A redo entry is created in a redo buffer for multiple storage volumes and slices for each write request. Write requests are acknowledged when the redo buffer is written to redo segments on a storage device. On restart, index buffers are reconstructed from the redo segments if needed.

Abstract: A new snapshot of a storage volume is created by suppressing write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A block map records segments where current data for an LBA of a slice of a storage volume is stored. Block maps may be written to a storage device in order to free memory. Block maps may be read back into memory when needed. Writing and reading of block maps may be performed upon fragments of block maps. On restarting of the storage node, block maps may be restored from block maps stored in the storage device.

Abstract: A storage scheme allocates portions of a logical volume to storage nodes in excess of the capacity of the storage nodes. Slices of the storage nodes and segments of slices are allocated in response to write requests such that actual allocation on the storage nodes is only in response to usage. Segments are identified with virtual segment identifiers that are retained when segments are moved to a different storage node. Logical volumes may therefore be moved seamlessly to different storage nodes to ensure sufficient storage capacity. Data is written to new locations in segments having space and a block map tracks the last segment to which data for a given address is written. Garbage collection is performed to free segments that contain invalid data, i.e. data for addresses that have been subsequently written to.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. The snapshots may be represented by a storage manager in a hierarchy. Deleted snapshots may be flagged as such in the hierarchy and deletion may be implemented only in memory on a storage node, which is then restored from the hierarchy in the event of a crash. A snapshot is removed from the hierarchy when all segments previously are freed by garbage collection. A hybrid storage node may perform both computing and storage services. Data may be written with tags indicating encoding protocols used to encode the data.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. The snapshots may be represented by a storage manager in a hierarchy. Deleted snapshots may be flagged as such in the hierarchy and deletion may be implemented only in memory on a storage node, which is then restored from the hierarchy in the event of a crash. A hybrid storage node may perform both computing and storage services. An IO module determines whether IOPs reference a locally-mounted storage device or a remote storage device. The IO module issues a library call for local IOPs using a shared memory in kernel space.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. The snapshots may be represented by a storage manager in a hierarchy. Deleted snapshots may be flagged as such in the hierarchy and deletion may be implemented only in memory on a storage node, which is then restored from the hierarchy in the event of a crash. A snapshot is removed from the hierarchy when all segments previously are freed by garbage collection. A hybrid storage node may perform both computing and storage services. Data may be written with tags indicating encoding protocols used to encode the data.

Abstract: A storage scheme allocates portions of a logical volume to storage nodes in excess of the capacity of the storage nodes. Slices of the storage nodes and segments of slices are allocated in response to write requests such that actual allocation on the storage nodes is only in response to usage. Segments are identified with virtual segment identifiers that are retained when segments are moved to a different storage node. Logical volumes may therefore be moved seamlessly to different storage nodes to ensure sufficient storage capacity. Data is written to new locations in segments having space and a block map tracks the last segment to which data for a given address is written. Garbage collection is performed to free segments that contain invalid data, i.e. data for addresses that have been subsequently written to.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. A snapshot of the application may be created and used to rollback or clone the application. Clones snapshots of storage volumes may be gradually populated with data from prior snapshots to reduce loading on a primary snapshot. Components of cloned applications may communicate with one another using addresses of these components in the parent application. Changes to application state may be written to a remote storage volume that may be mounted to a new instance or clone of the application to recreate its state. Jobs to create and execute a bundled application may be referenced with a simulated file system that generates reads to hosts only when the job log file is actually read. Storage nodes may implement interfaces to a SAN or cloud storage system.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. A snapshot of the application may be created and used to rollback or clone the application. Clones snapshots of storage volumes may be gradually populated with data from prior snapshots to reduce loading on a primary snapshot. Components of cloned applications may communicate with one another using addresses of these components in the parent application. Changes to application state may be written to a remote storage volume that may be mounted to a new instance or clone of the application to recreate its state. Jobs to create and execute a bundled application may be referenced with a simulated file system that generates reads to hosts only when the job log file is actually read. Storage nodes may implement interfaces to a SAN or cloud storage system.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. The snapshots may be represented by a storage manager in a hierarchy. Deleted snapshots may be flagged as such in the hierarchy and deletion may be implemented only in memory on a storage node, which is then restored from the hierarchy in the event of a crash. A hybrid storage node may perform both computing and storage services. An IO module determines whether IOPs reference a locally-mounted storage device or a remote storage device. The IO module issues a library call for local IOPs using a shared memory in kernel space.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. The snapshots may be represented by a storage manager in a hierarchy. Deleted snapshots may be flagged as such in the hierarchy and deletion may be implemented only in memory on a storage node, which is then restored from the hierarchy in the event of a crash. A snapshot is removed from the hierarchy when all segments previously are freed by garbage collection. A hybrid storage node may perform both computing and storage services. Data may be written with tags indicating encoding protocols used to encode the data.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. The snapshots may be represented by a storage manager in a hierarchy. Deleted snapshots may be flagged as such in the hierarchy and deletion may be implemented only in memory on a storage node, which is then restored from the hierarchy in the event of a crash. A snapshot is removed from the hierarchy when all segments previously are freed by garbage collection. A hybrid storage node may perform both computing and storage services. Data may be written with tags indicating encoding protocols used to encode the data.

Abstract: A distributed storage schemes manages implementation of QoS targets for IOPs across compute nodes executing applications, primary storage nodes storing a primary copy of a logical storage volume, and clone storage nodes. On the compute node, a maximum priority is assigned to a minimum number of IOPs in a queue within a time window from a time of receipt of a last unexecuted IOP. Other IOPs are assigned a minimum priority. On the storage node, maximum priority IOPs are assigned to high priority queues, from which IOPs are executed first, and low priority IOPs are assigned to low priority queues. Methods for determining the capacity of storage nodes and allocating storage requests are also disclosed.

Abstract: A distributed storage schemes manages implementation of QoS targets for IOPs across compute nodes executing applications, primary storage nodes storing a primary copy of a logical storage volume, and clone storage nodes. On the compute node, a maximum priority is assigned to a minimum number of IOPs in a queue within a time window from a time of receipt of a last unexecuted IOP. Other IOPs are assigned a minimum priority. On the storage node, maximum priority IOPs are assigned to high priority queues, from which IOPs are executed first, and low priority IOPs are assigned to low priority queues. Methods for determining the capacity of storage nodes and allocating storage requests are also disclosed.

Abstract: A storage scheme allocates portions of a logical volume to storage nodes in excess of the capacity of the storage nodes. Slices of the storage nodes and segments of slices are allocated in response to write requests such that actual allocation on the storage nodes is only in response to usage. Segments are identified with virtual segment identifiers that are retained when segments are moved to a different storage node. Logical volumes may therefore be moved seamlessly to different storage nodes to ensure sufficient storage capacity. Data is written to new locations in segments having space and a block map tracks the last segment to which data for a given address is written. Garbage collection is performed to free segments that contain invalid data, i.e. data for addresses that have been subsequently written to.

Abstract: A storage scheme allocates portions of a logical volume to storage nodes in excess of the capacity of the storage nodes. Slices of the storage nodes and segments of slices are allocated in response to write requests such that actual allocation on the storage nodes is only in response to usage. Segments are identified with virtual segment identifiers that are retained when segments are moved to a different storage node. Logical volumes may therefore be moved seamlessly to different storage nodes to ensure sufficient storage capacity. Data is written to new locations in segments having space and a block map tracks the last segment to which data for a given address is written. Garbage collection is performed to free segments that contain invalid data, i.e. data for addresses that have been subsequently written to.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A segment maps segments to a particular snapshot and metadata stored in the segment indicates storage volume addresses of data written to the segment. The snapshots may be represented by a storage manager in a hierarchy that identifies an ordering of snapshots and branches to clone snapshots.

Abstract: A new snapshot of a storage volume is created by instructing computing nodes to suppress write requests. Once pending write requests from the computing nodes are completed, storage nodes create a new snapshot for the storage volume by allocating a new segment to the new snapshot and finalizes and performs garbage collection with respect to segments allocated to the previous snapshot. Subsequent write requests to the storage volume are then performed on the segments allocated to the new snapshot. A segment maps segments to a particular snapshot and metadata stored in the segment indicates storage volume addresses of data written to the segment. The snapshots may be represented by a storage manager in a hierarchy that identifies an ordering of snapshots and branches to clone snapshots.

Abstract: Systems and methods are disclosed for interacting with a file system. The file system is operable to reside in user space of a computing system. A module, also within user space, may provide a messaging service supporting requests from an application to the file system. By bypassing a System-Call Interface (SCI) of the computing system's kernel space, the module may support requests from the application to the file system with enhanced efficiency and/or customizable features not provided by the SCI. In some examples, the module may include a library in an independent layer within user space and below the application, allowing the library to provide an application-independent messaging service for different applications. Furthermore, in some examples, the module may include a segment of memory, within user space, shared between the application and the file system for passing data involved in requests and/or responses to and/or from the file system.

Abstract: Systems and methods are disclosed for classifying traffic flows. A traffic agent operable to collect classification information for one or more traffic flows may be deployed at an end host communicatively coupled to a data-center network. The traffic agent, deployed in a user space independent of the operating system, may compare the classification information for a given traffic flow to a metric value. Where the classification information achieves a certain threshold indicated by the metric value, the traffic agent may classify the traffic flow as an elephant flow. In some examples, a library may be included with the traffic agent that may include a modified send function. The modified send function may provide classification information to the traffic agent indexed to the traffic flow for which it is called so that the traffic agent may analyze the classification information to potentially provide a classification for the traffic flow.