Abstract: A network storage server system includes a distributed object store and a metadata subsystem. The metadata subsystem stores metadata relating to the stored data objects and allows data objects to be located and retrieved easily via user-specified search queries. It manages and allows searches on at least three categories of metadata via the same user interface and technique. These categories include user-specified metadata, inferred metadata and system-defined metadata. Search queries for the metadata can include multi-predicate queries.

Abstract: Technology is disclosed for managing data in a distributed file system (“the technology”). The technology can gather metadata information associated with the data stored within a first file system, store the metadata information in association with a data identifier within a second file system, retrieve the stored metadata information using the data identifier from within the second file system and locate and retrieve the data associated with the metadata information from within first file system.

Abstract: Technology is disclosed for managing data in a distributed processing system (“the technology”). In various embodiments, the technology pushes “cold” data from a primary storage of the distributed processing system to a backup storage thereby maximizing the usage of the space on the primary storage to store “hot” data on which most data processing activities are performed in the distributed processing system. The cold data is retrieved from the backup storage into the primary storage on demand, for example, upon receiving an access request from a client. While the primary storage stores the data in a format specific to the distributed processing system, the backup storage stores the data in a different format, for example, format corresponding to the type of backup storage.

Abstract: Methods and apparatuses for efficiently migrating deduplicated data are provided. In one example, a data management system includes a data storage volume, a memory including machine executable instructions, and a computer processor. The data storage volume includes data objects and free storage space. The computer processor executes the instructions to perform deduplication of the data objects and determine migration efficiency metrics for groups of the data objects. Determining the migration efficiency metrics includes determining, for each group, a relationship between the free storage space that will result if the group is migrated from the volume and the resources required to migrate the group from the volume.

Abstract: A system and method for planning and configuring the components of a modular computing system is provided. In some embodiments, the method for planning an implementation of a modular computing system comprises presenting a user interface at a display device, the user interface including a plurality of user-selectable objects, each of the user-selectable objects representing a component of the modular computing system. A user selection is received via a user input device. The user selection is from among the user-selectable objects and specifies one of an enclosure, an existing component, and a future component of the modular computing system. A representation of the specified one of an enclosure, an existing component, and a future component is displayed at a display device. The user selection is verified with respect to an implementation guideline. An indicator of whether the user selection meets the implementation guideline is displayed at the display device.

Abstract: A system and method for providing customer guidance in deploying a modular computing system is provided. In some embodiments, the method comprises receiving a shipping container identifier. A computing system is used to determine, based on the shipping container identifier, that a component of the modular computing system has been received by a customer. It is determined whether the modular computing system can be deployed based on the component having been received by the customer. An indicator is provided of whether the modular computing system can be deployed. An instruction is provided for deploying the modular computing system, and a diagnostic procedure is performed on a deployed component of the modular computing system. In one such embodiment, the diagnostic procedure determines whether the instruction was correctly performed.

Abstract: A system and method for helping customers install a modular computing system is provided. In some embodiments, the method comprises determining, using a computing system, whether all components of a modular computing system have been received by a customer. Based on determining that not all the components of the modular computing system have been received by the customer, it is determined whether the modular computing system can be incrementally deployed. A customer instruction for incrementally deploying the modular computing system is provided at a display device. A communication link is established with a component of the modular computing system. Communications directed over the communication link are used to determine whether a customer correctly performed the provided customer instruction. In one such embodiment, the determining of whether the modular computing system can be incrementally deployed includes determining whether a minimum resource requirement is met.

Abstract: Described herein is a system and method for a scalable crash-consistent snapshot operation. Write requests may be received from an application and a snapshot creation request may further be received. Write requests received before the snapshot creation request may be associated with pre-snapshot tags and write requests received after the snapshot creation request may be associated with post-snapshot tags. Furthermore, in response to the snapshot creation request, logical interfaces may begin to be switched from a pre-snapshot configuration to a post-snapshot configuration. The snapshot may then be created based on the pre-snapshot write requests and the post-snapshot write requests may be suspended until the logical interfaces have switched configuration.

Abstract: The techniques introduced herein provide for systems and methods for creating and managing a contention-free multi-path access to a distributed data set in a distributed processing system. In one embodiment, a distributed processing system comprises a plurality of compute nodes. The compute nodes are assembled into compute groups and configured such that each compute group has an attached or local storage system. Various data segments of the distributed data set are stored in data storage objects on the local storage system. The data storage objects are cross-mapped into each of the compute nodes in the compute group so that any compute node in the group can access any of the data segments stored in the local storage system via the respective data storage object.

Abstract: A network storage server system includes a distributed object store and a metadata subsystem. The metadata subsystem stores metadata relating to the stored data objects and allows data objects to be located and retrieved easily via user-specified search queries. It manages and allows searches on at least three categories of metadata via the same user interface and technique. These categories include user-specified metadata, inferred metadata and system-defined metadata. Search queries for the metadata can include multi-predicate queries.

Abstract: A network storage server system includes a distributed object store and a metadata subsystem. The metadata subsystem stores metadata relating to the stored data objects and allows data objects to be located and retrieved easily via user-specified search queries. It manages and allows searches on at least three categories of metadata via the same user interface and technique. These categories include user-specified metadata, inferred metadata and system-defined metadata. Search queries for the metadata can include multi-predicate queries.

Abstract: Methods and systems for concurrently reading direct and indirect data blocks of a data object stored in a network storage server system. In one embodiment, upon receiving a request to read a data object, the storage server identifies a location of an indirect data-block of the data object and a total number of direct data-blocks associated with the data object. Using this information, the storage server concurrently reads, using a single read operation, both the indirect data-block and a specific number of data blocks that are contiguous with the location of the indirect data-block. The specific number is commensurate with the total number of direct-data blocks associated with the data object. In one embodiment, the storage server verifies whether the data object is represented using a contiguous-mode layout scheme before performing the concurrent single-read operation to read the data object.

Abstract: Bandwidth consumption between a data replication source and destination and storage consumption at the destination are reduced, when logical block mirroring is used with source deduplication, by eliminating repeated transmission of data blocks from source to destination. A reference is created for each data block at the source, the reference being unique within a storage aggregate of the source. During a mirror update, the source initially sends only the references of modified data blocks to the destination. The destination compares those references against a data structure to determine whether the destination already has any of those data blocks stored. If the destination determines that it already has a data block stored, it does not request or receive that data block again from the source. Only if the destination determines that it has not yet received the referenced data block does it request and receive that data block from the source.

Abstract: Methods and systems for concurrently reading direct and indirect data blocks of a data object stored in a network storage server system. In one embodiment, upon receiving a request to read a data object, the storage server identifies a location of an indirect data-block of the data object and a total number of direct data-blocks associated with the data object. Using this information, the storage server concurrently reads, using a single read operation, both the indirect data-block and a specific number of data blocks that are contiguous with the location of the indirect data-block. The specific number is commensurate with the total number of direct-data blocks associated with the data object. In one embodiment, the storage server verifies whether the data object is represented using a contiguous-mode layout scheme before performing the concurrent single-read operation to read the data object.