Abstract: Exemplary embodiments provide the ability to associate multiple collections of metadata to objects. In one embodiment, a storage system comprises: a processor; a memory; at least one object, each object including content data; system metadata associated with each object; and a plurality of named collections of user-defined metadata associated with each object. Each named collection of the plurality of named collections is separately addressable by a request. In some embodiments, at least one named collection of user-defined metadata of one object of the at least one object includes an access control list, and the access control list of a named collection specifies permissible access to the named collection. The one object has an object-level access control list which specifies permissible access to the one object and which is different from the access control list for the named collection associated with the one object.

Abstract: Example implementations described herein include systems and methods for processing the deletion of a directory that can include one or more subdirectories. In a system including a server having a hardware-side and a software-side that manages one or more storage systems or storage devices, a client may request the deletion of a directory, whereupon the processing of the directory occurs in a work queue and subtree queue for processing subdirectories.

Abstract: A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into systems each having multiple nodes. A search index is provided in each system for locating objects within the system. A method of managing the search index comprises splitting the search index into shards to be divided among a plurality of regions (704). The regions are larger in number than the nodes in the system. Each region has a master and at least one backup copy of one of the shards. A shard core is a master or backup copy of a shard. The method further comprises distributing the shard cores to the nodes, and providing a region mapping table in each node in the system to map the shard cores to the nodes in which the shard cores are stored (706).

Abstract: A storage system stores objects and copies of objects on the storage system and other storage systems external to the storage system. The storage system stores the copies in storage pools of volumes, which are organized into one or more tiers. The configuration settings of each tier and each pool within the tier may be configured according to a user's preferences. In one example, the number of copies of data content and the number copies of metadata associated with the data content an individual pool stores may be specified. When objects are migrated between tiers, the objects are stored among the storage pools of the tiers. If the number of data content copies or metadata copies is increased, the data or metadata is copied from a determined copy source and if the number of copies decreases, the data is removed from the volumes in the pools.

Abstract: A method and apparatus for metadata favored replication in an active/active topology. When replicating data objects between multiple storage systems, metadata is sent first while the associated data content is sent at a subsequent time. Metadata is stored and made visible on a target storage system thereby reserving the object path in a namespace to reduce collisions in the namespace. After metadata is transferred to the target storage system, the target system performs collision processing to determine if an object already exists in the namespace prior to receiving the transferred metadata. After collision processing is performed, the winning object is made visible for access on the target storage system regardless of whether the associated data content has been transferred to the target system.

Abstract: In some examples, both object data and corresponding metadata are stored on thin provisioned block storage in which blocks for the storage of the object data and the metadata are dynamically provisioned from a shared pool. For instance, the metadata may be stored on the block storage using a tree data structure that may grow proportionally with the corresponding stored object data. The object storage may automatically adapt to the size and number of objects being stored, and storage space for metadata may be allocated dynamically on demand. Additionally, some implementations include storing data across multiple trays. For instance, a plurality of storage extents of an extent group may be allocated across a plurality of storage devices on multiple trays. The extent allocation may be tray-aware by allocating extents to maximize the probability that data remains available/recoverable in the presence of a failure in one or more trays.

Abstract: In some examples, analysis metadata may be generated for received digital assets based on analysis of content of the digital assets. As one example, a service computing device may receive and store a digital asset and first metadata for the digital asset. The service computing device may determining a content category of content of the digital asset, and may analyze the content of the digital asset based at least in part on the content category to obtain at least one analysis result as additional metadata for the digital asset. The service computing device may store the analysis result in association with the digital asset and may store analysis metadata in association with the first metadata.

Abstract: The apparatus comprises a plurality of interfaces, each interface having an associated interface ID; and a hardware-side processing device including at least one programmable hardware-implemented chip configured to process request packets, which are received from host computers and relate to access requests to one or more file system managed by the apparatus, and to generate response packets for the processed request packets; wherein, for a request packet which is received from a first host computer, at least one programmable hardware-implemented chip is configured to: determine the client ID being associated with the first host computer, determine the interface ID being associated with the first interface, determine whether the determined client ID and interface ID represent a permitted ID set or a prohibited ID set, and refrain from processing the received request packet if the determined client ID and interface ID represent a prohibited ID set.

Abstract: A file cloning process allows for quickly cloning files within a filesystem, such as when a user makes a copy of a directory containing files and subdirectories of files. A priority based main queue is filled by a depth first traversal of the directory to be copied with work items corresponding to files and subdirectories. Worker threads refer to the main queue for file cloning, subdirectory copying and directory traversal. Files that have not been modified since a most recent checkpoint are cloned without issuing a checkpoint request. Files that have been modified since the most recent checkpoint are moved to a delay queue. Once a checkpoint of the file system completes, the files in the delay queue are then moved to the main queue to retry cloning thereof. A secondary queue is also provided to prevent worker threads from becoming deadlocked in the main queue.

Abstract: Object-level replication of cloned objects from a source file system to a target file system in a data storage system maintains relationships between related objects including shared data blocks so that storage requirements in the target file system match storage requirements of the source file system. Specialized processing may be used to scan an indirection object that refers to other file system objects such that objects requiring replication can be identified on an incremental basis based on checkpoint numbers. Checkpoints in the target file system are managed so that checkpoint number requirements for replicated clone objects are enforced in the target file system.

Abstract: Example implementations are directed to restoration of data not only from the servers, but also from the clients connected to the server. Algorithms are incorporated to identify content that was created or modified after the last backup. This algorithm also identifies and resolves changes in the mount points for shared folders, preventing information leakage. When the server recovers from a failure, it notifies the clients about the recovery on their next connection. Each client then determines the current state of its mount points and file paths and compares them with the server's mount points and file paths. After the comparison, the client comprehends and indicates the mount point differences by renaming them and sends entire local data (all files, folders, mount points) to the server. The server reconciles the differences.

Abstract: The storage system includes storage devices which have plural storage blocks. A first node with a first processor memory is configured to store first information which manages states of the storage blocks, and a second node including a second processor and memory is configured to store second information which manages the states of the storage blocks are included in the storage system to handle file operations. When a failure occurs on the second node, the first processor is configured to change the state of one or more storage blocks, previously reserved by the second node to be used for storing data by the second node, to one or more garbage blocks and perform garbage resolution processing thereon before becoming available for data storage. This process allows for rapid recovery of the filesystem as the entirety of the filesystem metadata need not be examined.

Abstract: A method, of managing storage of content of a system in storage units including run units which do not allow spin-down functionality and spin-down units which allow spin-down functionality, comprises: establishing a storage tiering rule which sets a policy indicating what content is eligible to be stored on a run unit and what content is eligible to be stored on a spin-down unit, the storage tiering rule being applicable to at least a group of the contents in the content system to determine stored content eligibility thereof for storage in the storage units and migration between the storage units; and identifying candidates of contents to migrate between the storage units, based on the storage tiering rule, state of the content system, and the stored content eligibility of the at least a group of the contents.

Abstract: Example implementations described herein are directed to a transaction query engine involving a token that is generated by the server and sent to the corresponding the client. The client uses the token to query transactions made on the server. The token can include mountpoint information detailing shared and private information of the filesystems, transaction identifiers identifying transactions of a filesystem, and a filesystem identifier to identify the corresponding filesystem. The server processes this token and updates the token based on the transaction information of the filesystem, and sends the token back to the corresponding client.

Abstract: A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into a plurality of systems each having multiple nodes. The systems have one or more replication links each being formed to indicate replication of data from one system to another system in a replicated environment where each system is configured as a sub-domain in a Domain Name System (DNS) infrastructure. A DNS alias synchronization method comprises maintaining updated information, within each system, of all replication links involving that system and of DNS aliases of other systems associated with all replication links involving that system (S602-S607). This enables that system to process network-based requests, on behalf of the other systems, without redirecting the requests from the other systems to that system.

Abstract: A storage system comprises a controller; a memory; and one or more objects each including content data and metadata. The metadata is used to construct user-defined content properties, each content property providing an ability to extract a specific metadata field from the objects by referring to a user-defined content property name of the content property. The content properties are organized into user-defined content classes, each content class grouping a set of content properties into a named category under a user-defined content class name. The controller is operable to index the content properties of the content classes to create an index. The content properties being indexed are identified by the content property names. In some embodiments, the controller is operable to de-dupe the index, for content properties having same values expressed differently in different metadata formats, by directing the differently expressed values to same index field under same content property name.

Abstract: A storage system has a plurality of nodes which are grouped into a plurality of cluster systems each having multiple nodes, each cluster system being logically partitioned into a plurality of namespaces, each namespace including a collection of data objects, each cluster system having multiple tenants, each tenant being a grouping of namespaces, each cluster system having a plurality of capabilities, at least some of the capabilities being bound to the tenants. A node in the cluster system comprises: a memory, and a controller operable to bind each capability to one of a plurality of IP networks so that each capability is bound to only one of the IP networks and has a destination IP address of the IP network to which the capability is bound. It is permissible for one or more capabilities to be bound to the same IP network. Each IP network has one corresponding network interface.