Abstract: A content platform (or “cluster”) that comprises a redundant array of independent nodes is logically partitioned. Using a web-based interface, an administrator defines one or more “tenants” within the cluster, wherein a tenant has a set of attributes: namespaces, administrative accounts, data access accounts, and a permission mask. A namespace is a logical partition of the cluster that serves as a collection of objects typically associated with at least one defined application. Each namespace has a private file system with respect to other namespaces. This approach enables a user to segregate cluster data into logical partitions. Using the administrative interface, a namespace associated with a given tenant is selectively configured without affecting a configuration of at least one other namespace in the set of namespaces. One configuration option is a “disposition service” that, once enabled for a namespace, automatically deletes objects that were once under retention and whose retention time has expired.

Abstract: An “operate with missing region” feature of this disclosure allows the cluster to continue servicing reads for available regions even when some regions are missing. In particular, upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. As used herein, “read-only” means that no client write or update requests will succeed while the cluster is in this state. In particular, those requests should fail with an error message (or the like) that indicates failure because of the read-only status. Preferably, such requests are then re-tried. In this mode, all regions are only allowed to perform read operations. During the read-only state, the cluster continues to operate with missing regions, and missing regions are entered on the region map.

Abstract: An automated system randomly generates test cases for hardware or software quality assurance testing. A test case comprises a sequence of discrete, atomic steps (or “building blocks”). A particular test case has a variable number of building blocks. The system takes a set of test actions and links them together to create a much larger library of test cases or “chains.” The chains comprise a large number of random sequence tests that facilitate “chaos-like” or exploratory testing of the overall system under test. Upon execution in the system under test, the test case is considered successful if each building block in the chain executes successfully; if any building block fails, the test case, in its entirety, is considered a failure.

Abstract: An archival storage cluster of preferably symmetric nodes includes a data privacy scheme that implements key management through secret sharing. In one embodiment, the protection scheme is implemented at install time. At install, an encryption key is generated, split, and the constituent pieces written to respective archive nodes. The key is not written to a drive to ensure that it cannot be stolen or otherwise compromised. Due to the secret sharing scheme, any t of the n nodes must be present before the cluster can mount the drives. Thus, to un-share the secret, a process runs before the cluster comes up. It contacts as many nodes as possible to attempt to reach a sufficient t value. Once it does, the process un-shares the secret and mounts the drives locally. Given bidirectional communication, this mount occurs more or less at the same time on all t nodes. Once the drives are mounted, the cluster can continue to boot as normal.

Abstract: An archival storage cluster of symmetric nodes includes a metadata management system that organizes metadata objects. Each metadata object may have a unique name, and metadata objects are organized into regions. A region is selected by hashing one or more object attributes and extracting a given number of bits of the resulting hash value. The number of bits may be controlled by a configuration parameter. Each region is stored redundantly. A region comprises a set of region copies. In particular, there is one authoritative copy of the region, and zero or more backup copies. The number of backup copies may be controlled by a configuration parameter. Region copies are distributed across the nodes of the cluster to balance the number of authoritative region copies per node, and the number of total region copies per node. Backup region copies are maintained synchronized to their associated authoritative region copy.

Abstract: Embodiments of the invention provide a method of storage array with frame forwarding capability to reduce the complexity of an IT platform system having a large number of hardware devices. In one embodiment, a storage system comprises one or more storage devices; a storage controller having a storage controller processor and a storage controller memory and being coupled with the one or more storage devices; an interface controller receiving an input/output command directly from a host computer; and an internal bus. The interface controller and the storage controller are coupled by the internal bus. In another embodiment, a storage system comprises one or more storage devices; a storage controller having a processor and a memory and being coupled with the one or more storage devices; and a network interface controller which provides virtual Ethernet bridging.

Abstract: An archival storage cluster of preferably symmetric nodes includes a metadata management system that organizes and provides access to given metadata, preferably in the form of metadata objects. Each metadata object may have a unique name, and metadata objects are organized into regions. Preferably, a region is selected by hashing one or more object attributes (e.g., the object's name) and extracting a given number of bits of the resulting hash value. The number of bits may be controlled by a configuration parameter. Each region is stored redundantly. A region comprises a set of region copies. In particular, there is one authoritative copy of the region, and zero or more backup copies. The number of backup copies may be controlled by a configuration parameter. Region copies are distributed across the nodes of the cluster so as to balance the number of authoritative region copies per node, as well as the number of total region copies per node.

Abstract: A generic testing framework to automatically allocate, install and verify a given version of a system under test, to exercise the system against a series of tests in a “hands-off” objective manner, and then to export information about the tests to one or more developer repositories (such as a query-able database, an email list, a developer web server, a source code version control system, a defect tracking system, or the like). The framework does not “care” or concern itself with the particular implementation language of the test as long as the test can issue directives via a command line or configuration file. During the automated testing of a given test suite having multiple tests, and after a particular test is run, the framework preferably generates an “image” of the system under test and makes that information available to developers, even while additional tests in the suite are being carried out.

Abstract: Embodiments of the invention provide a technique to model applications and storage used thereby. An aspect of the present invention is directed to a method for managing a computer system which includes at least one host computer and at least one storage system which are connected via a network, the at least one host computer having an application running thereon. The method comprises modeling an application running on a host of a computer system as an application object, which includes associating storage utilized by the application with the application object; tracking operation of the storage associated with the application object as a service being delivered by the storage to the application; and presenting result on the service being delivered by the storage to the application based on tracking the operation of the storage.

Abstract: A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into a plurality of systems each having multiple nodes. A data protection level method comprises setting a numerical value of a system dynamic data protection level for each system of nodes; and in response to user input, setting a value of a data protection level for one or more data objects in a system, the value being a fixed numerical value or a dynamic value. If the one or more data objects have a dynamic value of the data protection level in a system, the one or more data objects inherit the system dynamic data protection level of the system in which the one or more data objects reside. The value of the data protection level represents a number of copies of data of the one or more data objects to be kept where the one or more data objects reside.

Abstract: An “operate with missing region” feature of this disclosure allows the cluster to continue servicing reads for available regions even when some regions are missing. In particular, upon a given node failure condition, the cluster is placed in an effective read-only mode for all regions. The node failure condition typically is one where there has been a failure of an authoritative region copy and no backup copy is then available. As used herein, “read-only” means that no client write or update requests will succeed while the cluster is in this state. In particular, those requests should fail with an error message (or the like) that indicates failure because of the read-only status. Preferably, such requests are then re-tried. In this mode, all regions are only allowed to perform read operations. During the read-only state, the cluster continues to operate with missing regions, and missing regions are entered on the region map.

Abstract: This disclosure enables a user to incrementally increase or decrease the size of the cluster and, in particular, to incrementally increase or decrease the number of map regions implemented in a metadata manager associated with the cluster while maintaining availability of the cluster. In a representative embodiment, a configurable target map size is identified, either manually (e.g., by the user) or programmatically. If the target map size is found to be within an acceptable value (given the available node configuration), a procedure to create a new map is triggered. The new map preferably is generated as follows. Initially, one or more new temporary region(s) are created either to split or join the existing regions (depending on whether the map is being increased or decreased). For each temporary region, the process also generates MDPL backup regions. Metadata from an original region (or regions) is then migrated to the temporary region(s).

Abstract: Cluster recovery time is reduced by an incremental refresh technique that is described herein. The goal of the technique is to salvage the portion of the redundant distributed database lost (e.g., during a fault) by performing an incremental refresh of only the updates occurring to that portion of the database since the time of loss.

Abstract: A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. The nodes are grouped into a plurality of systems each having multiple nodes. An object recovery method comprises: receiving, by a first system of the plurality of systems from a client application, a read request for an object, the object having been replicated to/from at least one second system among the plurality of systems; if the object of the read request is available in the first system, returning by the first system the object of the read request to the client application; and if the object of the read request is not available in the first system, performing a read from replica process by the first system to access a replica of the object from a second system among the plurality of systems and using the replica of the object to return the object of the read request to the client application.

Abstract: A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. Metadata objects are stored in a set of regions distributed among the nodes across the array. A given region is identified by hashing a metadata object attribute and extracting a given set of bits of a resulting hash value. A method of managing query results comprises: receiving, by a first node of the plurality of independent nodes from a client application, a request for a list of objects with a criterion; issuing by the first node a query to all the nodes based on the received request; processing the query by each node over the regions in the node using the metadata objects stored in the regions; aggregating and filtering by the first node results of the query from all the nodes; and returning by the first node the aggregated and filtered results to the client application.

Abstract: Embodiments of the invention provide a method of storage array with frame forwarding capability to reduce the complexity of an IT platform system having a large number of hardware devices. In one embodiment, a storage system comprises one or more storage devices; a storage controller having a storage controller processor and a storage controller memory and being coupled with the one or more storage devices; an interface controller receiving an input/output command directly from a host computer; and an internal bus. The interface controller and the storage controller are coupled by the internal bus. In another embodiment, a storage system comprises one or more storage devices; a storage controller having a processor and a memory and being coupled with the one or more storage devices; and a network interface controller which provides virtual Ethernet bridging.

Abstract: A cluster recovery process is implemented across a set of distributed archives, where each individual archive is a storage cluster of preferably symmetric nodes. Each node of a cluster typically executes an instance of an application that provides object-based storage of fixed content data and associated metadata. According to the storage method, an association or “link” between a first cluster and a second cluster is first established to facilitate replication. The first cluster is sometimes referred to as a “primary” whereas the “second” cluster is sometimes referred to as a “replica.” Once the link is made, the first cluster's fixed content data and metadata are then replicated from the first cluster to the second cluster, preferably in a continuous manner. Upon a failure of the first cluster, however, a failover operation occurs, and clients of the first cluster are redirected to the second cluster.

Abstract: A method for representing a computer-related resource in a computer-related system environment comprises creating an object model of the resource, the object model including objects that correspond to components of the resource, configuration information of the objects, and relationships among the objects; providing a resource application program interface to allow communication with and access to the components of the resource; mapping the objects of the object model to files and directories of a file and directory structure; presenting the file and directory structure via a file system application program interface to allow an application to manipulate the files and directories so as to manipulate the components of the resource via the resource application program interface; and manipulating the components of the resource based on manipulation of the files and directories from the application.

Abstract: A cluster recovery process is implemented across a set of distributed archives, where each individual archive is a storage cluster of preferably symmetric nodes. Each node of a cluster typically executes an instance of an application that provides object-based storage of fixed content data and associated metadata. According to the storage method, an association or “link” between a first cluster and a second cluster is first established to facilitate replication. The first cluster is sometimes referred to as a “primary” whereas the “second” cluster is sometimes referred to as a “replica.” Once the link is made, the first cluster's fixed content data and metadata are then replicated from the first cluster to the second cluster, preferably in a continuous manner. Upon a failure of the first cluster, however, a failover operation occurs, and clients of the first cluster are redirected to the second cluster.

Abstract: A cluster recovery process is implemented across a set of distributed archives, where each individual archive is a storage cluster of preferably symmetric nodes. Each node of a cluster typically executes an instance of an application that provides object-based storage of fixed content data and associated metadata. According to the storage method, an association or “link” between a first cluster and a second cluster is first established to facilitate replication. The first cluster is sometimes referred to as a “primary” whereas the “second” cluster is sometimes referred to as a “replica.” Once the link is made, the first cluster's fixed content data and metadata are then replicated from the first cluster to the second cluster, preferably in a continuous manner. Upon a failure of the first cluster, however, a failover operation occurs, and clients of the first cluster are redirected to the second cluster.