Abstract: Providing a service in a storage system includes using a predicate that corresponds to a subset of storage objects in a set of storage objects stored in the storage system. The predicate is used to select or reject the objects to which the service is to be provided. The predicate is applied to the set of storage objects to obtain the subset of storage objects. The service is then invoked for the subset of storage objects selected according to the predicate. A trigger event causes invocation of the service. The trigger event may be periodic and/or may be a storage object lifecycle event, a storage object access related event, an event corresponding to a state change of the storage system, or an event that is external to the storage system. The trigger event may include the predicate being modified.

Abstract: Providing a snapshot copy of data includes selectively collecting a plurality of data objects based on evaluation of a predicate associated with each of the data objects and performing a snapshot operation in connection with modifying at least one of the plurality of data objects. The snapshot operation may be performed prior to modifying the at least one of the plurality of data objects. The snapshot operation may be a copy on write operation. Providing a snapshot copy of data may also include providing a data structure that maintains a plurality of snapshot instances. The data structure may be a table. Each of the snapshot instances may include at least one of: an object set description, a timestamp indicating when the snapshot was initiated, a snapshot name and a handling policy.

Abstract: Determining at least one resource node for deployment of an application in a system having a plurality of compute and storage resource nodes includes determining criteria for nodes based on a policy provided for the application, pruning nodes that do not meet a criteria for deploying the application to provide a plurality of remaining nodes, determining a cost of deploying the application on each of the plurality of remaining nodes, where the cost is based on a metric associated with the application, and selecting for deployment a node having a lowest cost. The application may be a vApp that includes a plurality of virtual machine images. The nodes may be interconnected by communication links having associated therewith costs of moving the application from one node to another.

Abstract: Managing file objects in a data storage system includes providing a plurality of metadata servers, each metadata server having information for only a subset of file objects in the storage system, each subset for each of the metadata servers being less than a total number of file objects for the system and includes providing at least one metadata location server, where an entity accessing a specific file object of the system determines which of the metadata servers contains data for the specific file object by first accessing the at least one metadata location server. Information for the subset of file objects may include a table having file identifiers and metadata location information. The metadata location information may point to data storage for a metadata object for a file.

Abstract: Example embodiments of the present invention effectively manage a large set of records such that each can be quickly accessed while still reducing the system capacity used for storing the records by taking into account specifics of the record structure. A template document is constructed for a large set of similar documents, such that it represents the maximum common portion of content in the document set. The template is compressed and stored. Every document in the set is then concatenated individually to the uncompressed template and the concatenated result is compressed. The compressed template is then subtracted from the combined compressed result. The result of this subtraction is stored in the data store for each document. Effectively, only the compressed difference between each document and the template is stored, which reduces significantly the amount of capacity necessary for storing the document set (e.g., by a factor of 5 or 10).

Abstract: Locating a file object in a storage system includes determining at least one characteristic of the file object, obtaining a subset of nodes in the storage system by applying the at least one characteristic to criteria for storing file objects at particular ones of the nodes in the storage system, and examining the subset of nodes to determine if any contain the file object. A metadata location server may be used to obtain the subset of nodes. The criteria may include at least one policy relating to placement of objects corresponding to files at the time the files are created. The policies may relate to owners of the files. The characteristic information may include the owner of a target file corresponding to the file object, the date that the target file was created, a business organization associated with the target file, and/or the size of the target file.

Abstract: Tracking storage resources includes providing a table containing storage resources along with capabilities and statuses thereof, updating the table in response to a change of status of a storage resource, updating the table in response to a change in capabilities of a storage resource and, in response to an inquiry for a storage resource having a particular capability, searching the table for a storage resource having the particular capability. Tracking storage resources may also include adding an element to the table in response to a new resource being added to the system. The capabilities may include RAID striping, data deduplication, and green operation. The status may be one of: on-line, off-line, and full.

Abstract: Accessing data includes determining if the data is provided on a local group of servers or on an external group of servers. If the data is provided on a local group of servers, a storage server is used to access the data. If the data is provided on an external group of servers, a proxy server is used to access the data. The proxy server interacts with an entity accessing the data in a manner that is substantially similar to interaction between the entity and the storage server. Using the proxy server may include initially providing an account id and a password. Following providing an account id and a password, using the proxy server may include using an account id and a shared secret. Using the proxy server may include using RSA ID tokens or cryptographic certificates.

Abstract: Providing a snapshot copy of data includes selectively collecting a plurality of data objects based on evaluation of a predicate associated with each of the data objects and performing a snapshot operation in connection with modifying at least one of the plurality of data objects. The snapshot operation may be performed prior to modifying the at least one of the plurality of data objects. The snapshot operation may be a copy on write operation. Providing a snapshot copy of data may also include providing a data structure that maintains a plurality of snapshot instances. The data structure may be a table. Each of the snapshot instances may include at least one of: an object set description, a timestamp indicating when the snapshot was initiated, a snapshot name and a handling policy.

Abstract: Locating a file object in a storage system includes determining at least one characteristic of the file object, obtaining a subset of nodes in the storage system by applying the at least one characteristic to criteria for storing file objects at particular ones of the nodes in the storage system, and examining the subset of nodes to determine if any contain the file object. A metadata location server may be used to obtain the subset of nodes. The criteria may include at least one policy relating to placement of objects corresponding to files at the time the files are created. The policies may relate to owners of the files. The characteristic information may include the owner of a target file corresponding to the file object, the date that the target file was created, a business organization associated with the target file, and/or the size of the target file.

Abstract: Managing file objects in a data storage system includes providing a plurality of metadata servers, each metadata server having information for only a subset of file objects in the storage system, each subset for each of the metadata servers being less than a total number of file objects for the system and includes providing at least one metadata location server, where an entity accessing a specific file object of the system determines which of the metadata servers contains data for the specific file object by first accessing the at least one metadata location server. Information for the subset of file objects may include a table having file identifiers and metadata location information. The metadata location information may point to data storage for a metadata object for a file.

Abstract: Managing file objects in a data storage system includes providing a plurality of metadata servers, each metadata server having information for only a subset of file objects in the storage system, each subset for each of the metadata servers being less than a total number of file objects for the system and includes providing at least one metadata location server, where an entity accessing a specific file object of the system determines which of the metadata servers contains data for the specific file object by first accessing the at least one metadata location server. Information for the subset of file objects may include a table having file identifiers and metadata location information. The metadata location information may point to data storage for a metadata object for a file.

Abstract: Managing file objects in a data storage system includes providing a plurality of metadata servers, each metadata server having information for only a subset of file objects in the storage system, each subset for each of the metadata servers being less than a total number of file objects for the system and includes providing at least one metadata location server, where an entity accessing a specific file object of the system determines which of the metadata servers contains data for the specific file object by first accessing the at least one metadata location server. Information for the subset of file objects may include a table having file identifiers and metadata location information. The metadata location information may point to data storage for a metadata object for a file.

Abstract: Accessing data file objects includes providing a file system interface on a client, where the file system interface allows an application running on the client to make file system calls to access the data file objects. A Web Services interface may also be provided on the client, where the Web Services interface allows an application to access file objects using the Web Services at the same time that file objects are being accessed through the file system interface. The Web Services interface may be a SOAP interface and/or a REST interface. Accessing data file objects may also include providing a direct file object interface on the client, where the direct file object interface allows an application to directly access file objects at the same time that file objects are being accessed using the Web Services interface and using the file system interface.

Abstract: Accessing data file objects includes providing a file system interface on a client, where the file system interface allows an application running on the client to make file system calls to access the data file objects. A Web Services interface may also be provided on the client, where the Web Services interface allows an application to access file objects using the Web Services at the same time that file objects are being accessed through the file system interface. The Web Services interface may be a SOAP interface and/or a REST interface. Accessing data file objects may also include providing a direct file object interface on the client, where the direct file object interface allows an application to directly access file objects at the same time that file objects are being accessed using the Web Services interface and using the file system interface.

Abstract: Tracking storage resources includes providing a table containing storage resources along with capabilities and statuses thereof, updating the table in response to a change of status of a storage resource, updating the table in response to a change in capabilities of a storage resource and, in response to an inquiry for a storage resource having a particular capability, searching the table for a storage resource having the particular capability. Tracking storage resources may also include adding an element to the table in response to a new resource being added to the system. The capabilities may include RAID striping, data deduplication, and green operation. The status may be one of: on-line, off-line, and full.

Abstract: Managing files includes receiving a plurality of policies for a new file, where different policies may be applied according to predetermined conditions, receiving a request to create a new file, selecting an appropriate policy for the new file according to the predetermined conditions, determining if sufficient resources are available according to the appropriate policy, and obtaining file objects to create the file in response to their being sufficient resources according to the appropriate policy. Policies may be received via a user management interface. The predetermined conditions may include an identity of a client creating a file. The appropriate policy may include information regarding mirrors for the file. The appropriate policy may indicate a minimum geographic distance between the mirrors.

Abstract: Managing file objects in a data storage system includes providing a plurality of metadata servers, each metadata server having information for only a subset of file objects in the storage system, each subset for each of the metadata servers being less than a total number of file objects for the system and includes providing at least one metadata location server, where an entity accessing a specific file object of the system determines which of the metadata servers contains data for the specific file object by first accessing the at least one metadata location server. Information for the subset of file objects may include a table having file identifiers and metadata location information. The metadata location information may point to data storage for a metadata object for a file.

Abstract: Accessing data file objects includes providing a file system interface on a client, where the file system interface allows an application running on the client to make file system calls to access the data file objects. A Web Services interface may also be provided on the client, where the Web Services interface allows an application to access file objects using the Web Services at the same time that file objects are being accessed through the file system interface. The Web Services interface may be a SOAP interface and/or a REST interface. Accessing data file objects may also include providing a direct file object interface on the client, where the direct file object interface allows an application to directly access file objects at the same time that file objects are being accessed using the Web Services interface and using the file system interface.

Abstract: Managing a plurality of files includes providing at least one server having a table with plurality of file identifiers, each of the file identifiers including a pointer to a metadata file object containing a layout storage object that indicates storage parameters for corresponding files and includes accessing a particular one of the files by obtaining information from the at least one server to access the layout storage object corresponding to the particular file to determine synchronous and asynchronous mirrors for the file. Each layout storage object may be provided as a tree and leaves of the tree may map logical addresses of a corresponding file to data storage locations. At least one internal node of the tree may indicate that at least one leaf node thereof is a mirror of an other leaf node thereof.