Abstract: Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.

Abstract: Systems and methods for scheduling storage management tasks over predicted user tasks in a distributed storage system. A method commences upon receiving a set of historical stimulus records that characterize management tasks that are run in the storage system. A corresponding set of historical response records comprising system metrics associated with execution of the system tasks is also received. A learning model is formed from the stimulus records and the response records and formatted to be used as a predictor. A set of forecasted user tasks is input as new stimulus records to the predictor to determine a set of forecasted system metrics that would result from running the forecasted user tasks. Management tasks are selected so as not to impact the forecasted user tasks. Management tasks can be selected based on non-contentions resource usage between historical management task resource usage and predictions of resource usage by the user tasks.

Abstract: Systems and methods for scheduling storage management tasks over predicted user tasks in a distributed storage system. A method commences upon receiving a set of historical stimulus records that characterize management tasks that are run in the storage system. A corresponding set of historical response records comprising system metrics associated with execution of the system tasks is also received. A learning model is formed from the stimulus records and the response records and formatted to be used as a predictor. A set of forecasted user tasks is input as new stimulus records to the predictor to determine a set of forecasted system metrics that would result from running the forecasted user tasks. Management tasks are selected so as not to impact the forecasted user tasks. Management tasks can be selected based on non-contentions resource usage between historical management task resource usage and predictions of resource usage by the user tasks.

Abstract: Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.

Abstract: Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.

Abstract: Dynamic erasure coding for computing and data storage systems. A method embodiment commences upon accessing a set of fault tolerance policy attributes associated with the computing and data storage system. The topology of the system is analyzed to form mappings between the computing nodes of the system and the availability domains of the system. Based on the fault tolerance policy attributes, the topology, and the generated mapping, a plurality of feasible erasure coding configurations are generated. The feasible erasure coding configurations are scored. One or more high-scoring feasible erasure coding configurations are selected and deployed to the computing and data storage system. The method is repeated when there is a change in the fault tolerance policy attributes or in the topology. Depending on the topology and/or the nature of a change in the topology, more than one erasure coding configurations can be deployed onto the computing and data storage system.

Abstract: Systems for high-performance computing. A method operates in a distributed storage cluster platform that has a storage pool and computing nodes that concurrently execute foreground tasks and background tasks. A uses interacts with a user interface to input specifications of background task time windows. Background tasks that run within the time frame of a background task time window are permitted to be scheduled at a relatively higher resource usage rate that consumes relatively higher cluster resources than do background task tasks that run outside of the background task time window. When the background task time window closes, the relatively higher resource usage rate of the running cluster background tasks is reduced to a relatively lower resource usage rate. Background tasks can self-observe the background task time windows and/or can be controlled by messages received from a virtualized controller that is designated to perform cluster-wide observations and to make cluster-wide determinations.

Abstract: Systems for ad-hoc analysis of metadata in distributed data storage systems. A distributed storage system comprises computing nodes and a storage pool that is accessible by computing nodes. The storage pool comprises stored information and respective metadata that describes the stored information. Instances of a metadata search engine are installed on the computing nodes such that the metadata search engines have access to both local data stored in the storage pool as well as to networked storage in the storage pool. A user defines metadata management application extensions for extending the metadata search engine using computer programming languages. When executed by the metadata search engine, the extensions perform user-defined functions. A metadata analysis command is associated with the user-defined function and the metadata analysis command is launched from within the metadata search engine to perform the user-defined function over metadata stored in the system.

Abstract: Dynamic erasure coding for computing and data storage systems. A method embodiment commences upon accessing a set of fault tolerance policy attributes associated with the computing and data storage system. The topology of the system is analyzed to form mappings between the computing nodes of the system and the availability domains of the system. Based on the fault tolerance policy attributes, the topology, and the generated mapping, a plurality of feasible erasure coding configurations are generated. The feasible erasure coding configurations are scored. One or more high-scoring feasible erasure coding configurations are selected and deployed to the computing and data storage system. The method is repeated when there is a change in the fault tolerance policy attributes or in the topology. Depending on the topology and/or the nature of a change in the topology, more than one erasure coding configurations can be deployed onto the computing and data storage system.

Abstract: Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.

Abstract: Systems and methods for scheduling storage management tasks over predicted user tasks in a distributed storage system. A method commences upon receiving a set of historical stimulus records that characterize management tasks that are run in the storage system. A corresponding set of historical response records comprising system metrics associated with execution of the system tasks is also received. A learning model is formed from the stimulus records and the response records and formatted to be used as a predictor. A set of forecasted user tasks is input as new stimulus records to the predictor to determine a set of forecasted system metrics that would result from running the forecasted user tasks. Management tasks are selected so as not to impact the forecasted user tasks. Management tasks can be selected based on non-contentions resource usage between historical management task resource usage and predictions of resource usage by the user tasks.

Abstract: Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.

Abstract: Systems and methods for scheduling storage management tasks over predicted user tasks in a distributed storage system. A method commences upon receiving a set of historical stimulus records that characterize management tasks that are run in the storage system. A corresponding set of historical response records comprising system metrics associated with execution of the system tasks is also received. A learning model is formed from the stimulus records and the response records and formatted to be used as a predictor. A set of forecasted user tasks is input as new stimulus records to the predictor to determine a set of forecasted system metrics that would result from running the forecasted user tasks. Management tasks are selected so as not to impact the forecasted user tasks. Management tasks can be selected based on non-contentions resource usage between historical management task resource usage and predictions of resource usage by the user tasks.

Abstract: Systems for self-configuring health monitoring instrumentation for clustered storage platforms. Master and slave health modules implement a health monitoring system in a clustered virtualization environment comprising a plurality of nodes of the cluster with an installed health module instance running on the nodes. The health module system may gather and analyze data on a node level and at a cluster level to manage the cluster. The cluster health module system observes I/O commands issued to, and I/O command responses returned from, a common storage pool. Health data is stored in the storage pool.

Abstract: A method for providing real time replication status for a networked virtualization environment for storage management, includes scanning metadata to identify replication status for all virtual disks (vDisks) in the networked virtualization environment, generating replication tasks for vDisks that are identified as under replicated based on the scan, performing the replication tasks, monitoring the progress of the replication tasks and determining the real time replication status of the networked virtualization environment based on the scanned metadata and the monitored progress of the replication tasks.

Abstract: A method for providing real time replication status for a networked virtualization environment for storage management, includes scanning metadata to identify replication status for all virtual disks (vDisks) in the networked virtualization environment, generating replication tasks for vDisks that are identified as under replicated based on the scan, performing the replication tasks, monitoring the progress of the replication tasks and determining the real time replication status of the networked virtualization environment based on the scanned metadata and the monitored progress of the replication tasks.

Abstract: A method for providing real time replication status for a networked virtualization environment for storage management, includes scanning metadata to identify replication status for all virtual disks (vDisks) in the networked virtualization environment, generating replication tasks for vDisks that are identified as under replicated based on the scan, performing the replication tasks, monitoring the progress of the replication tasks and determining the real time replication status of the networked virtualization environment based on the scanned metadata and the monitored progress of the replication tasks.

Abstract: A method for providing real time replication status for a networked virtualization environment for storage management, includes scanning metadata to identify replication status for all virtual disks (vDisks) in the networked virtualization environment, generating replication tasks for vDisks that are identified as under replicated based on the scan, performing the replication tasks, monitoring the progress of the replication tasks and determining the real time replication status of the networked virtualization environment based on the scanned metadata and the monitored progress of the replication tasks.

Abstract: A net traffic optimization method provides for clients and servers to communicate with each other over a busy network. Each client process will make an Oracle database service request that is recognized by a listener process in the servers. As each database instance comes up, it generates a globally unique instance identifier (“UII”). Each UII remains valid as long as the database instance that generated it is running. The same UII identifier is given to every client needing a connection to that particular instance. Client processes can simultaneously or sequentially connect to database instances, and must never see the same UII being used to identify two different instances.

Abstract: A net traffic optimization method provides for clients and servers to communicate with each other over a busy network. Each client process will make an Oracle database service request that is recognized by a listener process in the servers. As each database instance comes up, it generates a globally unique instance identifier (“UII”). Each UII remains valid as long as the database instance that generated it is running. The same UII identifier is given to every client needing a connection to that particular instance. Client processes can simultaneously or sequentially connect to database instances, and must never see the same UII being used to identify two different instances.