Abstract: A cloud-based data protection service is disclosed. In an embodiment, the data protection service may support backup of data sets from one or more sites associated with one or more organizations. In an embodiment, deduplication of backup data across multiple sites of an organization and/or multiple sites associated with different organizations may be supported. In an embodiment, backup data may be post-processed in the cloud to insert fingerprints corresponding to data blocks that did not change since a previous backup was performed, to scan the backup for security threats such as viruses, other malware, personally identifiable information, etc. In an embodiment, restore may be supported from the cloud, where restore blocks may be larger than backup data blocks. In another embodiment, restore may be based on blocks that have changed since the most recent backup (or a user-selected backup).

Abstract: A cloud-based data protection service is disclosed. In an embodiment, the data protection service may support backup of data sets from one or more sites associated with one or more organizations. In an embodiment, deduplication of backup data across multiple sites of an organization and/or multiple sites associated with different organizations may be supported. In an embodiment, backup data may be post-processed in the cloud to insert fingerprints corresponding to data blocks that did not change since a previous backup was performed, to scan the backup for security threats such as viruses, other malware, personally identifiable information, etc. In an embodiment, restore may be supported from the cloud, where restore blocks may be larger than backup data blocks. In another embodiment, restore may be based on blocks that have changed since the most recent backup (or a user-selected backup).

Abstract: A cloud-based data protection service is disclosed. In an embodiment, the data protection service may support backup of data sets from one or more sites associated with one or more organizations. In an embodiment, deduplication of backup data across multiple sites of an organization and/or multiple sites associated with different organizations may be supported. In an embodiment, backup data may be post-processed in the cloud to insert fingerprints corresponding to data blocks that did not change since a previous backup was performed, to scan the backup for security threats such as viruses, other malware, personally identifiable information, etc. In an embodiment, restore may be supported from the cloud, where restore blocks may be larger than backup data blocks. In another embodiment, restore may be based on blocks that have changed since the most recent backup (or a user-selected backup).

Abstract: A cloud-based data protection service is disclosed. In an embodiment, the data protection service may support backup of data sets from one or more sites associated with one or more organizations. In an embodiment, deduplication of backup data across multiple sites of an organization and/or multiple sites associated with different organizations may be supported. In an embodiment, backup data may be post-processed in the cloud to insert fingerprints corresponding to data blocks that did not change since a previous backup was performed, to scan the backup for security threats such as viruses, other malware, personally identifiable information, etc. In an embodiment, restore may be supported from the cloud, where restore blocks may be larger than backup data blocks. In another embodiment, restore may be based on blocks that have changed since the most recent backup (or a user-selected backup).

Abstract: A cloud-based data protection service is disclosed. In an embodiment, the data protection service may support backup of data sets from one or more sites associated with one or more organizations. In an embodiment, deduplication of backup data across multiple sites of an organization and/or multiple sites associated with different organizations may be supported. In an embodiment, backup data may be post-processed in the cloud to insert fingerprints corresponding to data blocks that did not change since a previous backup was performed, to scan the backup for security threats such as viruses, other malware, personally identifiable information, etc. In an embodiment, restore may be supported from the cloud, where restore blocks may be larger than backup data blocks. In another embodiment, restore may be based on blocks that have changed since the most recent backup (or a user-selected backup).

Abstract: A cloud-based data protection service is disclosed. In an embodiment, the data protection service may support backup of data sets from one or more sites associated with one or more organizations. In an embodiment, deduplication of backup data across multiple sites of an organization and/or multiple sites associated with different organizations may be supported. In an embodiment, backup data may be post-processed in the cloud to insert fingerprints corresponding to data blocks that did not change since a previous backup was performed, to scan the backup for security threats such as viruses, other malware, personally identifiable information, etc. In an embodiment, restore may be supported from the cloud, where restore blocks may be larger than backup data blocks. In another embodiment, restore may be based on blocks that have changed since the most recent backup (or a user-selected backup).

Abstract: A cloud-based data protection service is disclosed. In an embodiment, the data protection service may support backup of data sets from one or more sites associated with one or more organizations. In an embodiment, deduplication of backup data across multiple sites of an organization and/or multiple sites associated with different organizations may be supported. In an embodiment, backup data may be post-processed in the cloud to insert fingerprints corresponding to data blocks that did not change since a previous backup was performed, to scan the backup for security threats such as viruses, other malware, personally identifiable information, etc. In an embodiment, restore may be supported from the cloud, where restore blocks may be larger than backup data blocks. In another embodiment, restore may be based on blocks that have changed since the most recent backup (or a user-selected backup).

Abstract: A cloud-based data protection service is disclosed. In an embodiment, the data protection service may support backup of data sets from one or more sites associated with one or more organizations. In an embodiment, deduplication of backup data across multiple sites of an organization and/or multiple sites associated with different organizations may be supported. In an embodiment, backup data may be post-processed in the cloud to insert fingerprints corresponding to data blocks that did not change since a previous backup was performed, to scan the backup for security threats such as viruses, other malware, personally identifiable information, etc. In an embodiment, restore may be supported from the cloud, where restore blocks may be larger than backup data blocks. In another embodiment, restore may be based on blocks that have changed since the most recent backup (or a user-selected backup).

Abstract: Techniques are provided for using an intermediate cache to provide some of the items involved in a scan operation, while other items involved in the scan operation are provided from primary storage. Techniques are also provided for determining whether to service an I/O request for an item with a copy of the item that resides in the intermediate cache based on factors such as a) an identity of the user for whom the I/O request was submitted, b) an identity of a service that submitted the I/O request, c) an indication of a consumer group to which the I/O request maps, d) whether the I/O request is associated with an offloaded filter provided by the database server to the storage system, or e) whether the intermediate cache is overloaded. Techniques are also provided for determining whether to store items in an intermediate cache in response to the items being retrieved, based on logical characteristics associated with the requests that retrieve the items.

Abstract: When a virtual machine sends a write command to a host operating system, a cache system stores the record included in the write command and later sends the write command to a storage system. Systems and methods store an indication of when the write command was received along with the received records in the cache. The cache system determines an order of the records from the indication. In some instances, records received at the same time are serialized and/or assigned record numbers. The simultaneously received records can be assigned a batch number. According to the determined order, the records are retrieved from the cache memory and included in write commands sent to the storage system. In some instances, checkpoints can be stored to prevent records from being re-sent to the storage system following a failure.

Abstract: A cache manager maintains coherency of cached objects used by applications in a computing system. The cache manager maintains a cache memory of objects as they are accessed by an application. The cache manager tracks cached objects on a per-application basis using a cache database, storing for each object metadata comprising the generation number of the application using the cached object, and the file generation number of the file from which the object was read. The cache manager determines whether cached objects are valid and can be used by the application, or are stale and must be reloaded, by using these generation numbers when an application attempts to use a file or object. In this fashion, cache coherency is maintained even in a distributed environment without the centralized cache management bottlenecks of prior approaches and without the need to eagerly purge the cache of stale versions of objects.

Abstract: Systems and methods for managing records stored in a storage cache are provided. A cache index is created and maintained to track where records are stored in buckets in the storage cache. The cache index maps the memory locations of the cached records to the buckets in the cache storage and can be quickly traversed by a metadata manager to determine whether a requested record can be retrieved from the cache storage. Bucket addresses stored in the cache index include a generation number of the bucket that is used to determine whether the cached record is stale. The generation number allows a bucket manager to evict buckets in the cache without having to update the bucket addresses stored in the cache index. Further, the cache index can be expanded to accommodate very small records, such as those generated by legacy systems.

Abstract: Techniques are provided for using an intermediate cache to provide some of the items involved in a scan operation, while other items involved in the scan operation are provided from primary storage. Techniques are also provided for determining whether to service an I/O request for an item with a copy of the item that resides in the intermediate cache based on factors such as a) an identity of the user for whom the I/O request was submitted, b) an identity of a service that submitted the I/O request, c) an indication of a consumer group to which the I/O request maps, or d) whether the intermediate cache is overloaded. Techniques are also provided for determining whether to store items in an intermediate cache in response to the items being retrieved, based on logical characteristics associated with the requests that retrieve the items.

Abstract: Processing resources at a storage system for a database server are utilized to perform aspects of a join operation that would conventionally be performed by the database server. When requesting a range of data units from a storage system, the database server includes join metadata describing aspects of the join operation for which the data is being requested. The join metadata may be, for instance, a bloom filter. The storage system reads the requested data from disk as normal. However, prior to sending the requested data back to the storage system, the storage system analyzes the raw data based on the join metadata, removing a certain amount of data that is guaranteed to be irrelevant to the join operation. The storage system then returns filtered data to the database server. The database system thereby avoids the unnecessary transfer of certain data between the storage system and the database server.

Abstract: Systems and methods for managing records stored in a storage cache are provided. A cache index is created and maintained to track where records are stored in buckets in the storage cache. The cache index maps the memory locations of the cached records to the buckets in the cache storage and can be quickly traversed by a metadata manager to determine whether a requested record can be retrieved from the cache storage. Bucket addresses stored in the cache index include a generation number of the bucket that is used to determine whether the cached record is stale. The generation number allows a bucket manager to evict buckets in the cache without having to update the bucket addresses stored in the cache index. Further, the cache index can be expanded to accommodate very small records, such as those generated by legacy systems.

Abstract: When a virtual machine sends a write command to a host operating system, a cache system stores the record included in the write command and later sends the write command to a storage system. Systems and methods store an indication of when the write command was received along with the received records in the cache. The cache system determines an order of the records from the indication. In some instances, records received at the same time are serialized and/or assigned record numbers. The simultaneously received records can be assigned a batch number. According to the determined order, the records are retrieved from the cache memory and included in write commands sent to the storage system. In some instances, checkpoints can be stored to prevent records from being re-sent to the storage system following a failure.

Abstract: A cache manager maintains coherency of cached objects used by applications in a computing system. The cache manager maintains a cache memory of objects as they are accessed by an application. The cache manager tracks cached objects on a per-application basis using a cache database, storing for each object metadata comprising the generation number of the application using the cached object, and the file generation number of the file from which the object was read. The cache manager determines whether cached objects are valid and can be used by the application, or are stale and must be reloaded, by using these generation numbers when an application attempts to use a file or object. In this fashion, cache coherency is maintained even in a distributed environment without the centralized cache management bottlenecks of prior approaches and without the need to eagerly purge the cache of stale versions of objects.

Abstract: Processing resources at a storage system for a database server are utilized to perform aspects of a join operation that would conventionally be performed by the database server. When requesting a range of data units from a storage system, the database server includes join metadata describing aspects of the join operation for which the data is being requested. The join metadata may be, for instance, a bloom filter. The storage system reads the requested data from disk as normal. However, prior to sending the requested data back to the storage system, the storage system analyzes the raw data based on the join metadata, removing a certain amount of data that is guaranteed to be irrelevant to the join operation. The storage system then returns filtered data to the database server. The database system thereby avoids the unnecessary transfer of certain data between the storage system and the database server.

Abstract: Processing resources at a storage system for a database server are utilized to perform aspects of a join operation that would conventionally be performed by the database server. When requesting a range of data units from a storage system, the database server includes join metadata describing aspects of the join operation for which the data is being requested. The join metadata may be, for instance, a bloom filter. The storage system reads the requested data from disk as normal. However, prior to sending the requested data back to the storage system, the storage system analyzes the raw data based on the join metadata, removing a certain amount of data that is guaranteed to be irrelevant to the join operation. The storage system then returns filtered data to the database server. The database system thereby avoids the unnecessary transfer of certain data between the storage system and the database server.

Abstract: Allocation of memory is optimized across multiple pools of memory, based on minimizing the time it takes to successfully retrieve a given data item from each of the multiple pools. First data is generated that indicates a hit rate per pool size for each of multiple memory pools. In an embodiment, the generating step includes continuously monitoring attempts to access, or retrieve a data item from, each of the memory pools. The first data is converted to second data that accounts for a cost of a miss with respect to each of the memory pools. In an embodiment, the second data accounts for the cost of a miss in terms of time. How much of the memory to allocate to each of the memory pools is determined, based on the second data. In an embodiment, the steps of converting and determining are automatically performed, on a periodic basis.