Abstract: The disclosed embodiments disclose techniques for accessing a scale-out block interface in a cloud-based distributed computing environment (CBDCE). During operation, an instance of a block device service (BDS) receives a data request from a client. The BDS instance translates the data request into a set of data block accesses, and then sends the translated data request to a data processing layer service (DPL). The DPL instance services the data request using a set of data operations that leverage one or more of a distributed cache, a distributed database, and a cloud storage system. The BDS service leverages the DPL instance to provide to the client an abstraction of a highly-available block storage device with unlimited storage space via the scale-out block interface.

Abstract: The disclosed embodiments disclose techniques for journaling data received in a cloud-based distributed computing environment (CBDCE). Multiple services simultaneously execute on the CBDCE compute nodes, with each service comprising multiple service instances that simultaneously execute on multiple, distinct compute nodes of the CBDCE. The CBDCE includes a distributed database that enables coordination between the service instances of services that execute in the CBDCE; this distributed database also includes multiple distributed database instances that simultaneously executing on multiple different CBDCE compute nodes. During operation, a service instance executing on one of these compute nodes receives a client request. The service instance submits this client request to a distributed database instance and, in parallel, also submits the client request and its associated user data to a distributed journaling service.

Abstract: The disclosed embodiments disclose techniques for managing a cloud-based distributed computing environment (CBDCE) that comprises multiple geographically-distributed compute nodes. Multiple services simultaneously execute on the CBDCE compute nodes, with each service comprising multiple service instances that simultaneously execute on multiple distinct compute nodes of the CBDCE. During operation, the system uses a distributed database to track the status of the CBDCE to ensure the ongoing stability and scalability of the CBDCE. Upon receiving a request that is associated with the configuration of the CBDCE, a service accesses CBDCE status information from the distributed database to respond to the request.

Abstract: The disclosed embodiments disclose techniques for managing a distributed cache in a cloud-based distributed computing environment (CBDCE). During operation, an instance of a data processing layer service (DPL) receives a data request from a client that specifies an address and an operation for a target data block. The DPL instance uses these to determine a first cache instance of the distributed cache that is assigned to cache a metadata entry that links the address with a data block fingerprint for the target data block. The DPL instance then uses the data block fingerprint and the cache mapping to determine a second cache instance that is assigned to store the target data block, and then accesses the second cache instance to complete the operation for the target data block.

Abstract: The disclosed embodiments disclose techniques for managing cloud-based storage using a time-series database. A distributed cloud data management system (DCDMS) manages objects stored in a cloud storage system. The DCDMS leverages a distributed time-series database to track objects accessed via the DCDMS. During operation, the DCDMS receives a request to access an object using a path identifier and an object identifier. The DCDMS determines from the path identifier that the request is associated with one of its supported extended capabilities, and uses the previously tracked object operations that are stored in the time-series database to determine the actual target bucket in the cloud storage system that contains the requested object; the target bucket that contains the object may be different from the bucket identified in the path identifier that is received. The object identifier is then used to access the requested object from the target bucket to service the request.

Abstract: The disclosed embodiments disclose techniques for accessing a scale-out block interface in a cloud-based distributed computing environment (CBDCE). During operation, an instance of a block device service (BDS) receives a data request from a client. The BDS instance translates the data request into a set of data block accesses, and then sends the translated data request to a data processing layer service (DPL). The DPL instance services the data request using a set of data operations that leverage one or more of a distributed cache, a distributed database, and a cloud storage system. The BDS service leverages the DPL instance to provide to the client an abstraction of a highly-available block storage device with unlimited storage space via the scale-out block interface.

Abstract: The disclosed embodiments disclose techniques for managing a distributed cache in a cloud-based distributed computing environment (CBDCE). During operation, an instance of a data processing layer service (DPL) receives a data request from a client that specifies an address and an operation for a target data block. The DPL instance uses these to determine a first cache instance of the distributed cache that is assigned to cache a metadata entry that links the address with a data block fingerprint for the target data block. The DPL instance then uses the data block fingerprint and the cache mapping to determine a second cache instance that is assigned to store the target data block, and then accesses the second cache instance to complete the operation for the target data block.

Abstract: A method and apparatus for utilizing virtual machines to pool memory from disparate server systems that may have disparate types of memory is described. The method may include establishing communication between a pool virtual machine and two or more publisher virtual machines. The method may also include aggregating, by the pool virtual machine, portions of memory from each of two or more publisher servers to generate a pool of memory, and providing an application with access to the pool of memory, through the pool virtual machine.

Abstract: The disclosed embodiments disclose techniques for managing a cloud-based distributed computing environment (CBDCE) that comprises multiple geographically-distributed compute nodes. Multiple services simultaneously execute on the CBDCE compute nodes, with each service comprising multiple service instances that simultaneously execute on multiple distinct compute nodes of the CBDCE. During operation, the system uses a distributed database to track the status of the CBDCE to ensure the ongoing stability and scalability of the CBDCE. Upon receiving a request that is associated with the configuration of the CBDCE, a service accesses CBDCE status information from the distributed database to respond to the request.

Abstract: The disclosed embodiments disclose techniques for journaling data received in a cloud-based distributed computing environment (CBDCE). Multiple services simultaneously execute on the CBDCE compute nodes, with each service comprising multiple service instances that simultaneously execute on multiple, distinct compute nodes of the CBDCE. The CBDCE includes a distributed database that enables coordination between the service instances of services that execute in the CBDCE; this distributed database also includes multiple distributed database instances that simultaneously executing on multiple different CBDCE compute nodes. During operation, a service instance executing on one of these compute nodes receives a client request. The service instance submits this client request to a distributed database instance and, in parallel, also submits the client request and its associated user data to a distributed journaling service.

Abstract: The disclosed embodiments disclose techniques for managing cloud-based storage using a time-series database. A distributed cloud data management system (DCDMS) manages objects stored in a cloud storage system. The DCDMS leverages a distributed time-series database to track objects accessed via the DCDMS. During operation, the DCDMS receives a request to access an object using a path identifier and an object identifier. The DCDMS determines from the path identifier that the request is associated with one of its supported extended capabilities, and uses the previously tracked object operations that are stored in the time-series database to determine the actual target bucket in the cloud storage system that contains the requested object; the target bucket that contains the object may be different from the bucket identified in the path identifier that is received. The object identifier is then used to access the requested object from the target bucket to service the request.

Abstract: A method and apparatus for utilizing virtual machines to pool memory from disparate server systems that may have disparate types of memory is described. The method may include establishing communication between a pool virtual machine and two or more publisher virtual machines. The method may also include aggregating, by the pool virtual machine, portions of memory from each of two or more publisher servers to generate a pool of memory, and providing an application with access to the pool of memory, through the pool virtual machine.

Abstract: The present disclosure provides for performing virus scans at a storage device that stores one or more virtual machine disk image files (VMDK files). A secure AV module can coordinate communication between a file system on the storage device, a file system (FS) decoder, and an anti-virus engine to perform a virus scan of files contained within a VMDK file. A secure AV module can determine a subset of files that include changed data, where the subset of files is stored in a file system volume within a VMDK file. The secure AV module can use an FS decoder to translate file addresses relative to the file system volume into file addresses relative to the network storage file system. A secure AV module can provide the network storage file system addresses of the subset of files to the anti-virus engine, which can perform a virus scan on the files.

Abstract: Systems and methods for information storage replication are presented. In one embodiment, a namespace conversion process is performed. Node information regarding a file systems operation change is received. A changed node to pathname object conversion process is performed. An unchanged node to pathname object conversion process is performed. In one exemplary implementation, the changed node to pathname object conversion process and the unchanged node to pathname object conversion process utilize data structures that return the object indications and parent node indications. An object indication is inserted in a pathname.

Abstract: A method and apparatus for rapid replication of deduplicated file system data is described. The method may include initiating replication of a file from a source deduplication system to a destination deduplication system, and transferring deduplication metadata for each block of the file from the source deduplication system to the destination deduplication system. The method may also include transferring an identifier file from the source deduplication system to the destination deduplication system that includes a block number corresponding to a block of the file and a unique identifier value generated from the block of the file. The method may also include receiving a data request file from the destination deduplication system, and transferring the blocks of data identified in the data request file to complete replication of the file on the destination deduplication system.

Abstract: Systems and methods for information storage replication are presented. In one embodiment, a namespace conversion process is performed. Node information regarding a file systems operation change is received. A changed node to pathname object conversion process is performed. An unchanged node to pathname object conversion process is performed. In one exemplary implementation, the changed node to pathname object conversion process and the unchanged node to pathname object conversion process utilize data structures that return the object indications and parent node indications. An object indication is inserted in a pathname.

Abstract: A data server platform includes a security file system layer interposed between the platform operating system kernel and file system. The secure file system layer is structured to implement a file access control function that selectively constrains data transfer operations initiated through the operating system kernel by an application program to transfer file data through the file system with respect to a persistent data store. A file access controller, implemented independent of the operating system kernel, is coupled to the security file system layer and supports the file access control function by defining permitted file data transfers through the file system. Management of the file access controller separate from the data server platform ensures that any security breach of the platform operating system kernel cannot compromise the function of the security file system layer.

Abstract: Network data files are secure through the operation of an infrastructure gateway-based network file access appliance. Network file data, corresponding to network pocket payload data, are further reduced to a sequence of data blocks that are secured through any combination of block encryption, compression, and digital signatures. File meta-data, including encryption, compression and block-level digital signatures are persistently stored with the file data, either in-band in the file as stored or out-of-band key as a separately stored file or file policy record. File meta-data is recovered with accesses of the file data to support bidirectional encryption and compression and to detect tampering with the file data by comparison against block-level digital signatures.

Abstract: A data server platform includes a security file system layer interposed between the platform operating system kernel and file system. The secure file system layer is structured to implement a file access control function that selectively constrains data transfer operations initiated through the operating system kernel by an application program to transfer file data through the file system with respect to a persistent data store. A file access controller, implemented independent of the operating system kernel, is coupled to the security file system layer and supports the file access control function by defining permitted file data transfers through the file system. Management of the file access controller separate from the data server platform ensures that any security breach of the platform operating system kernel cannot compromise the function of the security file system layer.

Abstract: A network file access appliance operates as a secure portal for network file access operations between client computer systems and network storage resources. The file access appliance terminates network file access transactions, identified by packet information including client system, mount point, and file request identifiers, between client systems and mount points supported by the access controller. A policy parser determines, based on the packet information, to selectively initiate network file access transactions between the access controller and network storage resources to enable completion of selected network file access transactions directed from the clients to the network file access appliance. The network file access transactions directed to the network storage resources are modified counterparts of policy selected client network file access transactions modified to reference mapped network storage resource mount points and support the secure transfer and storage of network file data.