Abstract: The disclosed embodiments disclose techniques for extracting encryption keys to enable monitoring services. During operation, an encrypted connection is detected on a computing device. A monitoring service harvests an encryption key for this encrypted connection from the memory of a computing device and then forwards the encryption key to an intercepting agent in an intermediate computing environment that intercepts encrypted traffic that is sent between the computing device and a remote service via the encrypted connection.

Abstract: The disclosed embodiments disclose techniques for leveraging instrumentation capabilities to enable monitoring services. During operation, an operating system kernel is instrumented to associate a sub-program with a target operation. Upon receiving a request from an application to perform the target operation, the operating system kernel executes the sub-program with kernel privileges in the process context of the application. The sub-program analyzes the memory space associated with the application to extract a desired data value. This extracted data value is returned to at least one of a specified target process or target location.

Abstract: The disclosed embodiments disclose techniques for extracting encryption keys to enable monitoring services. During operation, an encrypted connection is detected on a computing device. A monitoring service harvests an encryption key for this encrypted connection from the memory of a computing device and then forwards the encryption key to an intercepting agent in an intermediate computing environment that intercepts encrypted traffic that is sent between the computing device and a remote service via the encrypted connection.

Abstract: The disclosed embodiments disclose techniques for leveraging instrumentation capabilities to enable monitoring services. During operation, an operating system kernel is instrumented to associate a sub-program with a target operation. Upon receiving a request from an application to perform the target operation, the operating system kernel executes the sub-program with kernel privileges in the process context of the application. The sub-program analyzes the memory space associated with the application to extract a desired data value. This extracted data value is returned to at least one of a specified target process or target location.

Abstract: The disclosed embodiments disclose techniques for optimizing data transfer costs for cloud-based security services. During operation, an intermediary computing device receives a network request from a client located in a remote enterprise location that is sending the network request to a distinct, untrusted remote site (e.g., a site separate from the distinct locations of the remote enterprise, the cloud data center, and the intermediary computing device). The intermediary computing device caches a set of data associated with the network request while forwarding the set of data to the cloud-based security service for analysis. Upon receiving a confirmation from the cloud-based security service that the set of data has been analyzed and is permitted to be transmitted to the specified destination, the intermediary computing device forwards the cached set of data to the specified destination.

Abstract: The disclosed embodiments disclose techniques for seamlessly updating a cloud-based security service. A dispatcher virtual machine (VM) executing in a cloud data center receives network requests sent from clients located in a remote enterprise location to untrusted remote sites, and routes this network traffic through a chain of security service VMs that analyze the network traffic. During operation, the dispatcher VM determines that an existing security service VM in the chain needs to be upgraded to an updated version, and instantiates an updated chain of security service VMs that includes this updated version. The dispatcher VM then seamlessly transfers the flow of network traffic from the initial chain to the updated chain to seamlessly update the cloud-based security service without interruption. Upon determining that the updated version is operating correctly, the dispatcher VM halts and deallocates the previous version and any other unneeded portions of the initial chain.

Abstract: The disclosed embodiments disclose techniques for providing a cloud-based security service. During operation, a dispatcher virtual machine (VM) executing in a cloud data center receives a network request from a remote enterprise client. The dispatcher VM executes multiple docker containers, including a set of ingress docker containers that decode the request and then forward it to a session router docker container that in turn forwards the request to a set of security service VMs. After these security service VMs have analyzed the contents of the request and determined that the request is valid and permitted, a SNAT docker container then sends the request out to an untrusted network to be serviced.

Abstract: The disclosed embodiments disclose techniques for optimizing data transfer costs for cloud-based security services. During operation, an intermediary computing device receives a network request from a client located in a remote enterprise location that is sending the network request to a distinct, untrusted remote site (e.g., a site separate from the distinct locations of the remote enterprise, the cloud data center, and the intermediary computing device). The intermediary computing device caches a set of data associated with the network request while forwarding the set of data to the cloud-based security service for analysis. Upon receiving a confirmation from the cloud-based security service that the set of data has been analyzed and is permitted to be transmitted to the specified destination, the intermediary computing device forwards the cached set of data to the specified destination.

Abstract: The disclosed embodiments disclose techniques for providing a cloud-based security service. During operation, a dispatcher virtual machine (VM) executing in a cloud data center receives a network request from a remote enterprise client. The dispatcher VM executes multiple docker containers, including a set of ingress docker containers that decode the request and then forward it to a session router docker container that in turn forwards the request to a set of security service VMs. After these security service VMs have analyzed the contents of the request and determined that the request is valid and permitted, a SNAT docker container then sends the request out to an untrusted network to be serviced.

Abstract: The disclosed embodiments disclose techniques for seamlessly updating a cloud-based security service. A dispatcher virtual machine (VM) executing in a cloud data center receives network requests sent from clients located in a remote enterprise location to untrusted remote sites, and routes this network traffic through a chain of security service VMs that analyze the network traffic. During operation, the dispatcher VM determines that an existing security service VM in the chain needs to be upgraded to an updated version, and instantiates an updated chain of security service VMs that includes this updated version. The dispatcher VM then seamlessly transfers the flow of network traffic from the initial chain to the updated chain to seamlessly update the cloud-based security service without interruption. Upon determining that the updated version is operating correctly, the dispatcher VM halts and deallocates the previous version and any other unneeded portions of the initial chain.

Abstract: The disclosed embodiments disclose techniques for transferring and caching a cloud file in a cloud controller. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives a client request for a data block of a target file that is stored in the distributed filesystem but not currently cached in the cloud controller. The cloud controller initiates a request to a cloud storage system for a cloud file containing the requested data block. While receiving the cloud file from the cloud storage system, the cloud controller uses a set of block metadata in the portion of the cloud file that has already been received to determine the portions of the cloud file that should be downloaded to and cached in the cloud controller.

Abstract: The disclosed embodiments disclose techniques that facilitate of avoiding client timeouts in a distributed filesystem. Multiple cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem in a local storage pool. During operation, a cloud controller receives from a client system a request for a data block in a target file that is stored in the distributed filesystem. Although the cloud controller is already caching the requested data block, the cloud controller delays transmission of the cached data block; this additional delay gives the cloud controller more time to access uncached data blocks for the target file from a cloud storage system, thereby ensuring that subsequent requests of such data blocks do not exceed a timeout interval on the client system.

Abstract: The disclosed embodiments provide a system that uses overlay metadata in a cloud controller to generate incremental snapshots for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems. More specifically, the cloud controllers cache and ensure data consistency for the data stored in the cloud storage systems, with each cloud controller maintaining a metadata hierarchy that reflects the current state of the distributed filesystem. During operation, a cloud controller receiving new data from a client: (1) stores the new data in the cloud controller; (2) creates a metadata entry for the new data in the locally maintained metadata hierarchy; (3) updates the overlay metadata to point to the metadata entry and the new data stored in the cloud controller; and (4) then uses the overlay metadata to generate an incremental snapshot for the new data.

Abstract: The disclosed embodiments disclose techniques that facilitate the process of performing anti-virus checks for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a write request from a client system that seeks to store a target file in the distributed system. A scan is then performed for this target file. For instance, the scan may be an anti-virus scan that ensures that viruses are not spread to the distributed filesystem or the clients of the distributed filesystem.

Abstract: The disclosed embodiments disclose techniques for executing a cloud command for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller presents a distributed-filesystem-specific capability to a client system as a file in the distributed filesystem (e.g., using a file abstraction). Upon receiving a request from the client system to access and/or operate upon this file, the client controller executes an associated cloud command. More specifically, the cloud controller initiates a specially-defined operation that accesses additional functionality for the distributed filesystem that exceeds the scope of individual reads and writes to a typical data file.

Abstract: The disclosed embodiments disclose techniques for managing metadata and data storage for a cloud controller in a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems. More specifically, the cloud controllers cache and ensure data consistency for the data stored in the cloud storage systems, with each cloud controller maintaining (e.g., storing) in a local storage device: (1) one or more metadata regions containing a metadata hierarchy that reflects the current state of the distributed filesystem; and (2) cached data for the distributed filesystem. During operation, the cloud controller receives an incremental metadata snapshot that references new data written to the distributed filesystem. The cloud controller stores updated metadata from this incremental metadata snapshot in one of the metadata regions on the local storage device.

Abstract: The disclosed embodiments provide a system that adjusts the characteristics of a distributed filesystem using a locality policy. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives a locality policy that specifies one or more management policies for the cloud controller. The portion of the distributed filesystem's data that is managed, created, and/or cached at the cloud controller is then managed based on this locality policy. Locality policies facilitate customizing and optimizing data management for the distributed filesystem to fit the needs of an organization (e.g., specific sets of users, applications, and/or datasets).

Abstract: The disclosed embodiments provide techniques for performing deduplication for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives an incremental metadata snapshot that references new data that was added to the distributed filesystem by a remote cloud controller. The cloud controller extracts a set of deduplication information from this incremental metadata snapshot. Upon receiving a subsequent client write request (e.g., a file write that includes one or more data blocks), the cloud controller uses the extracted deduplication information to determine that one or more data blocks in the client write request have already been written to the distributed filesystem.

Abstract: The disclosed embodiments disclose techniques for deleting a file from a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers store metadata for the distributed filesystem, and cache and ensure data consistency for the data stored in the cloud storage systems. During operation, a cloud controller receives a request from a client to delete a file from the distributed filesystem. The cloud controller updates a user view of the distributed filesystem to present the appearance of the target file being deleted to the client, and then initiates a background deletion operation to delete the target file without negatively affecting the performance of the other users of the distributed filesystem.

Abstract: The disclosed embodiments disclose techniques that facilitate the recovery of a virtual machine using a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem in a local storage pool. During operation, a host server executes program instructions for an application in a virtual machine (VM); data associated with this application and/or this virtual machine is stored in the distributed filesystem. Upon detecting a subsequent failure, the system can recover and resume the execution of the virtual machine and application using the previous application and virtual machine data that was stored in the distributed filesystem.