Abstract: A data block storage management capability is presented. A cloud file system management capability manages storage of data blocks of a file system across multiple cloud storage services (e.g., including determining, for each data block to be stored, a storage location and a storage duration for the data block). A cloud file system management capability manages movement of data blocks of a file system between storage volumes of cloud storage services. A cloud file system management capability provides a probabilistic eviction scheme for evicting data blocks from storage volumes of cloud storage services in advance of storage deadlines by which the data blocks are to be removed from the storage volumes. A cloud file system management capability enables dynamic adaptation of the storage volume sizes of the storage volumes of the cloud storage services.

Abstract: A data block storage management capability is presented. A cloud file system management capability manages storage of data blocks of a file system across multiple cloud storage services (e.g., including determining, for each data block to be stored, a storage location and a storage duration for the data block). A cloud file system management capability manages movement of data blocks of a file system between storage volumes of cloud storage services. A cloud file system management capability provides a probabilistic eviction scheme for evicting data blocks from storage volumes of cloud storage services in advance of storage deadlines by which the data blocks are to be removed from the storage volumes. A cloud file system management capability enables dynamic adaptation of the storage volume sizes of the storage volumes of the cloud storage services.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a set of partitions containing a set of records, wherein the records include a set of values for a measure and a set of dimensions associated with the values. Next, the system reorganizes the records across the partitions by performing a distributed sort of the records by the measure. For each dimensional subset in the records, the system counts occurrences of the dimensional subset in each of the partitions and groups values of the counted occurrences by the dimensional subset so that the values reside in a single processing node. The system uses the values to identify one or more locations in the partitions for calculating a statistic for the dimensional subset and uses the location(s) to calculate the statistic. Finally, the system outputs the statistic in response to a query containing the dimensional subset.

Abstract: A capability is provided for reducing the costs of data transfers for networks using pricing heterogeneity. For data to be transferred between an application of a first network and a user device, the data may be transferred directly without using a second network or indirectly using a second network including a transfer of the data between the first network and the second network. In a network-based file system, a first network is primarily used for write requests and a second network is primarily used for read requests, and a data block is transferred from the first network to the second network based on a determination that a threshold number of read requests for the data block have been received. The determination as to when to transfer data between networks may be determined based on various costs associated with the networks.

Abstract: Various embodiments provide a method and apparatus of providing a network and storage-aware virtual machine (VM) placement that optimizes placement based on network layer metrics, performance characteristics of the storage arrays and application constraints. Advantageously, since storage is often necessary in servicing application requests, basing VM placement on performance characteristics of the storage arrays as well as network layer metrics can lead to a significant improvement in VM performance.

Abstract: A capability is provided for reducing the costs of data transfers for networks using pricing heterogeneity. For data to be transferred between an application of a first network and a user device, the data may be transferred directly without using a second network or indirectly using a second network including a transfer of the data between the first network and the second network. In a network-based file system, a first network is primarily used for write requests and a second network is primarily used for read requests, and a data block is transferred from the first network to the second network based on a determination that a threshold number of read requests for the data block have been received. The determination as to when to transfer data between networks may be determined based on various costs associated with the networks.

Abstract: A security management capability enables migration of individual security rules between storage/application locations. The migration of a security rule may include selection of a location at which the security rule is to be applied and migration of the security rule to the selected location at which the security rule is to be applied. The selection of the location at which the security rule is to be applied may be performed based on security rule policies and/or security rule location selection information. The security rule is migrated from a current location (e.g., a location at which the security rule is currently applied, a management system, or the like) to the selected location at which the security rule is to be applied. In this manner, a fluid security layer may be provided. The fluid security layer may be optimized for one or more of security level, performance, cost, or the like.

Abstract: Various embodiments provide a method and apparatus of providing a distributed security service that runs light instances in a number of security devices and central instances of the security services in select security devices. A received or transmitted client content segment is directed to a light instance which either applies a security policy corresponding to the client content segment if the client content segment has been previously analyzed and has a valid security policy, or else, the light instance sends the client content segment to a central instance to be analyzed. The central instance may then provide a complete security analysis on the client content segment, determine a security policy corresponding to the client content segment and push the determined security policy to one or more of the light instances. Advantageously, a distributed security service delivery may provide highly secure, network efficient and cost effective security service delivery.

Abstract: A flow deflection capability is provided for deflecting data flows within a Software Defined Network (SDN) in order to provide security for the SDN. A flow forwarding rule is generated for a first network element of the SDN based on detection of a condition (e.g., TCAM utilization condition, CPU utilization condition, or the like) associated with the first network element. The flow forwarding rule is generated by a control element of the SDN or the first network element of the SDN. The flow forwarding rule is indicative that at least a portion of new flow requests received at the first network element are to be forwarded from the first network element to a second network element of the SDN. The flow forwarding rule may specify full flow deflection or selective flow deflection.

Abstract: A system, method, and apparatus are provided for organizing and joining massive sets of data (e.g., tens or hundreds of millions of event records). A dataset is Blocked by first identifying a partition key, which comprises one or more columns of the data. Each Block will contain all dataset records that have partition key values assigned to that Block. A cost constraint (e.g., a maximum size, a maximum number of records) may also be applied to the Blocks. A Block index is generated to identify all Blocks, their corresponding (sequential) partition key values, and their locations. A second dataset that includes the partition key column(s) and that must be correlated with the first dataset may then be Blocked according to the same ranges of partition key values (but without the cost constraint). Corresponding Blocks of the datasets may then be Joined/Aggregated, and analyzed as necessary.

Abstract: A flow deflection capability is provided for deflecting data flows within a Software Defined Network (SDN) in order to provide security for the SDN. A flow forwarding rule is generated for a first network element of the SDN based on detection of a condition (e.g., TCAM utilization condition, CPU utilization condition, or the like) associated with the first network element. The flow forwarding rule is generated by a control element of the SDN or the first network element of the SDN. The flow forwarding rule is indicative that at least a portion of new flow requests received at the first network element are to be forwarded from the first network element to a second network element of the SDN. The flow forwarding rule may specify full flow deflection or selective flow deflection.

Abstract: A capability is provided for reducing the costs of data transfers for networks using pricing heterogeneity. For data to be transferred between an application of a first network and a user device, the data may be transferred directly without using a second network or indirectly using a second network including a transfer of the data between the first network and the second network. In a network-based file system, a first network is primarily used for write requests and a second network is primarily used for read requests, and a data block is transferred from the first network to the second network based on a determination that a threshold number of read requests for the data block have been received. The determination as to when to transfer data between networks may be determined based on various costs associated with the networks.

Abstract: A multi-tenancy storage node is provided. The storage node hosts partitions of multiple databases accessed by multiple applications. Each database is governed by a corresponding service-level agreement (SLA) or policy that specifies a maximum load or level of operation of the database in terms of one or more metrics (e.g., number or frequency of reads/writes, maximum size of reads/writes). To determine whether another database can be hosted by the node, a snapshot of the node's database operations is replayed on a test node, along with operations that exercise the other database. If maximum thresholds of the node for the metrics are not exceeded, the other database can be added to the storage node. An SLA is generated for it automatically, based on the metrics it exhibited during the operations on the test node. The storage node may enforce the SLA by rate-limiting activity of one or more applications.

Abstract: Various embodiments provide a method and apparatus of providing an RE-aware technique for placing slots based on redundancy across and within slot communication pairs.

Abstract: A system, method, and apparatus are provided for organizing and joining massive sets of data (e.g., tens or hundreds of millions of event records). A dataset is Blocked by first identifying a partition key, which comprises one or more columns of the data. Each Block will contain all dataset records that have partition key values assigned to that Block. A cost constraint (e.g., a maximum size, a maximum number of records) may also be applied to the Blocks. A Block index is generated to identify all Blocks, their corresponding (sequential) partition key values, and their locations. A second dataset that includes the partition key column(s) and that must be correlated with the first dataset may then be Blocked according to the same ranges of partition key values (but without the cost constraint). Corresponding Blocks of the datasets may then be Joined/Aggregated, and analyzed as necessary.

Abstract: A method and apparatus of providing an RE-aware technique for placing slots based on redundancy across and within slot communication pairs. In particular, the RE-aware placement strategy takes into account the redundancy in data transfers for slot-to-slot communications and place slots to exploit redundancy in data transfers while minimizing the overall inter-rack (or inter-data center) bandwidth usage.

Abstract: A data block storage management capability is presented. A file system includes a plurality of data blocks which are managed using a first storage service and a second storage service, where the first storage service has a lower storage cost and a higher input-output cost than the second storage service. The data blocks stored using the second storage service have associated therewith respective expected storage durations indicative of respective lengths of time for which the data blocks are to be stored using the second storage service (which may be the same or different across the ones of the data blocks stored using the second storage service). The expected storage durations of the data blocks are modified based on a comparison of an expected hit rate of the second storage service and a current hit rate of the second storage service or current hit rates of the data blocks.

Abstract: An exemplary data transfer manager includes a datacenter configured to communicate over at least one link and a scheduler that is configured to schedule a plurality of jobs for communicating data from the datacenter. The scheduler determines a minimum bandwidth requirement of each job and determines a maximum bandwidth limit of each job. The scheduler determines a flex parameter of each job. The flex parameter indicates how much a data transfer rate can vary between adjacent data transfer periods for the job.

Abstract: Various embodiments provide a method and apparatus of providing accelerated encrypted connections in a cloud network supporting transmission of data including per-user encrypted data. Transmission of encrypted data from an application server uses an encryption scheme that encrypts static data using a first encryption scheme that derives keys from the content itself and encrypts dynamic data, such as dynamic website content with personalized user data, using a second encryption scheme.

Abstract: A data block storage management capability is presented. A file system includes a plurality of data blocks which are managed using a first storage service and a second storage service, where the first storage service has a lower storage cost and a higher input-output cost than the second storage service. The data blocks stored using the second storage service have associated therewith respective expected storage durations indicative of respective lengths of time for which the data blocks are to be stored using the second storage service (which may be the same or different across the ones of the data blocks stored using the second storage service). The expected storage durations of the data blocks are modified based on a comparison of an expected hit rate of the second storage service and a current hit rate of the second storage service or current hit rates of the data blocks.