Abstract: Some embodiments provide systems and methods for sandboxing content optimization to occur entirely within a network edge or PoP of a CDN. Some embodiments pass a first request for a first URL to a first back-end at the network edge that is configured to cache an optimized instance of the particular object. When the optimized instance of the particular object is not cached at the first back-end, a second request is issued for a second URL identifying a non-optimized instance of the particular object. The second request resolves internally within the network edge to a second back-end that is configured to cache the non-optimized object. The non-optimized object from the second back-end is optimized and passed to the first back-end. The first back-end caches the optimized instance of the non-optimized object and serves the optimized instance to a requesting end user.

Abstract: Some embodiments set forth a control message header rewriting methodology. Incoming packets are inspected to identify control messages. Each control message is then inspected to determine whether it originates from a client engaged in a session with a server or from an intermediary node along the path connecting the client and the server. The determination is predicated on a comparison of the addressing provided in the control message header and the addressing provided in the offending packet header, wherein the offending packet is the packet that triggers the intermediary node to issue the control message. If the addressing differs, the header addressing of control message is rewritten using the header addressing of the offending packet. Otherwise, a session table lookup is performed to identify which session the control message is directed to based in part on a hash of the control message header addressing.

Abstract: Some embodiments provide a proprietary 64-bit consistent distribution scheme that preserves the efficiencies of CARP while providing a significantly more balanced distribution of requests that is on par with schemes reliant on computationally expensive cryptographic hashes. The scheme performs hashing of requested URLs and identifiers of available servers over a 64-bit space while optimizing the hashing to remove computationally expensive operations. Some embodiments provide a variant of the scheme to provide a differentiated distribution on the basis of one or more differentiating factors. A first variant utilizes load factor values to adjust the resulting hashes and to produce a first distribution of differentiated content that varies from a second distribution of undifferentiated content.

Abstract: Some embodiments provide a director agent, a server agent, and a specialized hand-off protocol for improving scalability and resource usage within a server farm. A first network connection is established between a client and the director agent in order to receive a content request from the client from which to select a server from a set of servers that is responsible for hosting the requested content. A second network connection is established between the server agent that is associated with the selected server and a protocol stack of the selected server. The first network connection is handed-off to the server agent using the specialized hand-off protocol. The server agent performs network connection state parameter transformations between the two connections to create a network connection through which content can be passed from the selected server to the client without passing through the director.

Abstract: Some embodiments set forth systems and methods enabling a first network to use the resources of various second networks in order to localize delivery of the first network content from the various second networks in a secure manner. Some embodiments provide a token-based authentication scheme to ensure that any configured content access restrictions are effectuated at the first network and any of the second networks providing localized content delivery for the first network. The scheme involves a two phase user authentication, wherein the user is separately authenticated at the first network and the redirected to second network using either the same or different set of access restrictions. The first network exchanges a first encryption key with content providers for encrypting/decrypting the first access restriction and a second encryption key with a second network for encrypting/decrypting the second access restriction.

Abstract: Some embodiments set forth probability based caching, whereby a probability value determines in part whether content identified by an incoming request should be cached or not. Some embodiments further set forth probability based eviction, whereby a probability value determines in part whether cached content should be evicted from the cache. Selection of the content for possible eviction can be based on recency and/or frequency of the content being requested. The probability values can be configured manually or automatically. Automatic configuration involves using a function to compute the probability values. In such scenarios, the probability values can be computed as a function of any of fairness, cost, content size, and content type as some examples.

Abstract: Some embodiments provide firewalls and methods for guarding against attacks by leveraging the Document Object Model (DOM). The firewall renders the DOM tree to produce a white-list rendering of the data which presents the non-executable elements of the data and, potentially, outputs of the executable elements of the data without the executable elements that could be used to carry a security threat. Some embodiments provide control over which nodes of the DOM tree are included in producing the white-list rendering. Specifically, a configuration file is specified to white-list various nodes from the DOM tree and the white-list rendering is produced by including the DOM tree nodes that are specified in the white-list of the configuration file while excluding those nodes that are not in the white-list. Some embodiments provide a hybrid firewall that executes a set of black-list rules over white-listed nodes of the DOM tree.

Abstract: Some embodiments provide a scalable content streaming system that performs server-side archiving. The content streaming system includes a set of streaming server clusters, a remote storage server, and a set of distributed servers. Each streaming server cluster includes at least one streaming server and a local storage server. Each streaming server of a particular cluster distributes a content stream that is uploaded to the streaming server while also recording the content stream to the local storage server of that particular cluster. The remote storage server stores files from the local storage servers that are transferred when recording to those files is complete. The set of distributed servers distribute a live stream of content from a particular streaming server when the content is being uploaded to the particular streaming server and distribute a recorded stream of the content from the remote storage server when the upload is complete.

Abstract: Some embodiments provide systems and methods for determining a server of a distributed hosting system to optimally distribute content to an end user. The method includes identifying an IP address of the end user. Based on the IP address, a set of servers send packets to the end user to derive performance metrics. The performance metrics are used to determine a server from the set of servers that optimally distributes content to the end user. The method modifies a configuration for resolving end user requests such that the optimal server is identified to the end user when the end user requests content from the hosting system. Some embodiments determine the optimal server by providing downloadable content that is embedded with a monitoring tool. The monitoring tool causes the end user to derive performance metrics for the hosting system when downloading a particular object from a set of servers.

Abstract: Some embodiments provide a file system for purging content based on a single traversal of the file system that identifies the directory containing the content without performing a secondary traversal within the directory to target the operation to only the file that are associated with content such that other files contained in the directory are unaffected. The file system supplements traditional directory structures with file-level directories. Each file-level directory is created to contain a root file associated with particular content, different variants of the particular content, and supporting files. Consequently, the file system can complete an operation targeting particular content by performing that operation on the file-level directory containing the particular content, thereby eliminating the need to conduct a file-by-file traversal of the containing directory as a prerequisite to identifying the files associated with the particular content and performing the operation on the files individually.

Abstract: Some embodiments set forth systems and methods enabling a first network to use the resources of various second networks in order to localize delivery of the first network content from the various second networks in a secure manner. Some embodiments provide a token-based authentication scheme to ensure that any configured content access restrictions are effectuated at the first network and any of the second networks providing localized content delivery for the first network. The scheme involves a two phase user authentication, wherein the user is separately authenticated at the first network and the redirected to second network using either the same or different set of access restrictions. The first network exchanges a first encryption key with content providers for encrypting/decrypting the first access restriction and a second encryption key with a second network for encrypting/decrypting the second access restriction.

Abstract: Some embodiments provide a capacity management agent that modifies content requests to adjust bandwidth consumption when streaming requested content from a content provider to a requesting user. The modifications include modifying a URL or header information of the request. The agent performs a process that receives a request for content of a content provider. The process identifies a parameter of the carrier network and modifies the request when the parameter satisfies a threshold. The process passes the request to the content provider and the content provider provides content that consumes a first set of resources in response to an unmodified request and a second set of resources in response to a modified request. When the parameter identifies congestion, the first set of resources is greater than the second set of resources. When the condition parameter identifies underutilization, the first set of resources is less than the second set of resources.

Abstract: Some embodiments set forth a control message header rewriting methodology. Incoming packets are inspected to identify control messages. Each control message is then inspected to determine whether it originates from a client engaged in a session with a server or from an intermediary node along the path connecting the client and the server. The determination is predicated on a comparison of the addressing provided in the control message header and the addressing provided in the offending packet header, wherein the offending packet is the packet that triggers the intermediary node to issue the control message. If the addressing differs, the header addressing of control message is rewritten using the header addressing of the offending packet. Otherwise, a session table lookup is performed to identify which session the control message is directed to based in part on a hash of the control message header addressing.

Abstract: Some embodiments provide systems and methods for implementing discrete mapping for targeted caching in a carrier network. In some embodiments, discrete mapping is implemented using a method that caches content from a content provider to a caching server. The method modifies a DNS entry at a particular DNS server to resolve a request that identifies either a hostname or a domain for the content provider to an address of the caching server so that the requested content is passed from the cached content of the caching server and not the source content provider. In some embodiments, the particular DNS server is a recursive DNS server, a local DNS server of the carrier network, or a DNS server that is not authoritative for the hostname or domain of the content provider.

Abstract: Some embodiments provide a content delivery network (CDN) solution that affords the CDN control over those elements of customer content that are delivered by third parties. The CDN integrates a distributed set of monitoring agents. Each monitoring agent monitors the delivery performance of third parties to the region in which the agent operates. The CDN uses the performance monitoring information to dynamically manage the content tags to the third-party delivered elements of CDN-customer content. Specifically, a CDN server retrieves the parent page for requested CDN-customer content. The CDN server identifies the region from where the request originates and retrieves the logs from the monitoring agents monitoring from that region. The CDN server then modifies the base page by dynamically removing the tags to the third-party delivered elements that are reported in the monitoring agent logs as being unavailable, inaccessible, or underperforming in the identified region.

Abstract: Some embodiments provide systems and methods for validating cached content based on changes in the content instead of an expiration interval. One method involves caching content and a first checksum in response to a first request for that content. The caching produces a cached instance of the content representative of a form of the content at the time of caching. The first checksum identifies the cached instance. In response to receiving a second request for the content, the method submits a request for a second checksum representing a current instance of the content and a request for the current instance. Upon receiving the second checksum, the method serves the cached instance of the content when the first checksum matches the second checksum and serves the current instance of the content upon completion of the transfer of the current instance when the first checksum does not match the second checksum.

Abstract: Some embodiments provide a repository that manages configurations for a distributed platform and that automatedly configures servers of the distributed platform with different hierarchical sets of configurations while ensuring integrity and consistency across the servers and in the repository. In some embodiment, the repository includes a data store that stores configurations for a first set of servers that are operated by a first service provider and a second set of servers that are operated by a second service provider. The data store also identifies different sets of configurations to deploy to different sets of servers from the first and second sets of servers. The repository also includes a function processor to automatedly deploy the different sets of configurations to the different sets of servers and to perform functions for updating the configurations in a manner that ensures integrity and consistency.

Abstract: Some embodiments provide instantaneous and non-blocking content purging across storage servers of a distributed platform. When a server receives a purge operation, it extracts an identifier from the purge operation. The server then generates a content purge pattern from the identifier and injects the pattern to its configuration. Instantaneous purging is then realized as the server averts access to any cached content identified by the pattern. The purging also occurs in a non-blocking fashion as the physical purge of the content occurs in-line with the server's cache miss operation. The content purge pattern causes the server to respond to a subsequently received content request with a cache miss, whereby the server retrieves the requested content from an origin source, serves the retrieved content to the requesting user, and replaces a previously cached copy of the content that is to be purged with the newly retrieved copy.

Abstract: Some embodiments provide methods and systems for improving the scalability and redundancy of a distributed content streaming system. Such scalability and redundancy is provided with zero configuration changes to the addressing used by content providers to publish content and zero configuration changes to existing servers of the system. The system includes ingest servers and edge servers. Content providers supply content streams to the ingest servers using a virtual or load balanced address that distributes the content streams across the ingest servers. Accordingly, ingest servers can be added or removed without changing content provider configurations. The ingest servers are configured to notify the edge servers of which content streams are available for streaming at which ingest server. When an ingest server is added to the system, its functionality may be assimilated without modifying the configurations of the other servers. Some embodiments also provide multiple caching layers.

Abstract: Some embodiments provide systems and methods for performing localized and real-time server-side network performance monitoring. These systems and methods leverage the distributed architecture of a content delivery network (CDN) so as to perform distributed monitoring with each Point-of-Presence of the CDN responsible for monitoring performance to a localized set of end users. These systems and methods also leverage existing traffic flows from a server to a particular end user in order to perform real-time server-side network performance monitoring without the injection of specialized monitoring packets and without active involvement of the end user in deriving the performance measurements. The performance measurements are then used to optimize delivery of existing and future traffic flows to the end user.