Abstract: Some embodiments provide a capacity management agent that modifies content requests to adjust bandwidth consumption when streaming requested content from a content provider to a requesting user. The modifications include modifying a URL or header information of the request. The agent performs a process that receives a request for content of a content provider. The process identifies a parameter of the carrier network and modifies the request when the parameter satisfies a threshold. The process passes the request to the content provider and the content provider provides content that consumes a first set of resources in response to an unmodified request and a second set of resources in response to a modified request. When the parameter identifies congestion, the first set of resources is greater than the second set of resources. When the condition parameter identifies underutilization, the first set of resources is less than the second set of resources.

Abstract: Some embodiments provide firewalls and methods for guarding against attacks by leveraging the Document Object Model (DOM). The firewall renders the DOM tree to produce a white-list rendering of the data which presents the non-executable elements of the data and, potentially, outputs of the executable elements of the data without the executable elements that could be used to carry a security threat. Some embodiments provide control over which nodes of the DOM tree are included in producing the white-list rendering. Specifically, a configuration file is specified to white-list various nodes from the DOM tree and the white-list rendering is produced by including the DOM tree nodes that are specified in the white-list of the configuration file while excluding those nodes that are not in the white-list. Some embodiments provide a hybrid firewall that executes a set of black-list rules over white-listed nodes of the DOM tree.

Abstract: Some embodiments provide a capacity exchange whereby capacity from different content delivery networks (CDNs) can be bought, sold, and traded. The capacity exchange is part of an “Open CDN” platform. The Open CDN platform federates the independent operation of CDNs and other operators and service providers to distributed platforms participating in the Open CDN platform. The Open CDN platform includes one or more APIs for facilitating intercommunication between the federation participants by performing configuration mapping, command interoperability, traffic management, and reporting aggregation.

Abstract: Some embodiments provide systems and methods for validating cached content based on changes in the content instead of an expiration interval. One method involves caching content and a first checksum in response to a first request for that content. The caching produces a cached instance of the content representative of a form of the content at the time of caching. The first checksum identifies the cached instance. In response to receiving a second request for the content, the method submits a request for a second checksum representing a current instance of the content and a request for the current instance. Upon receiving the second checksum, the method serves the cached instance of the content when the first checksum matches the second checksum and serves the current instance of the content upon completion of the transfer of the current instance when the first checksum does not match the second checksum.

Abstract: Some embodiments provide a repository that manages configurations for a distributed platform and that automatedly configures servers of the distributed platform with different hierarchical sets of configurations while ensuring integrity and consistency across the servers and in the repository. In some embodiment, the repository includes a data store that stores configurations for a first set of servers that are operated by a first service provider and a second set of servers that are operated by a second service provider. The data store also identifies different sets of configurations to deploy to different sets of servers from the first and second sets of servers. The repository also includes a function processor to automatedly deploy the different sets of configurations to the different sets of servers and to perform functions for updating the configurations in a manner that ensures integrity and consistency.

Abstract: Some embodiments provide multi-tenant billing systems and methods whereby data network usage fees that are associated with delivering select content of different content providers over a data network are passed to the content provider that originates or is the source for that select content instead of the end users that request and receive the select content. A deep packet inspection server or a billing server may be configured with a list of identifiers to distinguish between the network usage that is related to the delivery of the designated free content from the network usage that is not related to the delivery of the designated free content. The network usage can be distinguished by filtering or processing usage logs that track the network usage using the configured list of identifiers. Such filtering or processing can be done in real-time or in a delayed manner in a standalone or integrated system.

Abstract: Some embodiments provide a content delivery network (CDN) solution that affords the CDN control over those elements of customer content that are delivered by third parties. The CDN integrates a distributed set of monitoring agents. Each monitoring agent monitors the delivery performance of third parties to the region in which the agent operates. The CDN uses the performance monitoring information to dynamically manage the content tags to the third-party delivered elements of CDN-customer content. Specifically, a CDN server retrieves the parent page for requested CDN-customer content. The CDN server identifies the region from where the request originates and retrieves the logs from the monitoring agents monitoring from that region. The CDN server then modifies the base page by dynamically removing the tags to the third-party delivered elements that are reported in the monitoring agent logs as being unavailable, inaccessible, or underperforming in the identified region.

Abstract: Some embodiments provide systems and methods for determining a server of a distributed hosting system to optimally distribute content to an end user. The method includes identifying an IP address of the end user. Based on the IP address, a set of servers send packets to the end user to derive performance metrics. The performance metrics are used to determine a server from the set of servers that optimally distributes content to the end user. The method modifies a configuration for resolving end user requests such that the optimal server is identified to the end user when the end user requests content from the hosting system. Some embodiments determine the optimal server by providing downloadable content that is embedded with a monitoring tool. The monitoring tool causes the end user to derive performance metrics for the hosting system when downloading a particular object from a set of servers.

Abstract: Some embodiments provide a content delivery network (CDN) solution that affords the CDN control over those elements of customer content that are delivered by third parties. The CDN integrates a distributed set of monitoring agents. Each monitoring agent monitors the delivery performance of third parties to the region in which the agent operates. The CDN uses the performance monitoring information to dynamically manage the content tags to the third-party delivered elements of CDN-customer content. Specifically, a CDN server retrieves the parent page for requested CDN-customer content. The CDN server identifies the region from where the request originates and retrieves the logs from the monitoring agents monitoring from that region. The CDN server then modifies the base page by dynamically removing the tags to the third-party delivered elements that are reported in the monitoring agent logs as being unavailable, inaccessible, or underperforming in the identified region.

Abstract: Some embodiments implement systems and methods to monitor CDN performance from the end user's perspective. To achieve full end-to-end monitoring, the systems and methods monitor the performance of Anycast routing for routing an end user to the optimal PoP of the CDN and monitor the performance of different routes that deliver content from the optimally selected PoP back to the end user. Systems and methods are also provided to leverage the monitored performance in order to automatically optimize the performance of the CDN by optimizing or modifying Anycast routing and by performing route grooming as needed to ensure that the optimal route is selected to deliver content to different sets of end users that request content from a particular PoP.

Abstract: Some embodiments provide a scalable content streaming system that performs server-side archiving. The content streaming system includes a set of streaming server clusters, a remote storage server, and a set of distributed servers. Each streaming server cluster includes at least one streaming server and a local storage server. Each streaming server of a particular cluster distributes a content stream that is uploaded to the streaming server while also recording the content stream to the local storage server of that particular cluster. The remote storage server stores files from the local storage servers that are transferred when recording to those files is complete. The set of distributed servers distribute a live stream of content from a particular streaming server when the content is being uploaded to the particular streaming server and distribute a recorded stream of the content from the remote storage server when the upload is complete.

Abstract: Some embodiments provide a file system for purging content based on a single traversal of the file system that identifies the directory containing the content without performing a secondary traversal within the directory to target the operation to only the file that are associated with content such that other files contained in the directory are unaffected. The file system supplements traditional directory structures with file-level directories. Each file-level directory is created to contain a root file associated with particular content, different variants of the particular content, and supporting files. Consequently, the file system can complete an operation targeting particular content by performing that operation on the file-level directory containing the particular content, thereby eliminating the need to conduct a file-by-file traversal of the containing directory as a prerequisite to identifying the files associated with the particular content and performing the operation on the files individually.

Abstract: Some embodiments provide a director agent, a server agent, and a specialized hand-off protocol for improving scalability and resource usage within a server farm. A first network connection is established between a client and the director agent in order to receive a content request from the client from which to select a server from a set of servers that is responsible for hosting the requested content. A second network connection is established between the server agent that is associated with the selected server and a protocol stack of the selected server. The first network connection is handed-off to the server agent using the specialized hand-off protocol. The server agent performs network connection state parameter transformations between the two connections to create a network connection through which content can be passed from the selected server to the client without passing through the director.

Abstract: Some embodiments provide a proprietary 64-bit consistent distribution scheme that preserves the efficiencies of CARP while providing a significantly more balanced distribution of requests that is on par with schemes reliant on computationally expensive cryptographic hashes. The scheme performs hashing of requested URLs and identifiers of available servers over a 64-bit space while optimizing the hashing to remove computationally expensive operations. Some embodiments provide a variant of the scheme to provide a differentiated distribution on the basis of one or more differentiating factors. A first variant utilizes load factor values to adjust the resulting hashes and to produce a first distribution of differentiated content that varies from a second distribution of undifferentiated content.

Abstract: Some embodiments provide systems and methods for determining a server of a distributed hosting system to optimally distribute content to an end user. The method includes identifying an IP address of the end user. Based on the IP address, a set of servers send packets to the end user to derive performance metrics. The performance metrics are used to determine a server from the set of servers that optimally distributes content to the end user. The method modifies a configuration for resolving end user requests such that the optimal server is identified to the end user when the end user requests content from the hosting system. Some embodiments determine the optimal server by providing downloadable content that is embedded with a monitoring tool. The monitoring tool causes the end user to derive performance metrics for the hosting system when downloading a particular object from a set of servers.

Abstract: The invention relates to systems and methods of global load balancing in a content delivery network having a plurality of edge servers which may be distributed across multiple geographic locations. According to one aspect of the invention, a global load balancing system includes a first load balancing server for receiving a packet requesting content to be delivered to a client, selecting one of the plurality of edge servers to deliver the requested content to the client, and forwarding the packet across a network connection to a second load balancing server, which forwards the packet to the selected edge server. The selected edge server, in response to receiving the packet, sends across a network connection the requested content with an address for direct delivery to the client, thereby allowing the requested content to be delivered to the client while bypassing a return path through the first load balancing server.

Abstract: Some embodiments implement systems and methods to monitor CDN performance from the end user's perspective. To achieve full end-to-end monitoring, the systems and methods monitor the performance of Anycast routing for routing an end user to the optimal PoP of the CDN and monitor the performance of different routes that deliver content from the optimally selected PoP back to the end user. Systems and methods are also provided to leverage the monitored performance in order to automatically optimize the performance of the CDN by optimizing or modifying Anycast routing and by performing route grooming as needed to ensure that the optimal route is selected to deliver content to different sets of end users that request content from a particular PoP.

Abstract: Some embodiments implement systems and methods to monitor CDN performance from the end user's perspective. To achieve full end-to-end monitoring, the systems and methods monitor the performance of Anycast routing for routing an end user to the optimal PoP of the CDN and monitor the performance of different routes that deliver content from the optimally selected PoP back to the end user. Systems and methods are also provided to leverage the monitored performance in order to automatically optimize the performance of the CDN by optimizing or modifying Anycast routing and by performing route grooming as needed to ensure that the optimal route is selected to deliver content to different sets of end users that request content from a particular PoP.

Abstract: Some embodiments provide a scalable content streaming system that performs server-side archiving. The content streaming system includes a set of streaming server clusters, a remote storage server, and a set of distributed servers. Each streaming server cluster includes at least one streaming server and a local storage server. Each streaming server of a particular cluster distributes a content stream that is uploaded to the streaming server while also recording the content stream to the local storage server of that particular cluster. The remote storage server stores files from the local storage servers that are transferred when recording to those files is complete. The set of distributed servers distribute a live stream of content from a particular streaming server when the content is being uploaded to the particular streaming server and distribute a recorded stream of the content from the remote storage server when the upload is complete.

Abstract: Some embodiments provide systems and methods for determining a server of a distributed hosting system to optimally distribute content to an end user. The method includes identifying an IP address of the end user. Based on the IP address, a set of servers send packets to the end user to derive performance metrics. The performance metrics are used to determine a server from the set of servers that optimally distributes content to the end user. The method modifies a configuration for resolving end user requests such that the optimal server is identified to the end user when the end user requests content from the hosting system. Some embodiments determine the optimal server by providing downloadable content that is embedded with a monitoring tool. The monitoring tool causes the end user to derive performance metrics for the hosting system when downloading a particular object from a set of servers.