Abstract: The routing of requests in a shared hosting environment. The shared hosting environment includes a network farm of servers, each capable of processing the request corresponding to the host name. When a request is received, a router determines whether or not there is already a host name affinitization of subset of the servers to the host name corresponding to the request. If so, the message is dispatched to one of those affinitized servers. If not, one or more of the servers are affinitized to the host name to create a subset of affinitized server(s) for that host name. Different host names may have different subsets of servers that they are affinitized to. Over time, the affinitization may be terminated as appropriate.

Abstract: Embodiments disclose a reverse lookup using an IP:Port-to-hostname table to identify a hostname when only an IP address and port is present in an SSL hello connection, which may occur, for example, when a non-SNI-capable client initiates the SSL hello. Once the hostname is successfully looked up, a naming convention is used to simplify the management and identification of SSL certificates. Different types of SSL certificates are supported. Multiple hostname matches may be associated with a given IP address and port in the IP:Port-to-hostname table. In such case, the first-matching hostname is always used with the naming convention to identify related SSL certificates. The naming convention is applied in such a way that it will first look for the most matching file name to the least matching file name.

Abstract: A distributed caching hierarchy that includes multiple edge routing servers, at least some of which receiving content requests from client computing systems via a load balancer. When receiving a content request, an edge routing server identifies which of the edge caching servers the requested content would be in if the requested content were to be cached within the edge caching servers, and distributes the content request to the identified edge caching server in a deterministic and predictable manner to increase the likelihood of increasing a cache-hit ratio.

Abstract: Servicing resource requests. A method includes at a first caching node, receiving a request for a first resource. The method further includes at the first caching node, determining at least one of health or load information of the first caching node. The method further includes generating a response to the request for the first resource, including generating a header to the response. The header is a custom header. The custom header includes at least one of health or load information. The method further includes sending the response to the request including sending the custom header with the at least one of health or load information.

Abstract: A load balancing system is described herein that proactively balances client requests among multiple destination servers using information about anticipated loads or events on each destination server to inform the load balancing decision. The system detects one or more upcoming events that will affect the performance and/or capacity for handling requests of a destination server. Upon detecting the event, the system informs the load balancer to drain connections around the time of the event. Next, the event occurs on the destination server, and the system detects when the event is complete. In response, the system informs the load balancer to restore connections to the destination server. In this way, the system is able to redirect clients to other available destination servers before the tasks occur. Thus, the load balancing system provides more efficient routing of client requests and improves responsiveness.

Abstract: Architecture that facilitates the capture of connection state of a connection established between a client and an intermediate server and forwards the state to one or more target servers. A software component at the target server (as well as the intermediate server) uses this connection state to reply back to the client directly, thereby bypassing the intermediate server. All packets from the client related to the request are received at the intermediate server and then forwarded to the target server. The migration can be accomplished without any change in the client operating system and client applications, without assistance from a gateway device such as a load balancer or the network, without duplication of all packets between the multiple servers, and without changes to the transport layer stack of the intermediate and target servers.

Abstract: A live caching system is described herein that reduces the burden on origin servers for serving live content. In response to receiving a first request that results in a cache miss, the system forwards the first request to the next tier while “holding” other requests for the same content. If the system receives a second request while the first request is pending, the system will recognize that a similar request is outstanding and hold the second request by not forwarding the request to the origin server. After the response to the first request arrives from the next tier, the system shares the response with other held requests. Thus, the live caching system allows a content provider to prepare for very large events by adding more cache hardware and building out a cache server network rather than by increasing the capacity of the origin server.

Abstract: Requests for content can be received from clients and forwarded to servers, and responses to the requests can be received from the servers and forwarded to the clients. A health model can also be maintained. The health model can be based on information in the responses and possibly also on information in the requests, and the health model can indicate the health of the servers in responding to different types of requests. The health model may differentiate between health in responding to requests with different features in URLs of the requests, such as different namespaces and/or different extensions.

Abstract: An intelligent caching system is described herein that intelligently consolidates the name-value pairs in content requests containing query strings so that only substantially non-redundant responses are cached, thereby saving cache proxy resources. The intelligent caching system determines which name-value pairs in the query string can affect the redundancy of the content response and which name-value pairs can be ignored. The intelligent caching system organically builds the list of relevant name-value pairs by relying on a custom response header or other indication from the content server. Thus, the intelligent caching system results in fewer requests to the content server as well as fewer objects in the cache.

Abstract: A selective pre-caching system reduces the amount of content cached at cache proxies by limiting the cached content to that content that a particular cache proxy is responsible for caching. This can substantially reduce the content stored on each cache proxy and reduces the amount of resources consumed for pre-caching in preparation for a particular event. The cache proxy receives a list of content items that and an indication of the topology of the cache network. The cache proxy uses the received topology to determine the content items in the received list of content items that the cache proxy is responsible for caching. The cache proxy then retrieves the determined content items so that they are available in the cache before client requests are received.

Abstract: Dynamically upsizing and/or downsizing a network farm in response to network demand. An application message router routes messages to the network farm. When the network farm approaches or is anticipated to be approaching capacity, a group of one or more servers may be added to the network farm. When the added server(s) are capable of participating in the network farm, the application message router is triggered to route also to the added servers. When the network farm has excess capacity, a group of one or more servers may be dropped from the network farm. This may be accomplished by triggering the application message router to no longer route messages to the removed servers. The removed servers may be either immediately or gracefully removed from service.

Abstract: A caching system segments content into multiple, individually cacheable chunks cached by a cache server that caches partial content and serves byte range requests with low latency and fewer duplicate requests to an origin server. The system receives a request from a client for a byte range of a content resource. The system determines the chunks overlapped by the specified byte range and sends a byte range request to the origin server for the overlapped chunks not already stored in a cache. The system stores the bytes of received responses as chunks in the cache and responds to the received request using the chunks stored in the cache. The system serves subsequent requests that overlap with previously requested ranges of bytes from the already retrieved chunks in the cache and makes requests to the origin server only for those chunks that a client has not previously requested.

Abstract: A load balancing system is described herein that proactively balances client requests among multiple destination servers using information about anticipated loads or events on each destination server to inform the load balancing decision. The system detects one or more upcoming events that will affect the performance and/or capacity for handling requests of a destination server. Upon detecting the event, the system informs the load balancer to drain connections around the time of the event. Next, the event occurs on the destination server, and the system detects when the event is complete. In response, the system informs the load balancer to restore connections to the destination server. In this way, the system is able to redirect clients to other available destination servers before the tasks occur. Thus, the load balancing system provides more efficient routing of client requests and improves responsiveness.

Abstract: A live caching system is described herein that reduces the burden on origin servers for serving live content. In response to receiving a first request that results in a cache miss, the system forwards the first request to the next tier while “holding” other requests for the same content. If the system receives a second request while the first request is pending, the system will recognize that a similar request is outstanding and hold the second request by not forwarding the request to the origin server. After the response to the first request arrives from the next tier, the system shares the response with other held requests. Thus, the live caching system allows a content provider to prepare for very large events by adding more cache hardware and building out a cache server network rather than by increasing the capacity of the origin server.

Abstract: The routing of requests in a shared hosting environment. The shared hosting environment includes a network farm of servers, each capable of processing the request corresponding to the host name. When a request is received, a router determines whether or not there is already a host name affinitization of subset of the servers to the host name corresponding to the request. If so, the message is dispatched to one of those affinitized servers. If not, one or more of the servers are affinitized to the host name to create a subset of affinitized server(s) for that host name. Different host names may have different subsets of servers that they are affinitized to. Over time, the affinitization may be terminated as appropriate.

Abstract: An application request router that routes incoming application message to various servers in a network farm, even though the original network request itself does not directly specify which server is to handle the request. The application request routing module uses intra-farm routing policy and characteristics of the request itself to identify which of the servers is to handle the message and then dispatches the message to the appropriate server. This allows a user to reconfigure the network farm by reconfiguring the capabilities of the servers in the network farm, and then altering the routing policy accordingly to take advantage of the reconfigured network farm capabilities.

Abstract: Dynamically upsizing and/or downsizing a network farm in response to network demand. An application message router routes messages to the network farm. When the network farm approaches or is anticipated to be approaching capacity, a group of one or more servers may be added to the network farm. When the added server(s) are capable of participating in the network farm, the application message router is triggered to route also to the added servers. When the network farm has excess capacity, a group of one or more servers may be dropped from the network farm. This may be accomplished by triggering the application message router to no longer route messages to the removed servers. The removed servers may be either immediately or gracefully removed from service.