Abstract: The present invention is directed towards forwarding network packets in a cluster network. A predetermined identifier may be inserted into a Media Access Control (MAC) ID field of an Ethernet header of a packet to distinguish various types of traffic. Newly received packets may be identified due to the absence of the identifier. The identifier may be added to the source MAC ID field of the Ethernet header of the packet, and the packet may be distributed to cluster nodes for processing via an inter-node communication bus. Thus, received packets with the identifier in the source MAC ID field may be identified as steered for processing by an internal node of the cluster. After processing the packet, the internal node may transmit the processed packets via the inter-node bus with a destination MAC ID including the identifier.

Abstract: Systems and methods for reducing file sizes for files delivered over a network are disclosed. A method comprises receiving a first file comprising sequences of data; creating a hash table having entries corresponding to overlapping sequences of data; receiving a second file comprising sequences of data; comparing each of the sequences of data in the second file to the sequences of data in the hash table to determine sequences of data present in both the first and second files; and creating a third file comprising sequences of data from the second file and representations of locations and lengths of said sequences of data present in both the first and second files.

Abstract: In a cluster environment, nodes participating in the cluster may generate packets with the same source IP to the same destination IP. If those packets get fragmented down stream, then reassembling this packets depends on the source IP, destination IP, protocol and the IP identifier field (IPID) of the IP packet. As the source IP, destination IP and protocol may be fixed, the IPID generation is coordinated across the nodes to support reassembly down stream and avoid tuple collision. The IPID space may be equally split among the nodes in the cluster. Each node generates IPID within its range to avoid a collision with an IPID generated from another node.

Abstract: The present application is directed towards sharing data link layer information of network traffic distributed across a cluster of intermediary devices. A method for sharing data link layer information across a cluster includes receiving a request packet at a first intermediary device. The first intermediary device identifies a first set of data link layer information from a data link layer of the request packet. The first intermediary device modifies the request packet for transmission on a common data backplane of the cluster to include the first set of data link layer information in the request packet. The modified request packet includes a second set of data link layer information that differs from the first set of data link layer information at the data link layer. The first intermediary device transmits the modified request packet on the common data backplane of the cluster to other devices of the cluster.

Abstract: Described herein is a method and system for directing outgoing data packets from packet engines to a transmit queue of a NIC in a multi-core system, and a method and system for directing incoming data packets from a receive queue of the NIC to the packet engines. Packet engines store outgoing traffic in logical transmit queues in the packet engines. An interface module obtains the outgoing traffic and stores it in a transmit queue of the NIC, after which the NIC transmits the traffic from the multi-core system over a network. The NIC receives incoming traffic and stores it in a NIC receive queue. The interface module obtains the incoming traffic and applies a hash to a tuple of each obtained data packet. The interface module then stores each data packet in the logical receive queue of a packet engine on the core identified by the result of the hash.

Abstract: The present disclosure presents systems and methods for controlling network traffic traversing an intermediary device based on a license or a permit granted for the intermediary device. The systems and methods control a rate of a traffic of a device in accordance with a rate limit identified by a rate limiting license. A rate limiting manager of an intermediary device that processes network traffic between a plurality of clients and a plurality of servers, may identify presence of a rate limiting license that further identifies a performance level. The rate limiting manager may establish a rate limit based on the performance level of the rate limiting license. A throttler of the intermediary may control a rate of receiving network packets in accordance with the rate limit.

Abstract: Systems and methods for reducing file sizes for files delivered over a network are disclosed. A method comprises receiving a first file comprising sequences of data; creating a hash table having entries corresponding to overlapping sequences of data; receiving a second file comprising sequences of data; comparing each of the sequences of data in the second file to the sequences of data in the hash table to determine sequences of data present in both the first and second files; and creating a third file comprising sequences of data from the second file and representations of locations and lengths of said sequences of data present in both the first and second files.

Abstract: Systems and methods for reducing file sizes for files delivered over a network are disclosed. A method comprises receiving a first file comprising sequences of data; creating a hash table having entries corresponding to overlapping sequences of data; receiving a second file comprising sequences of data; comparing each of the sequences of data in the second file to the sequences of data in the hash table to determine sequences of data present in both the first and second files; and creating a third file comprising sequences of data from the second file and representations of locations and lengths of said sequences of data present in both the first and second files.

Abstract: Described herein is a method and system for distributing whole and fragmented requests and responses across a multi-core system. Each core executes a packet engine that further processes data packets and data packet fragments allocated to that core. A flow distributor executing within the multi-core system forwards client requests to a packet engine on a core that is selected based on a value generated when a hash is applied to a tuple comprising a client IP address, a client port, a server IP address and a server port identified in the request. The packet engine maintains each element of the tuple and forwards the request to the selected core. The packet engine can also process data packet fragments by assembling the fragments prior to transmitting them to the selected core, or by transmitting the data packet fragments to the selected core.

Abstract: The present invention is directed towards forwarding network packets in a cluster network. A predetermined identifier may be inserted into a Media Access Control (MAC) ID field of an Ethernet header of a packet to distinguish various types of traffic. Newly received packets may be identified due to the absence of the identifier. The identifier may be added to the source MAC ID field of the Ethernet header of the packet, and the packet may be distributed to cluster nodes for processing via an inter-node communication bus. Thus, received packets with the identifier in the source MAC ID field may be identified as steered for processing by an internal node of the cluster. After processing the packet, the internal node may transmit the processed packets via the inter-node bus with a destination MAC ID including the identifier.

Abstract: The present application is directed towards sharing data link layer information of network traffic distributed across a cluster of intermediary devices. A method for sharing data link layer information across a cluster includes receiving a request packet at a first intermediary device. The first intermediary device identifies a first set of data link layer information from a data link layer of the request packet. The first intermediary device modifies the request packet for transmission on a common data backplane of the cluster to include the first set of data link layer information in the request packet. The modified request packet includes a second set of data link layer information that differs from the first set of data link layer information at the data link layer. The first intermediary device transmits the modified request packet on the common data backplane of the cluster to other devices of the cluster.

Abstract: In a cluster environment, nodes participating in the cluster may generate packets with the same source IP to the same destination IP. If those packets get fragmented down stream, then reassembling this packets depends on the source IP, destination IP, protocol and the IP identifier field (IPID) of the IP packet. As the source IP, destination IP and protocol may be fixed, the IPID generation is coordinated across the nodes to support reassembly down stream and avoid tuple collision. The IPID space may be equally split among the nodes in the cluster. Each node generates IPID within its range to avoid a collision with an IPID generated from another node.

Abstract: The present invention is directed towards systems and methods for determining failure in and controlling access to a shared resource in a multi-core system. In some embodiments of a multi-core system, individual cores may share the same resource. Additionally, the resource may occasionally fail or need to be reset, and the period during which the resource is being reset may be non-instantaneous. In an embodiment without coordination between the cores, one core experiencing a failure may reset the resource. During the period in which the resource is resetting, another core may interpret the reset as a failure and reset the resource. As more cores interpret the resets as failures, they will trigger resets, quickly resulting in the resource being constantly reset and unavailable. Thus, in some embodiments, a coordination system may be utilized to determine failure of a shared resource and control resets and access to the shared resource.

Abstract: Described herein is a method and system for directing outgoing data packets from packet engines to a transmit queue of a NIC in a multi-core system, and a method and system for directing incoming data packets from a receive queue of the NIC to the packet engines. Packet engines store outgoing traffic in logical transmit queues in the packet engines. An interface module obtains the outgoing traffic and stores it in a transmit queue of the NIC, after which the NIC transmits the traffic from the multi-core system over a network. The NIC receives incoming traffic and stores it in a NIC receive queue. The interface module obtains the incoming traffic and applies a hash to a tuple of each obtained data packet. The interface module then stores each data packet in the logical receive queue of a packet engine on the core identified by the result of the hash.

Abstract: Systems and methods for load balancing services based on fewest connections by decreasing granularity of service selection as a number of fewest connections serviced by the services increases may include establishing, by an appliance, a set of identifiers corresponding to a number of connections serviced by a service, the set comprising a first plurality of identifiers each identifying a predetermined number of connections and a second plurality of identifiers each identifying a predetermined range of numbers of connections. The appliance assigns, to each service servicing connections, an identifier corresponding to the number of connections serviced by the service, at least one of the identifiers selected from the second plurality of identifiers. The appliance receives a request for a service, and forwards the request to a service assigned to the identifier corresponding to a fewest number of connections with at least one service assigned to the identifier.

Abstract: Described herein is a method and system for distributing whole and fragmented requests and responses across a multi-core system. Each core executes a packet engine that further processes data packets and data packet fragments allocated to that core. A flow distributor executing within the multi-core system forwards client requests to a packet engine on a core that is selected based on a value generated when a hash is applied to a tuple comprising a client IP address, a client port, a server IP address and a server port identified in the request. The packet engine maintains each element of the tuple and forwards the request to the selected core. The packet engine can also process data packet fragments by assembling the fragments prior to transmitting them to the selected core, or by transmitting the data packet fragments to the selected core.

Abstract: Described herein is a method and system for directing outgoing data packets from packet engines to a transmit queue of a NIC in a multi-core system, and a method and system for directing incoming data packets from a receive queue of the NIC to the packet engines. Packet engines store outgoing traffic in logical transmit queues in the packet engines. An interface module obtains the outgoing traffic and stores it in a transmit queue of the NIC, after which the NIC transmits the traffic from the multi-core system over a network. The NIC receives incoming traffic and stores it in a NIC receive queue. The interface module obtains the incoming traffic and applies a hash to a tuple of each obtained data packet. The interface module then stores each data packet in the logical receive queue of a packet engine on the core identified by the result of the hash.

Abstract: Described herein is a method and system for distributing whole and fragmented requests and responses across a multi-core system. Each core executes a packet engine that further processes data packets and data packet fragments allocated to that core. A flow distributor executing within the multi-core system forwards client requests to a packet engine on a core that is selected based on a value generated when a hash is applied to a tuple comprising a client IP address, a client port, a server IP address and a server port identified in the request. The packet engine maintains each element of the tuple and forwards the request to the selected core. The packet engine can also process data packet fragments by assembling the fragments prior to transmitting them to the selected core, or by transmitting the data packet fragments to the selected core.

Abstract: Described herein is a method and system for directing outgoing data packets from packet engines to a transmit queue of a NIC in a multi-core system, and a method and system for directing incoming data packets from a receive queue of the NIC to the packet engines. Packet engines store outgoing traffic in logical transmit queues in the packet engines. An interface module obtains the outgoing traffic and stores it in a transmit queue of the NIC, after which the NIC transmits the traffic from the multi-core system over a network. The NIC receives incoming traffic and stores it in a NIC receive queue. The interface module obtains the incoming traffic and applies a hash to a tuple of each obtained data packet. The interface module then stores each data packet in the logical receive queue of a packet engine on the core identified by the result of the hash.

Abstract: The present disclosure presents systems and methods for controlling network traffic traversing an intermediary device based on a license or a permit granted for the intermediary device. The systems and methods control a rate of a traffic of a device in accordance with a rate limit identified by a rate limiting license. A rate limiting manager of an intermediary device that processes network traffic between a plurality of clients and a plurality of servers, may identify presence of a rate limiting license that further identifies a performance level. The rate limiting manager may establish a rate limit based on the performance level of the rate limiting license. A throttler of the intermediary may control a rate of receiving network packets in accordance with the rate limit.