Abstract: In an embodiment, a system for handling a device in latency state in a redundant storage system includes a processor configured to process a write request to a plurality of devices associated with a redundant storage system. The processor is further configured to receive a set of indications of write successes from a set of devices included in the plurality of devices other than a first device, and send an indication of a completed write to a requestor associated with the write request based at least in part on an indication that the first device meets a set of one or more criteria associated with a latency state. The system further includes a memory coupled to the processor and configured to store the write request.

Abstract: Efficient deduplication for storage systems is disclosed, including: using the current data structure to track a set of deduplication data entries associated with a collection group; determining a flush criterion has been met; and storing the set of deduplication data entries of the current data structure as one or more persistent objects associated with the collection group in a persistent storage, wherein the one or more persistent objects are usable to update a set of persistently stored metadata associated with the collection group that tracks deduplication data entries generated before a previous merge operation.

Abstract: Efficient recovery of deduplication data for high capacity systems is disclosed, including: reading from the data storage device a data structure that tracks a plurality of segments to which a plurality of persistent objects have been recently written, wherein segments are written to in a monotonically increasing numerical order; selecting a checkpoint segment from among the plurality of segments based at least in part on a plurality of segment numbers corresponding to respective ones of the plurality of segments; using the checkpoint segment and a segment associated with a latest available segment number to determine a set of candidate segments; reading at least a portion of the set of candidate segments to identify a data storage block for which a corresponding deduplication data entry is not already stored in persistently stored deduplication data entries; and storing a new deduplication data entry to insert a fingerprint associated with the data storage block in a current data structure stored in a memory

Abstract: Dynamic RAID expansion is disclosed, including: incrementally expanding a redundant storage system comprising a plurality of storage drives by adding one or more storage drives to the plurality of storage drives; and storing, for each stripe included in the redundant storage system, metadata indicating which storage drives in the redundant storage system store valid data for the stripe.

Abstract: Efficient deduplication for storage systems is disclosed, including: using the current data structure to track a set of deduplication data entries associated with a collection group; determining a flush criterion has been met; and storing the set of deduplication data entries of the current data structure as one or more persistent objects associated with the collection group in a persistent storage, wherein the one or more persistent objects are usable to update a set of persistently stored metadata associated with the collection group that tracks deduplication data entries generated before a previous merge operation.

Abstract: Efficient recovery of deduplication data for high capacity systems is disclosed, including: reading from the data storage device a data structure that tracks a plurality of segments to which a plurality of persistent objects have been recently written, wherein segments are written to in a monotonically increasing numerical order; selecting a checkpoint segment from among the plurality of segments based at least in part on a plurality of segment numbers corresponding to respective ones of the plurality of segments; using the checkpoint segment and a segment associated with a latest available segment number to determine a set of candidate segments; reading at least a portion of the set of candidate segments to identify a data storage block for which a corresponding deduplication data entry is not already stored in persistently stored deduplication data entries; and storing a new deduplication data entry to insert a fingerprint associated with the data storage block in a current data structure stored in a memory

Abstract: A data storage system is disclosed, including: a first storage device of a first storage device type, a second storage device of a second storage device type, and a processor configured to implement a hybrid file system configured to store each of a plurality of data values in at least a corresponding primary location, which for a given data value may be in either the first storage device or the second storage device; wherein the hybrid file system is configured to use a first access approach optimized based on a first access attribute of the first storage device type to interact with the first storage device and a second access approach optimized based on a second access attribute of the second storage device type to interact with the second storage device.

Abstract: Disclosed are, inter alia, methods, apparatus, data structures, computer-readable media, mechanisms, and means for defining, creating and using tree bitmap data structures, such as for, but not limited to their use in performing lookup operations (e.g., longest prefix matching, etc.). The data structure typically includes a tree bitmap for identifying for each node of multiple nodes within a stride of a number of tree levels greater than one whether each node is a prefix or vacant node, the multiple nodes representing multiple tree levels, a lowest level subset of the multiple nodes corresponding to a lowest level of the tree levels in the stride, the lowest level subset of the multiple nodes including two or more nodes. A child bitmap is typically used for identifying which trie paths emanate and which trie paths do not emanate from the lowest level subset of the multiple nodes.

Abstract: Methods and apparatus are disclosed for accumulating traffic information and distributing flow control information in a packet switching system. Traffic information is collected in multiple elements and indications of congestion and/or other types of information useful in determining traffic conditions are forwarded to collecting elements. The collecting elements manipulate the received indications and generate flow control messages which are sent to individual sending components. In one implementation, a switching element maintains for each destination a count of packets within itself which are addressed to the particular destination. Indications of this collected information are sent to collecting switching elements. These collecting elements accumulate the information received from multiple sources. The accumulated information is evaluated, and when a congestion condition is located or anticipated, then flow control messages are distributed to all, or a subset of, the packet sources.

Abstract: Methods and apparatuses are disclosed for controlling the rate at which packets are sent from a first to a second component of a packet switching system. In one implementation, the first component represents an input line card to a packet switch, and the second component represents an output of the packet switch. In such a system, a state is maintained for each output at each line card. For example, these states may include an unconstrained state during which traffic is sent at a full rate to the output, an off state during which no traffic is sent to the output, and a constrained state during which traffic is sent at a reduced rate to the output. Typically, this reduced rate is proportional to the arrival rate of packets at the input line card which are destined for the output. The state of the output is changed based on received flow control information about the output and whether traffic remains queued for the output at the input line card.

Abstract: Methods and apparatus for forwarding packets in a multistage interconnection network are provided which timestamp packets using a substantially system-wide timing reference and a merge sorting variant to restore packets to the proper order, using the timestamp information carried in the packets. One implementation determines when packets passing along different paths in the network can be safely forwarded, even when no packets have recently been received on some of the paths, by forwarding status messages along otherwise idle paths. The status messages provide information that can be used by downstream components to allow them to determine when packets passing over other paths can safely be forwarded. One implementation simultaneously resequences packets being delivered to all n outputs of the multistage interconnection network. The resequencing operations are distributed among a plurality of switching elements making up the interconnection network.

Abstract: According to the invention, methods and apparatus are disclosed for selecting one of multiple of paths between two points over which to route a data item based on the destination of the data item and the traffic between the two points over the multiple paths. A switching system can use the disclosed methods and apparatus to more efficiently distribute data packets among switching fabrics than currently accomplished by known techniques. In one implementation, distribution cycles have been established for sending data between two points, where each path between the endpoints is used a predetermined number of times (e.g., one, two) within each cycle. To economize the amount of traffic data collected, the multiple paths can be partitioned into subsets for which traffic data is maintained only for the current subset. Additionally, the distribution of traffic between the two points can be further partitioned into traffic of a particular type or priority between the two points.

Abstract: In one implementation, a first set of packet switch sequence numbers is used for end-to-end resequencing of packets within a packet switch, and a second set of interconnection network sequence number is used in the resequencing of packets within an interconnection network of the packet switch. A packet switch sequence number is maintained at each input interface of the packet switch for each output interface, while each output interface maintains a packet switch sequence number for each input interface. A corresponding sequence number is added to packets sent between corresponding input-output interface pairs. Similarly, an interconnection network sequence number is maintained at each input port of an interconnection network for each output port, while each output port maintains an interconnection network sequence number for each input port. A corresponding sequence number is added to packets sent between corresponding input-output port pairs.

Abstract: Methods and apparatus are also disclosed for responding to received flow control messages indicating a previously congested port is now in a non-congested state. Many different components that have packets to send to a particular output will receive an indication that they are now allowed to send these packets at roughly the same time as the other components. If all components start sending at the same time, then the packet switch might become congested, possibly very quickly. If the packet switch cannot respond and transmit flow control messages to all of these sources fast enough, certain internal buffers could overflow and thus packets might be lost. On implementation causes components to start sending to the destination at varying times to gradually increase the traffic being sent to the destination.

Abstract: Methods and apparatus are disclosed for determining the order to send information arriving at the inputs of a packet switching system. A line card maintains its own data structure indicating flow control information and a queue for each destination it is sending data with its memory. Control logic then controls the placing of the incoming data into these queues and for taking the data out of the queues and sending the data, typically in the form of packets, to the packet switch. As information arrives at a line card, priority outgoing packet time slots are allocated for that destination. In this manner, each destination is given the opportunity to send information at its arrival rate. In the remaining bandwidth or packet cycles available on the outgoing link, the destination queues containing information retained due to a congestion condition are serviced.

Abstract: Methods and apparatus are disclosed for accumulating and distributing flow control information via update messages and piggybacked flow control information in other messages. One implementation operates using at least two techniques. Using a first technique, for every packet entering the switching system from a line card, the switching system conveys flow control information (typically congestion or both congestion and no-congestion indications) for the packet's destination to the line card. Using a second technique, the switching system will periodically convey congestion and no-congestion indications for all destinations to the line cards. In one implementation, when the first technique is used to provide only congestion indications, the periodic distribution of flow control information using the second technique provides non-congested indications which allows the line cards and their sources to resume or begin sending to the non-congested destinations.