APPARATUS AND METHOD FOR PRIORITY QUEUING WITH SEGMENTED BUFFERS
Apparatus and methods for efficient queuing and dequeuing using segmented output buffers comprising sub-buffers and priority queues. Output buffers are monitored for empty sub-buffers. When a newly empty sub-buffer is discovered, a refill request is enqueued in a ranked priority queue wherein the rank of the destination priority queue is based on the number of empty-sub-buffers in the requesting output buffer. All high priority refill requests are dequeued before lower priority refill requests, thereby reducing the possibility of starvation. Optionally, by using simple dequeuing criteria, such as a FIFO discipline, instead of complex algorithms designed to improve fairness, system resources may be conserved thereby improving system throughput.
Latest Software Site Applications, Limited Liability Company Patents:
This application is a continuation of U.S. pending application ser. No. 10/927,801 filed Aug. 26, 2004, the entire disclosure of which is incorporated herein by reference.
FIELD OF INVENTIONInvention relates to apparatus and methods for reducing the risk of underflow in a multiple queue system.
BACKGROUND OF INVENTIONQueues are used in a variety of applications, especially where resources are shared. For example, queues may be used to manage traffic in routers, file and data servers or to schedule processes and/or threads in single and/or multi-processor systems. A variety of techniques may be used to prioritize resource requests in an effort to improve throughput or efficiency or to guarantee a quality of service (QoS). However, many of these techniques run the risk of starving some requesters. In order to address the issue of starvation, complex algorithms may be used to enqueue and/or de-queue resource requests based on a variety of parameters. However, when complex algorithms are used to reduce starvation, precious system resources are consumed by the increased calculations and/or delays are introduced, thereby reducing overall throughput.
What is needed is a method and apparatus for enqueing and dequeuing requests that reduces starvation based on an economical use of system resources.
SUMMARY OF INVENTIONThe present invention provides apparatus and methods for efficient queuing and dequeuing using segmented output buffers and priority queues. According to the current invention, each output buffer is segmented into a multiplicity of sub-buffers. A set of ranked priority queues are used to hold refill requests for the output buffers. Output buffers are monitored for empty sub-buffers. When a newly empty sub-buffer is discovered, a refill request is enqueued in a priority queue wherein the rank of the destination priority queue is based on the number of empty-sub-buffers in the requesting output buffer. By dequeuing refill requests in an order according to the rank of the priority queues, all high priority refill requests are dequeued before lower priority refill requests. In this way, the possibility of starvation is reduced because an output buffer's refill request will eventually be registered in the highest priority queue as it nears starvation.
According to an example of the current invention, a first-in first-out (FIFO) queuing discipline may be used to dequeue each priority queue. By using simple dequeuing criteria, such as a FIFO discipline instead of complex algorithms designed to improve fairness, system resources may be conserved thereby improving system throughput.
According to an example of the current invention, all sub-buffers may be the same size. Adding this constraint may improve the fairness of the current invention. In some cases, the items in the sub-buffers may be pointers or references and the size of the reference or pointer may not be related to the magnitude of the referenced items; in some cases, an additional constraint may be added wherein each referenced item has the same maximum magnitude.
According to the current invention, examples of the queuing apparatus and method according to the current invention may be used to enqueue/dequeue data requests, queries and/or transactions, to prioritize processes and/or threads and/or to route network traffic. Furthermore, examples of the queuing apparatus and method according to the current invention may be used to enqueue/dequeue physical items in a manufacturing, packaging and/or distribution system. In some cases, a system may use two or more queuing apparatus according to the current invention.
Advantageously, the current invention may be implemented in software, hardware and/or firmware. The current invention may also be used in physical processes or systems where resources are distributed to multiple destinations.
The rank of the destination priority queue (such as 101, 102, 103, 104 or 105) for the refill request is based on the number of empty sub-buffers (for example, 1b-e, 2c-2, 3e, 4d and 4e) in the segmented output buffer (1, 2, 3, or 4). In this example, priority queue 101 has the highest rank and holds the most urgent requests; priority queue 105 has the lowest rank and holds the least urgent requests. In this example, a request in the lowest rank priority queue 105 corresponds to the case where a single sub-buffer in a segmented output buffer is empty. A request corresponding to two empty sub-buffers in a segmented output buffer could been queued in priority queue 104; a request corresponding to three empty sub-buffers in a segmented output buffer would be enqueued in priority queue 103; a request corresponding to four empty sub-buffers would e enqueued in priority queue 102; a request corresponding to all five sub-buffers being empty would be enqueued in priority queue 101. The following examples illustrate the operation of this example of the current invention:
Example 1For example, when sub-buffer 3e became empty, segmented output buffer 3 held one empty sub-buffer (3e) and a refill request was enqueued in priority queue 105; slot 105c holds this refill request; if slot 3d were to become empty, leaving two empty sub-buffers (3d and 3e) in segmented output buffer 3, a refill request would then be registered in an empty slot (104d) in priority queue 104.
Example 2For example, when sub-buffer 4d became empty, segmented output buffer 4 held two empty sub-buffers (4d and 4e) and a refill request was enqueued in priority queue 104; slot 104a holds this refill request.
Example 3For example, when sub-buffer 2c became empty, segmented output buffer 2 held A empty sub-buffers (2c-e) and refill request was enqueued in priority queue 103; slot 103a holds this request.
In the example illustrated in
Queue servicer 118 services the ranked priority queues 110. Queue servicer 118 dequeues requests (such as 102a, 103a, 103b, 104a-c and 105a-d) from the priority queues (such as 101, 102, 103, 104 and 105) servicing the highest priority, non-empty priority queues before servicing any other queues. In this case, a simple FIFO queuing discipline per priority queue (101, 102, 103, 104 and 105) is used; for example, if requests were enqueued in priority queue 105 into slots 105a, 105b, 105c and 105d in that order, they will be dequeued and serviced in that same order (first 105a, second 105b, third 105c and fourth 105d). However, in other examples according to the current invention, other queuing disciplines may be used.
In some cases, the queue servicer 118 may correlate a particular segmented output buffer with a source for satisfying the requests associated with that segmented output buffer. For example, when deployed in a file server application, a queue servicer 118 may maintain a lookup table, have access to a lookup table or have access to a service that identifies an index into a file on a particular file server identifying the next chunk of data that should be served to a particular segmented output buffer (1, 2, 3 or 4). In this case, a refill request such as (such as 102a, 103a, 103b, 104a-c and 105a-d) enqueued in a priority queue (such as 101, 102, 103 104 or 105) may be as simple as an identifier referring to the segmented output buffer (1, 2, 3 or 4) with a newly empty sub-buffer (for example, 1b-e, 2c-2, 3e, 4d and 4e). However, in other examples of the current invention, the queue servicer may be less sophisticated and additional information may be incorporated into the request such as, but not limited to, the source for filling the request. For example, when used in a networking implementation, the source may be a buffer associated with an incoming port.
In the example illustrated in
In the file server example illustrated in
The current invention may be implemented in software, hardware and/or firmware. For example, the current invention may be implemented in-part in a single threaded or multi-threaded environment. For example, a queue servicer (such as queue servicer 118) may comprise one or more threads in a multi-threaded environment or be part of a larger process in a single threaded environment. Similarly, a buffer monitor (such as buffer monitor 116) may comprise one or more threads in a multi-threaded environment or be part of a larger process in a single threaded environment. In some cases, there may be multiple buffer monitors; for example, each segmented output buffer may have its own buffer monitor. In some cases, some or all of the invention may be implemented in one or more Field Programmable Gate Arrays (FPGAs) and/or Application Specific Integrated Circuits (ASICS). In some cases, the invention may be implemented in software using various data structures, such as circular queues, lists and/or stacks may be used to implement the segmented output buffers.
In the example illustrated in
Foregoing descriptions of specific embodiments of the invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Modifications and variations are possible in light of the above teaching. For example, applicants contemplate that present invention may be used in various software, firmware and/or hardware applications such as network routers, process queuers, storage area networks and/or unicast, broadcast and/or multicast systems as well as industrial and/or manufacturing applications.
The embodiments were chosen and described in order to explain the principles and the application of the invention, thereby enabling others skilled in the art to utilize the invention in its various embodiments and modifications according to the particular purpose contemplated. The scope of the invention is intended to be defined by the claims appended hereto and their equivalents.
Claims
1. A method for efficient queuing, the method comprising:
- providing at least one output buffer wherein each output buffer comprises a multiplicity of sub-buffers;
- providing a plurality of priority queues for holding refill requests for the output buffers wherein the priority queues are ranked from highest priority to lowest priority;
- monitoring the output buffers for empty sub-buffers and establishing a refill request in one of the priority queues when at least one newly empty sub-buffer is discovered wherein the rank of the destination priority queue for the refill request is calculated according to the number of empty sub-buffers in the output buffer; and,
- dequeuing refill requests in an order according to the rank of the priority queues, thereby dequeuing all requests in the highest rank priority queue before dequeuing any requests of any other priority.
2. The method of claim 1 further comprising the step of servicing refill requests after they are dequeued, thereby filling empty sub-buffers.
3. The method of claim 1 wherein the step of dequeuing refill requests further comprises using a first-in first-out (FIFO) queuing discipline per priority queue.
4. The method of claim 1 wherein each output buffer has the same number of sub-buffers.
5. The method of claim 1 wherein each sub-buffer is the same size.
6. The method of claim 1 wherein each sub-buffer holds a reference to an item and wherein each referenced item has the same maximum magnitude.
7. The method of claim 1 wherein each sub-buffer holds items selected from the list comprising:
- data, pointers, process identifiers, thread identifiers, queries, transactions, transaction identifiers or network packets or any combination thereof.
8. The method of claim 1 wherein each request comprises an output buffer identifier.
9. An efficient computer queuing system:
- at least one output buffer wherein each output buffer comprises a plurality of sub-buffers;
- a plurality of priority queues for holding refill requests for the output buffers wherein the priority queues are ranked from highest priority to lowest priority;
- a buffer monitor for monitoring the output buffers for empty sub-buffers and establishing refill requests in the priority queues wherein the rank of the destination priority queue for the requests are based on the number of empty sub-buffers in the output buffer; and,
- a queue servicer for dequeuing refill requests in the priority queues in an order according to the rank of the priority queues, thereby dequeuing all refill requests in the highest rank priority queue before dequeuing any refill requests in a lower rank priority queue.
10. A computer system for efficient data queuing comprising:
- a queuing system comprising: at least one output buffer wherein each output buffer comprises a multiplicity of sub-buffers; a plurality of priority queues for holding refill requests for the output buffers wherein the priority queues are ranked from highest priority to lowest priority; a buffer monitor for monitoring the output buffers for empty sub-buffers and establishing refill requests in the priority queues wherein the rank of the destination priority queue for the requests are based on the number of empty sub-buffers in the output buffer; and, a queue servicer for dequeuing and servicing refill requests in the priority queues in an order according to the rank of the priority queues, thereby dequeuing all refill requests in the highest rank priority queue before dequeuing any refill requests in a lower rank priority queue; and, one or more interfaces for coupling said queuing apparatus to targets and sources wherein a target may be coupled to an outgoing queue enabling a target to receive items from an output buffer and wherein a source may provide items for servicing data refill requests.
Type: Application
Filed: Sep 2, 2008
Publication Date: Dec 25, 2008
Applicant: Software Site Applications, Limited Liability Company (Wilmington, DE)
Inventor: Pierre Seigneurbieux (San Jose, CA)
Application Number: 12/203,032
International Classification: H04L 12/56 (20060101);