Video Aware Traffic Management
A receiver for generating an video output from a stream of data packets includes circuitry for decoding the stream of packets into a video signal, circuitry for generating video frames from the video signal, circuitry for detecting whether a missing packet is associated with a video frame of a first type and circuitry for selectively requesting retransmission of a missing packet responsive to the detecting circuitry. The decoding circuitry further comprises circuitry for concealing errors using error recovery without requesting retransmission due to missing frames of the first type
Latest ALCATEL USA SOURCING, L.P. Patents:
The present U.S. Utility Patent Application claims priority pursuant to 35 U.S.C. §120, as a divisional, to U.S. Utility patent application Ser. No. 11/337,372, entitled “Video Aware Traffic Management,” (Attorney Docket No. 139444), filed Jan. 23, 2006, pending, which is hereby incorporated herein by reference in its entirety and made part of the present U.S. Utility Patent Application for all purposesSTATEMENT OF FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
The U.S. Government has a paid-up license in this invention and the right in limited circumstances to require the patent owner to license others on reasonable terms as provided for by the terms of Award No. 70NANB3H3053 awarded by National Institute of Standards and Technology.BACKGROUND OF THE INVENTION
1. Technical Field
This invention relates in general to network communications and, more particularly, to a method and apparatus for discarding packets.
2. Description of the Related Art
In a digital information delivery network, between a source device and a destination device, packets of data may be lost for a variety of reasons. Some packets are randomly lost due to uncontrollable errors—for example, errors caused by noise on a transmission line, synchronization issues, etc. Some packets are lost due to congestion, i.e., it is not possible for a network element to transmit all received packets in a timely manner. Current discard mechanisms for IP QoS (quality of service) algorithms implement random selection schemes to determine which packets to discard without regard to the relative effect on the eventual output.
For some data transfer protocols, missing packets cause the destination device to request a retransmission of the missing information. This is not very feasible, however, in a network that has multicasting of real-time streams such as audio or video. Normally, there will not be enough time available for requesting and receiving the retransmitted packets, unless buffers at the destination device are very large.
When an expected packet in a packet stream is not received at the destination device, the destination device waits for a certain amount of time before declaring a packet as lost. Once a packet is declared as lost, some decoders may request retransmission, other decoders may correct the problem to the extent possible by error concealment techniques. Error concealment techniques will in most cases result in degradation of output quality and are incapable of correcting some errors; further, the degree of the output error will be different depending upon the type of data in the lost packet, some of which will be more difficult to conceal than others. Thus, if packets must be discarded, some types of packets will be better candidates for discarding than others.
Accordingly, there is a need for a method and apparatus for identifying and discarding packets to minimize output errors.BRIEF SUMMARY OF THE INVENTION
In a first aspect of the present invention, a receiver for generating an video output from a stream of data packets comprises circuitry for generating video frames from the packets and circuitry for decoding the stream of packets into a video signal, where the decoding circuitry includes circuitry for concealing errors due to missing frames of a first type. When a missing packet is detected, the receiver selectively conceals the error or requests retransmission, based on whether the missing packet is of said first type.
This aspect of the present invention provides for superior receiving performance by concealing errors due to missing or corrupt low priority video frames and requesting retransmission only when high priority video frames are missing or corrupt.
For a more complete understanding of the present invention, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:
The present invention is best understood in relation to
In operation, the VHE sources 20 stream video information to the IP video receivers 22. For live video broadcasts, such as a live television signal, the video data is typically sent as a multicast transmission. For on-demand video, unicast transmission may be used. At the receiver side, on-demand video generally has a longer buffer, since the delay from source 20 to viewing is not important as broadcast video servers and, thus, on-demand video has a lower priority than live broadcast video services. The site 12 may have several IP video receivers 22 each receiving multiple streams of programming. For example, each IP video receiver 22 could receive two video data streams. If there were three IP video receivers 22 in the site 12, and each receiver 22 was receiving two video streams, then the link 31 between the multiplexer 30 and the modem 32 would be carrying video packets for six different data streams.
Modern day video protocols compress the video stream by periodically sending a full frame (compressed) of video data, followed by differential frames which indicate the changes between frames, rather than the frame itself. Accordingly, a scene which has a rapidly changing image will require a higher bandwidth than a frame that is relatively still. The total available bandwidth between the video heads 20 and the IP receivers 22 for a site 12 is generally fixed by the bandwidth of link 31, in view of the technology used by the multiplexer 30 and modem 32.
With a fixed bandwidth in which to transfer all packets for all data streams for a site 12, the number of data streams supported by the link 31 is determined by an average bandwidth for each received channel (link 31 can be carrying other data traffic such as Internet traffic, which has a lower priority than the live video data streams (lowest priority) and voice (VOIP—voice over Internet protocol, which generally has the highest priority). However, the data rates for the separate N data flows are not constant. At times, multiple channels may be simultaneously using more than their average bandwidth, resulting in congestion on link 31.
The congestion problem is illustrated in
In operation, the multiplexer 30 is designed to minimize the effect of dropping packets. A critical aspect of the problem is that all packets are time-critical. For each data stream all packets are generated on a single server (VHE 20). Once generated, each packet has a strict “use by” time. A packet that becomes stale in transit to the end user becomes unusable. To conserve shared link bandwidth, stale packets must be discarded without being transmitted over the link 31.
In operation, multiplexer 30 conforms to a policy that requires the minimum degradation of service to the end user when packets are discarded. This goal is accomplished in two basic ways: (1) the multiplexer 30 discards the minimum amount of data necessary to avoid congestion and (2) the multiplexer 30 makes use of a priority scheme to ensure the least useful packets are preferentially discarded.
The traffic management system 46 is split into queue entry logic 50, dequeue logic 52, channel change logic 54 and forward prediction logic 56. Each priority level has a threshold level in the FIFO 42, i.e., a P00 (“Priority 00”) threshold, a P01 threshold, a P10 threshold and a P11 threshold. Additionally, there is an Initial Hold-off threshold. When a threshold level is exceeded, a flag is set (a “P00 FG” notation is used to represent the flag from priority “00”). It is assumed that the thresholds are based on a time-to-dequeue statistic. In other words, if the P00 threshold is set to 50 msec, it is exceeded if there are packets in the queue which will not be dequeued within 50 msec. Since there may be packets in the FIFO 42 that will not be transmitted, the physical location of a packet may not be indicative of whether a threshold level has been exceeded.
In the illustrated embodiment, a single FIFO 42 is used for multiple channels (multiple data streams). In the preferred embodiment, the low priority flags, P00 FG and P01 FG, are maintained on a global basis, i.e., one flag is used to indicate that a packet has exceeded a threshold, regardless of the channel associated with that packet. The higher priority flags, P10 FG and P11 FG, are maintained on a per channel basis; for example, if a packet on channel “1” exceeds the “10” threshold, the P10 flag is set for channel “1”, but not for channel “2” (in the illustrated embodiment, only two channels are shown, although an actual embodiment may support more channels).
For background purposes,
If in step 66, the P01 threshold is exceeded, then the P01 flag is set in step 72. The queue entry logic 50 determines if queuing the packet in FIFO 42 will result in the priority threshold P10 for the associated channel being exceeded in step 74. If the priority threshold P10 threshold for the channel is not exceeded in step 74, the queue entry logic 50 determines whether the packet is a P00 or a P01 packet in step 76. If so, it is discarded in step 70.
If in step 74, the P10 threshold is exceeded, then the P10 flag is set in step 78. Queue entry logic 50 determines if queuing the packet in FIFO 42 will result in the priority threshold P11 for the associated channel being exceeded in step 80. If the priority threshold P11 threshold for the channel is not exceeded in step 80, the queue entry logic 50 determines whether the packet is a P00, a P01 or a P10 packet in step 82. If so, it is discarded in step 70.
If in step 80, the P11 threshold is exceeded, then the P11 flag is set in step 84. Queue entry logic 50 determines whether the FIFO 42 is full in step 86. If so, the packet is discarded in step 70. If the FIFO is not full, then the queue entry logic 50 determines whether the packet is a P11 packet in step 88. If not, it is discarded in step 70.
If the P00 threshold is not exceeded in step 62 or if the packet is determined not to be a P00 packet in step 68, or not to be a P00/P01 packet in step 76 or not to be a P00/P01/P10 packet in step 82, or is determined to be a P11 packet in step 88, then it is checked to see if it is a fragment of a frame which has had packets previously discarded in step 92. If so, it is discarded in step 70; if not, it is added to the queue in step 94.
After a packet is discarded in step 70, the queue entry logic 50 determines whether it is a fragment of a larger frame in step 96. If so, the frame ID is saved in step 98 to match with other fragments from the same frame.
It should be noted that the flags are reset upon receiving n packets during which the condition for setting the flag no longer exists. The value n is a configurable value.
If the P00 flag is set in step 106, then the packet will be discarded if it is a P00 packet (step 112). If the packet has a priority higher than P00 in step 112, the dequeue logic 52 will determine whether the P01 flag is set in step 114. If the P01 flag is set in step 114, then the packet will be discarded if it is a P01 packet (step 116). If the P01 flag is not set in step 114, the packet will be dequeued (step 108). If it is higher then a P01 packet in step 116, the dequeue logic 52 will determine whether the P10 flag is set (for the channel associated with the packet) in step 118. If the P10 flag for the channel is set in step 118, then the packet will be discarded if it is a P10 packet (step 120). If the P01 flag is not set in step 114, the packet will be dequeued (step 108). If the packet has a priority higher P10 packet in step 120, the dequeue logic 52 will determine whether the P11 flag is set (for the channel associated with the packet) in step 122. If the P11 flag for the channel is set in step 122, then the packet will be discarded. If the P11 flag is not set, the packet will be dequeued.
Referring again to
In step 130, the next packet is taken from the head of the FIFO 42. If it is associated with the “from” channel in step 132, it is discarded in step 134. If it is not associated with the “from” channel, but is associated with the “to” channel in step 136, the packet is discarded if it is a low priority packet (P00 or P01) in step 138. If it is a high priority packet in step 138, then the channel-change clearing process is complete in step 140.
Referring again to
In the embodiment described above, FIFO thresholds are used to keep packets from entering the queue based on the threshold exceeded and a priority associated with the packet. This embodiment provides a method of passing high priority packets using a minimum amount of computation resources. A second embodiment is described below which operates in a different manner. When the buffer is full (i.e., when new packets will not reach the end of the FIFO buffer within the predetermined time limit), packets within the FIFO are marked for discard within the FIFO. When these marked packets reach the head of the FIFO, they are simply not passed forward for transfer over link 31.
In this embodiment, packets already in the video queue can be marked for discard. Discarding enqueued packets results in faster video queue space recovery and can contribute to faster channel change support. This embodiment assumes that the video data streams are generated by an encoder or video server 20 that follows a set of rules, or a protocol, for transport of compressed video content on an IP packet network. More than one protocol definition may be accommodated. The protocols define how packet headers are assembled, and how video content priority indicators are coded at the application layer. The embodiment further assumes that the video data transport data rate is within the range defined for a particular network implementation. In addition, this embodiment assumes that a maximum packet size is defined at the application layer such that fragmentation at the lower layers will never be required. This is to ensure that every video packet entering the access node contains the video component priority indicators.
In operation, enqueue microblock 160 and dequeue microblock 162 control the flow of packets into and out of the multiplexer 30 and maintain the contents of the pseudo video queue 150 and video metatdata buffer 156. As video packets are received, they are stored in the physical video buffer 158. Each packet in the physical video buffer 158 has its metadata stored in an associated entry of the video metadata buffer 156. The metadata information is used for further packet processing. If multiple receivers 22 are subscribed to the same channel, multiple metadata will exist in the video metadata buffer 156 for the same video stream. The video metadata buffer 156 is preferably a FIFO queue of a predetermined finite depth that maintains the metadata in the order of the video packets.
When congestion is detected (i.e., the time between receiving a packet and transmitting the same packet exceeds a predetermined threshold), or if the physical video buffer 158 is full, packets within the physical video buffer 158 are marked for discard (when a packet marked for discard reaches the front of the physical video buffer 158, it will be removed without further transmission on the link 31). If there are no currently enqueued packets within the physical video buffer 158 that can be dropped to make room of the incoming packet, then the incoming packet will be dropped without enqueue.
The pseudo video buffer 150 is used to identify and mark packets for discard. The pseudo video buffer 150 uses circular buffers as its main data structure with head and tail pointers such that a new buffer entry is added at the tail and buffer entries are removed from the head. As shown below, this circular list data structure provides a simple mechanism to maintain the list.
The pseudo video buffer 150 includes a forward/drop list 154 and an index list 152 for each priority type. Each entry in the forward/drop list 154 is associated with an entry in the video metadata buffer 156. The contents of each entry in the forward/drop list 154 is either an indicator of the data stream (either “0” or “1” in the illustrated embodiment), if the packet is to be forwarded, or a discard marker (“D”) to indicate that the associated packet is to be dropped. Each priority index list 152 maintains an index of packets by priority. By maintaining a separate list of packet for each priority, packets or metadata of a certain priority can be easily located for marking without scanning the entire queue.
If the buffer is congested in step 172, the enqueue microblock looks at a index list 152 associated with lower priority packets (i.e., if the incoming packet is a P1 packet, the P0 index list will be used to determine whether there are lower priority packets in the physical video queue 158). If the appropriate index list 152 is empty in step 180, the incoming packet is discarded (not enqueued) in step 182. On the other hand, if the appropriate index list 152 is not empty in step 180, the lower priority packets designated in the index list 152 are marked for discard in the forward/drop list 154 to create additional space in the physical video buffer in steps 184-188. In the preferred embodiment, the enqueue process will only mark packets for discard until enough room is recovered to enqueue the incoming packet. In step 184, a packet is identified by an entry from the appropriate index list 152; the index in that entry points to a corresponding entry in the forward/drop list. That entry is marked for discard (by a “D” in the illustrated embodiment) in step 186. The entry is then deleted from the index list 152 and the queue level is adjusted to account for the discarded packet. Control continues at step 172, where it is determined whether the queue has room for the incoming packet after discarding the packet. If so, the incoming packet is enqueued in steps 174-178. If more space is needed to enqueue the incoming packet, the index lists are again checked for lower priority packets within the queue. The process is repeated until either enough room is obtained by discarding lower priority packets or, if no more room can be created, by discarding the incoming packet.
The operation of the dequeue microblock 162 is shown in
The embodiment of the invention described in
In either embodiment described herein, the receivers may be faced with lost packets.
Although the Detailed Description of the invention has been directed to certain exemplary embodiments, various modifications of these embodiments, as well as alternative embodiments, will be suggested to those skilled in the art. The invention encompasses any modifications or alternative embodiments that fall within the scope of the Claims.
1. A receiver for generating an video output from a stream of data packets, comprising:
- circuitry for decoding the stream of packets into a video signal;
- circuitry for generating video frames from the video signal;
- circuitry for detecting whether a missing packet is associated with a video frame of a first type; and
- circuitry for selectively requesting retransmission of a missing packet responsive to the detecting circuitry;
- wherein said decoding circuitry further comprises circuitry for concealing errors using error recovery without requesting retransmission due to missing frames of the first type.
2. The receiver of claim 1 wherein the detecting circuitry further comprises circuitry for determining a position of a video frame associated with a missing packet within an order of received frames.
3. The receiver of claim 1 further comprising:
- circuitry for detecting whether a missing packet is associated with a video frame of a second type.
4. The receiver of claim 3 wherein the second type is an I-frame or a video anchor frame.
5. The receiver of claim 3 wherein the requesting retransmission circuitry further comprises circuitry for requesting retransmission of said missing packet when said missing packet is associated with a video frame of the second type.
6. A method for generating a video output from a stream of data packets in a receiver, comprising:
- decoding the stream of packets into a video signal;
- generating video frames from the video signal;
- upon determining that a packet is missing from the stream, detecting a type of video frame associated with the missing packet and responsive to the type, selectively: concealing errors using error recovery without requesting retransmission due to missing frames of a first type; or requesting retransmission of a missing packet.
7. The method of claim 6 wherein the detecting step comprises the step of determining a position of a video frame associated with a missing packet within an order of received frames.
8. The method of claim 6 further comprising:
- detecting whether a missing packet is associated with a video frame of a second type.
9. The method of claim 8 wherein the second type is an I-frame or a video anchor frame.
10. The method of claim 8 wherein the requesting retransmission step further comprises requesting retransmission of said missing packet when said missing packet is associated with a video frame of the second type.
International Classification: H04N 11/02 (20060101);