Abstract: A technique is disclosed that can efficiently control congestion, while supporting heterogeneity for streaming data among multiple computers in a network. A plurality of nodes is divided into a plurality of distribution trees within a computer network, wherein the data is divided into a plurality of prioritized layers. When a node experiences packet loss, the location of the congestion is inferred. If the congestion is at or near the outgoing link, outgoing traffic is shed to alleviate the congestion by shedding child node(s) receiving descriptions in the least important layer of data that the child node(s) are receiving. Similarly, if the congestion is at or near the incoming link, incoming traffic is shed by shedding parent nodes that are sending descriptions in the least important layer of data that the node is receiving. Nodes with available bandwidth are further instructed to subscribe to additional descriptions.

Abstract: An “adaptive audio playback controller” operates by decoding and reading received packets of an audio signal into a signal buffer. Samples of the decoded audio signal are then played out of the signal buffer according to the needs of a player device. Jitter control and packet loss concealment are accomplished by continuously analyzing buffer content in real-time, and determining whether to provide unmodified playback from the buffer contents, whether to compress buffer content, stretch buffer content, or whether to provide for packet loss concealment for overly delayed or lost packets as a function of buffer content. Further, the adaptive audio playback controller also determines where to stretch or compress particular frames or signal segments in the signal buffer, and how much to stretch or compress such segments in order to optimize perceived playback quality.

Abstract: An adaptive “temporal audio scaler” is provided for automatically stretching and compressing frames of audio signals received across a packet-based network. Prior to stretching or compressing segments of a current frame, the temporal audio scaler first computes a pitch period for each frame for sizing signal templates used for matching operations in stretching and compressing segments. Further, the temporal audio scaler also determines the type or types of segments comprising each frame. These segment types include “voiced” segments, “unvoiced” segments, and “mixed” segments which include both voiced and unvoiced portions. The stretching or compression methods applied to segments of each frame are then dependent upon the type of segments comprising each frame. Further, the amount of stretching and compression applied to particular segments is automatically variable for minimizing signal artifacts while still ensuring that an overall target stretching or compression ratio is maintained for each frame.

Abstract: A “speech onset detector” provides a variable length frame buffer in combination with either variable transmission rate or temporal speech compression for buffered signal frames. The variable length buffer buffers frames that are not clearly identified as either speech or non-speech frames during an initial analysis. Buffering of signal frames continues until a current frame is identified as either speech or non-speech. If the current frame is identified as non-speech, buffered frames are encoded as non-speech frames. However, if the current frame is identified as a speech frame, buffered frames are searched for the actual onset point of the speech. Once that onset point is identified, the signal is either transmitted in a burst, or a time-scale modification of the buffered signal is applied for compressing buffered frames beginning with the frame in which onset point is detected. The compressed frames are then encoded as one or more speech frames.

Abstract: Systems and methods for improved streaming of variable bit rate multimedia content are described. The described systems and methods solve some streaming problems by streaming the variable bit rate content at a constant bit rate that is negotiated between a server and a client. As a result, a transmission channel from server to client is utilized in an efficient manner. If the client is able to buffer the streamed content (to a hard drive, for example) the content can be streamed at a rate that is significantly less than its peak bit rate, even at a rate that is less than its average bit rate.

Abstract: A system and process according to the present invention involves tagging prescribed portions of the data of each layer in a layered multicast or layered presentation with an indicator of the importance or utility that the data provides to the receiver. Additionally, the data is tagged with a cost factor involved with sending the data. The aforementioned portions of the data can be an entire data stream of a layer, or some part thereof all the way down to the individual packets making up the stream. The invention also involves determining the optimized scenario for sending the data from the sender to the receiver based on the data tags.

Abstract: A system and method for distributing streaming content data from a server to multiple clients enables the server to handle transient surges of requests by having the clients cooperate with the server and other clients to distribute content, thereby alleviating the load on the server. The server divides the streaming content into multiple sub-streams (e.g., by using multiple description coding), and constructs multiple distribution trees with itself at the root and each client as a node in each of the trees. Each sub-stream is transmitted down a corresponding one of the distribution trees. Clients that receive sub-streams from its parent nodes in the distribution trees in turn forward the sub-streams to their child nodes in the trees.

Abstract: Systems and methods for presenting time-varying multimedia content are described. In one aspect, a lower quality data stream for an initial portion of the multimedia content is received. The lower quality data stream is received at a rate faster than a real-time playback rate for the multimedia content. The lower quality data stream was encoded at a bit rate below a transmission rate. A higher quality data stream of a subsequent portion of the multimedia content is received. The higher quality data stream was encoded at a bit rate that equals the transmission rate. The initial portion and the subsequent portion of the multimedia content are presented at the real-time playback rate. Receiving the initial portion faster than the real-time playback rate provides for a reduction of latency due to buffering by a desired amount.

Abstract: A two-dimensional (2D) image model models the layout structure of a class of document images as an image grammar and includes production rules having explicit layout parameters as data items that indicate information about the spatial relationships among image constituents occurring in images included in the class. The parameters are explicitly represented in the grammar rules in a manner that permits them to be automatically trained by a training operation that makes use of sample document images from the class of modeled documents. After each sample image is aligned with the 2D grammar, document-specific measurements about the spatial relationships between image constituents are taken from the image. Optimal values for the layout parameters are then computed from the measurement data collected from all samples.

Abstract: A transmission method for video image data using an embedded bit stream in a hierarchical table-lookup vector quantizer comprises the steps encoding an image using hierarchical vector quantization and an embedding process to obtain an embedded bit stream for lossless transmission. The bit stream is selectively truncated and decoded to obtain a reconstructed image.

Abstract: To obtain real-time responses with interactive multimedia servers, the server provides at least two different audio/visual data streams. A first data stream has fewer bits per frame and provides a video image much more quickly than a second data stream with a higher number of bits and hence higher quality video image. The first data stream becomes available to a client much faster and may be more quickly displayed on demand while the second data stream is sent to improve the quality as soon as the playback buffer can handle it. In one embodiment, an entire video signal is layered, with a base layer providing the first signal and further enhancement layers comprising the second. The base layer may be actual image frames or just the audio portion of a video stream. The first and second streams are gradually combined in a manner such that the playback buffer does not overflow or underflow.

Abstract: Correction of errors and losses in a receiver-driven layered multicast (RLM) of real-time media over a network is augmented using one or more layers of error correction information. Each receiver separately optimizes the quality of received information by subscribing to at least one error correction layer. Ideally, each source layer in a RLM has one or more associated multicasted error correction data streams. Each error correction layer contains information for replacing lost packets from the associated source layer. More than one error correction layer is proposed to correct for lost packets in other error correction layers. Error correction streams are preferably generated using a pseudo-Automatic Repeat Request (ARQ) wherein a broadcaster sends both the source packets in a primary stream and delayed versions thereof in one or more redundant streams. A hybrid technique combines the psuedo-ARQ method with an adaptation of Forward Error Correction (FEC) techniques.

Abstract: A method and system including an improved generalized reference decoder that operates according to any number of sets of rate and buffer parameters for a given bit stream. Each set characterizes a leaky bucket model and contains three parameters representing the transmission bit rate, buffer size, and initial decoder buffer fullness. An encoder provides at least two sets of these parameters, whereby the decoder selects one or interpolates between them to operate at any desired peak bit rate, buffer size or delay. The generalized reference decoder may select the smallest buffer size and corresponding delay that decodes the bit stream without buffer underflow or overflow, or alternatively may select and operate at the minimum required peak transmission rate, or something between both. In practice, the buffer size, delay and/or the peak transmission rate can be reduced by significant factors, and/or the signal-to-noise ratio (SNR) can be increased.

Abstract: “Correction of errors and losses occurring during a receiver-driven layered multicast (RLM) of real-time media over a heterogeneous packet network such as the Internet is accomplished by augmenting RLM with one or more layers of error correction information. Each receiver separately optimizes the quality of received audio and video information by subscribing to at least one error correction layer. Ideally, each source layer in a RLM would have one or more associated multicasted error correction data streams (i.e., layers). Each error correction layer contains information that can be used to replace lost packets from the associated source layer. More than one error correction layer is proposed as some of the error correction packets contained in the data stream needed to replace the packets lost in the associated source stream may themselves be lost in transmission.

Abstract: A projection onto convex sets (POCS)-based method for consistent reconstruction of a signal from a subset of quantized coefficients received from an N×K overcomplete transform. By choosing a frame operator F to be the concatenization of two or more K×K invertible transforms, the POCS projections are calculated in RK space using only the K×K transforms and their inverses, rather than the larger RN space using pseudo inverse transforms. Practical reconstructions are enabled based on, for example, wavelet, subband, or lapped transforms of an entire image. In one embodiment, unequal error protection for multiple description source coding is provided. In particular, given a bit-plane representation of the coefficients in an overcomplete representation of the source, one embodiment of the present invention provides coding the most significant bits with the highest redundancy and the least significant bits with the lowest redundancy.

Abstract: A projection onto convex sets (POCS)-based method for consistent reconstruction of a signal from a subset of quantized coefficients received from an N×K overcomplete transform. By choosing a frame operator F to be the concatenization of two or more K×K invertible transforms, the POCS projections are calculated in RK space using only the K×K transforms and their inverses, rather than the larger RN space using pseudo inverse transforms. Practical reconstructions are enabled based on, for example, wavelet, subband, or lapped transforms of an entire image. In one embodiment, unequal error protection for multiple description source coding is provided. In particular, given a bit-plane representation of the coefficients in an overcomplete representation of the source, one embodiment of the present invention provides coding the most significant bits with the highest redundancy and the least significant bits with the lowest redundancy.

Abstract: The production of an interleaved multimedia stream for servers and client computers coupled to each other by a diverse computer network which includes local area networks (LANs) and/or wide area networks (WANs) such as the internet. Interleaved multimedia streams can include compressed video frames for display in a video window, accompanying compressed audio frames and annotation frames. In one embodiment, a producer captures separate video/audio frames and generates an interleaved multimedia file. In another embodiment, the interleaved file include annotation frames which provide either pointer(s) to the event(s) of interest or include displayable data embedded within the annotation stream. The interleaved file is then stored in the web server for subsequent retrieval by client computer(s) in a coordinated manner, so that the client computer(s) is able to synchronously display the video frames and displayable event(s) in a video window and event window(s), respectively.

Abstract: A transmission method for video image data using an embedded bit stream in a hierarchical table-lookup vector quantizer comprises the steps encoding an image using hierarchical vector quantization and an embedding process to obtain an embedded bit stream for lossless transmission. The bit stream is selectively truncated and decoded to obtain a reconstructed image.

Abstract: A transmission method for video image data using an embedded bit stream in a hierarchical table-lookup vector quantizer comprises the steps encoding an image using hierarchical vector quantization and an embedding process to obtain an embedded bit stream for lossless transmission. The bit stream is selectively truncated and decoded to obtain a reconstructed image.

Abstract: The production of an interleaved multimedia stream for servers and client computers coupled to each other by a diverse computer network which includes local area networks (LANs) and/or wide area networks (WANs) such as the internet. Interleaved multimedia streams can include compressed video frames for display in a video window, accompanying compressed audio frames and annotation frames. In one embodiment, a producer captures separate video/audio frames and generates an interleaved multimedia file. In another embodiment, the interleaved file include annotation frames which provide either pointer(s) to the event(s) of interest or include displayable data embedded within the annotation stream. The interleaved file is then stored in the web server for subsequent retrieval by client computer(s) in a coordinated manner, so that the client computer(s) is able to synchronously display the video frames and displayable event(s) in a video window and event window(s), respectively.