SYSTEMS, METHODS, AND COMPUTER PRODUCTS FOR ELEMENTARY STREAMS BROADCASTING
The systems, methods, and computer program products described herein provide for the streaming data from a server to a client without the use of a container. The server receives, from the client, a request for data. In response, the server prepares and sends metadata to the client describing the requested data. The server then wraps one or more frames of the requested data for transmission to the client. In an embodiment, the wrapping process includes the appending of header data to the one or more frames. The header data includes a frame size field, a stream identifier field, and a presentation timestamp (PTS) field.
Latest Rovi Corporation Patents:
- Contextual search by a mobile communications device
- SYSTEMS AND METHODS FOR CONVERTING INTERACTIVE MULTIMEDIA CONTENT AUTHORED FOR DISTRIBUTION VIA A PHYSICAL MEDIUM FOR ELECTRONIC DISTRIBUTION
- SYSTEMS AND METHODS FOR ELECTRONIC SELL-THROUGH OF INTERACTIVE MULTIMEDIA CONTENT AUTHORED FOR DISTRIBUTION VIA PHYSICAL MEDIA
Current computing and communications systems are often used for the transmission and receipt of multimedia information, including video and audio data. To enable such data transfers, the hypertext transfer protocol (HTTP) may be used. Existing HTTP streaming protocols provide for progressive downloads of such data. Using this mechanism, multiplexed data or a single elementary stream may be transferred.
Such a system includes a number of drawbacks. For example, data overhead may be significant. This is due, in part, to the use of a container or other metafile that specifies how data elements and metadata are formatted. The use of a container requires bandwidth, which in turn reduces throughput in a channel. The use of a container also increases storage requirements at a client.
What is needed, therefore, is a system that provides for the transmission of multimedia data from a server to a client using less bandwidth and requiring less storage at the client.
In the drawings, the leftmost digit(s) of a reference number identifies the drawing in which the reference number first appears.
DETAILED DESCRIPTIONAn embodiment is now described with reference to the figures, where like reference numbers indicate identical or functionally similar elements. While specific configurations and arrangements are discussed, it should be understood that this is done for illustrative purposes only. A person skilled in the relevant art will recognize that other configurations and arrangements can be used without departing from the spirit and scope of the description. It will be apparent to a person skilled in the relevant art that this can also be employed in a variety of other systems and applications other than what is described herein.
The systems, methods, and computer program products described herein provide for the streaming data from a server to a client without the use of a container. The server receives, from the client, a request for data. In response, the server prepares and sends metadata to the client describing the requested data. The server then wraps one or more frames of the requested data for transmission to the client. In an embodiment, the wrapping process includes the appending of header data to the one or more frames. The header data includes a frame size field, a stream identifier field, and a presentation timestamp (PTS) field.
The interaction between the client and server is illustrated as a process flowchart in
Processing at the client is illustrated in
In an embodiment, the construction of the request message at the client (310 of
Processing at the server is illustrated in
The server may then begin preparation of a response. At 540, headers for the response may be constructed. Example headers for a response will be described in greater detail below with respect to
At 570, requested data is wrapped by attaching the headers constructed at 540 to the requested data. At 580, metadata related to the requested data is prepared and sent to the client. The metadata includes media identification information and stream type, e.g., “media 0, AVC”, or “media 1, AAC”. In an embodiment, the metadata may also include codec configuration information for use of the client, to facilitate client initialization. The metadata may also include other stream qualities associated with the session. The format of the metadata is specified in a header of the metadata message. The metadata may be formatted in any of several ways. For example, the metadata may be formatted according to the Session Description Protocol (SDP). Alternatively, the metadata may be formatted using Extensible Mark-up Language/Synchronized Multimedia Integration Language (XML/SMIL) descriptors.
At 590, the response is sent to the client, including the wrapped requested data in the form of one or more frames.
An example frame is illustrated in
Given the data provided to the client through metadata 140 and headers 610 through 630, it is not necessary to provide a separate container metafile to the client. In an embodiment, this may permit a savings of 20 to 30% in data overhead.
The system and processing described herein allow for adaptive streaming, where the client may request that changes be made in bit rate and in quality of the stream of requested data. The processing of such request is illustrated in
The client-server interaction used in adaptive streaming is illustrated in
Processing at the client for adaptive streaming is described in
The construction of the PROP-SET message at 910 may include the incorporation of several parameters relating to the requested data. This is illustrated in
Processing of an adaptive streaming request by the server is illustrated in
At 1160, metadata is prepared and sent to the client, where the metadata describes the requested data. The metadata includes media identification information and stream type, e.g., “media 0, AVC”, or “media 1, AAC”. In an embodiment, the metadata may also include codec configuration information for use of the client, to facilitate client initialization. The metadata may also include other stream qualities associated with the session. The format of the metadata is specified in a header of the metadata message. The metadata may be formatted in any of several ways, e.g., according to the SDP or using XML/SMIL descriptors.
At 1170, the requested data, i.e., the stream with properties modified according to the PROP-SET message, is sent to the client in the form of a response. As before, a container is not used in the sending of the requested data.
The system and processing described herein allow for a seeking operation, where the client may request that the server jump to a particular point in a stream. The processing of such request is illustrated in
The client-server interaction used in seeking is illustrated in
Processing at the client in this context is illustrated in
The construction of the PROP-SET message at the client includes the incorporation of particular parameters relating to the stream. In an embodiment in which the client and server communicate using HTTP, these parameters may be incorporated in the form of headers. Such an embodiment is illustrated in
The server's processing of the client's request to seek is illustrated in
Methods and systems are disclosed herein with the aid of functional building blocks illustrating the functions, features, and relationships thereof. At least some of the boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries may be defined so long as the specified functions and relationships thereof are appropriately performed.
One or more features disclosed herein may be implemented in hardware, software, firmware, and combinations thereof, including discrete and integrated circuit logic, application specific integrated circuit (ASIC) logic, and microcontrollers, and may be implemented as part of a domain-specific integrated circuit package, or a combination of integrated circuit packages. The term software, as used herein, refers to a computer program product including at least one computer readable medium having computer program logic stored therein to cause a computer system to perform one or more features and/or combinations of features disclosed herein. The computer readable medium may be transitory or non-transitory. An example of a transitory computer readable medium may be a digital signal transmitted over a radio frequency or over an electrical conductor, through a local or wide area network, or through a network such as the Internet. An example of a non-transitory computer readable medium may be a compact disk, a flash memory, or other data storage device.
A server that performs the processing described above may incorporate a programmable computing system. Such a computing system executes software/firmware embodying the above processing. Such a system is shown in
Computing program logic 1750 may include parsing logic 1750, which is responsible for parsing messages from a client, such as GET and PROP-SET messages. This allows the server to locate and read the parameters contained in these messages. These parameters may include, for example, a session identifier, a time value and time format, and any quality parameters, stream positions, and stream rates. Computing program logic 1750 may also include encoder adjustment logic 1755, which may be responsible for making adjustments to the operation of one or more video and/or audio encoders in order to accommodate any changes requested by the client with respect to quality or stream rate. Computing program logic 1750 may also include seeking logic 1760, which may be responsible for finding a point in a stream specified in a PROP-SET message from a client.
Computing program logic 1750 may also include metadata logic 1765, which is responsible for constructing the metadata that is sent from the server to the client in advance of the response. Computing program logic 1750 may also include header construction logic 1765, responsible for creating the headers for the requested data as illustrated in
Note that the above logic modules represent an embodiment of a software or firmware implementation of the processing described herein. In alternative embodiments, one or more of these logic modules may be combined with another; moreover, one or more may be absent in other embodiments.
While various embodiments are disclosed herein, it should be understood that they have been presented by way of example only, and not limitation. It will be apparent to persons skilled in the relevant art that various changes in form and detail may be made therein without departing from the spirit and scope of the methods and systems disclosed herein. Thus, the breadth and scope of the claims should not be limited by any of the exemplary embodiments disclosed herein.
Claims
1. A method for streaming data, comprising:
- receiving, from a client, a request for data;
- sending metadata to the client, describing the requested data;
- creating a frame that comprises the requested data and headers appended to the requested data; and
- sending a first response to the client without a container, wherein the first response comprises the frame.
2. The method of claim 1, wherein said data comprises one or more of video data and audio data.
3. The method of claim 2, further comprising:
- interleaving the audio data and video data, performed prior to sending the first response to the client.
4. The method of claim 1, wherein communications with the client take place according to a version of the hypertext transfer protocol (HTTP).
5. The method of claim 1, wherein the first header comprises:
- a frame size field;
- a stream identifier field; and
- a presentation timestamp (PTS) field.
6. The method of claim 1, further comprising:
- receiving a property setting message from the client;
- sending additional metadata to the client, describing new properties of the requested data, wherein the new properties are consistent with the property setting message;
- creating an additional frame of the requested data having the new properties, the additional frame comprising the requested data having the new properties and a second header appended to the requested data having the new properties; and
- sending a second response to the client without a container, wherein the second response comprises the additional frame.
7. The method of claim 6, further comprising:
- adjusting one or more encoding parameters in accordance with the property setting message; and
- encoding the requested data in accordance with the adjusted encoding parameters to create the requested data having the new properties,
- performed before sending the second response.
8. The method of claim 7, wherein the property setting message comprises one or more of:
- a header identifying a current session;
- a header identifying a desired quality level of the requested data; and
- a header identifying a current stream position.
9. The method of claim 1, further comprising:
- receiving a property setting message from the client;
- sending additional metadata to the client, describing properties of the requested data;
- creating an additional frame of the requested data, the additional frame comprising requested data starting at a point specified in the property setting message, and further comprising a second header appended to the requested data that starts at the specified point; and
- sending a second response to the client without a container, wherein the second response comprises the additional frame.
10. The method of claim 9, wherein the property setting message comprises one or more of:
- a header identifying a current session;
- a header identifying a time format;
- a header identifying time values corresponding to the point in the requested data and to a stop time for outputting the requested data to a user;
- a header identifying a desired stream rate;
- a header identifying a current stream position; and
- a header identifying a current stream rate.
11. The method of claim 1, wherein the request comprises:
- a header identifying a current session;
- a header identifying a time format; and
- a header identifying time values indicating start and stop times for outputting the requested data to a user.
12. A server for streaming data, comprising:
- a processor; and
- a memory in communication with said processor, said memory for storing a plurality of processing instructions configured to direct said processor to receive, from a client, a request for data; send metadata to the client, describing the requested data; create a frame that comprises the requested data and headers appended to the requested data; and send a first response to the client without a container, wherein the first response comprises the frame.
13. The server of claim 12, wherein said data comprises one or more of video data and audio data.
14. The server of claim 13, wherein said plurality of processing instructions is further configured to direct said processor to:
- interleave the audio data and video data, performed prior to sending the first response to the client.
15. The server of claim 12, wherein communications with the client take place according to a hypertext transfer protocol (HTTP).
16. The server of claim 12, wherein the first header comprises:
- a frame size field;
- a stream identifier field; and
- a presentation timestamp (PTS) field.
17. The server of claim 12, wherein the plurality of processing instructions is further configured to direct said processor to:
- receive a property setting message from the client;
- send additional metadata to the client, describing the properties of the requested data, wherein the new properties are consistent with the property setting message;
- create an additional frame of the requested data having the new properties, the additional frame comprising the requested data having the new properties and a second header appended to the requested data having the new properties; and
- send a second response to the client without a container, wherein the second response comprises the additional frame.
18. The server of claim 17, wherein the plurality of processing instructions is further configured to direct said processor to:
- adjust one or more encoding parameters in accordance with the property setting message; and
- encode the requested data in accordance with the adjusted encoding parameters to create the requested data having the new properties,
- performed before sending the second response.
19. The server of claim 18, wherein the property setting message comprises one or more of:
- a header identifying a current session;
- a header identifying a desired quality level of the requested data; and
- a header identifying a current stream position.
20. The server of claim 12, wherein the plurality of processing instructions is further configured to direct said processor to:
- receive a property setting message from the client;
- send additional metadata to the client, describing properties of the requested data;
- create an additional frame of the requested data, the additional frame comprising requested data starting at a point specified in the property setting message, and further comprising a second header appended to the requested data that starts at the specified point; and
- send a second response to the client without a container, wherein the second response comprises the additional frame.
21. The server of claim 20, wherein the property setting message comprises one or more of:
- a header identifying a current session;
- a header identifying a time format;
- a header identifying time values corresponding to the point in the requested data and to a stop time for outputting the requested data to a user;
- a header identifying a desired stream rate;
- a header identifying a current stream position; and
- a header identifying a current stream rate.
22. The server of claim 12, wherein the request comprises:
- a header identifying a current session;
- a header identifying a time format; and
- a header identifying time values indicating start and stop times for outputting the requested data to a user.
23. A computer program product comprising a non-transitory computer useable medium having control logic stored therein, the computer control logic comprising logic to cause a processor to perform the following:
- receiving, from a client, a request for data;
- sending metadata to the client, describing the requested data;
- creating a frame that comprises the requested data and headers appended to the requested data; and
- sending a first response to the client without a container, wherein the first response comprises the frame.
24. The computer program product of claim 23, wherein said data comprises one or more of video data and audio data.
25. The computer program product of claim 24, the computer control logic further comprising logic to cause a processor to perform the following:
- interleaving the audio data and video data, performed prior to sending the first response to the client.
26. The computer program product of claim 23, wherein communications with the client take place according to a hypertext transfer protocol (HTTP).
27. The computer program product of claim 23, wherein the first header comprises:
- a frame size field;
- a stream identifier field; and
- a presentation timestamp (PTS) field.
28. The computer program product of claim 23, the computer control logic further comprising logic to cause a processor to perform the following:
- receiving a property setting message from the client;
- sending additional metadata to the client, describing the properties of the requested data, wherein the new properties are consistent with the property setting message;
- creating an additional frame of the requested data having the new properties, the additional frame comprising the requested data having the new properties and a second header appended to the requested data having the new properties; and
- sending a second response to the client without a container, wherein the second response comprises the additional frame.
29. The computer program product of claim 28, the computer control logic further comprising logic to cause a processor to perform the following:
- adjusting one or more encoding parameters in accordance with the property setting message; and
- encoding the requested data in accordance with the adjusted encoding parameters to create the requested data having the new properties,
- performed before sending the second response.
30. The computer program product of claim 29, wherein the property setting message comprises one or more of:
- a header identifying a current session;
- a header identifying a desired quality level of the requested data; and
- a header identifying a current stream position.
31. The computer program product of claim 23, the computer control logic further comprising logic to cause a processor to perform the following:
- receiving a property setting message from the client;
- sending additional metadata to the client, describing properties of the requested data;
- creating an additional frame of the requested data, the additional frame comprising requested data starting at a point specified in the property setting message, and further comprising a second header appended to the requested data that starts at the specified point; and
- sending a second response to the client without a container, wherein the second response comprises the additional frame.
32. The computer program product of claim 31, wherein the property setting message comprises one or more of:
- a header identifying a current session;
- a header identifying a time format;
- a header identifying time values corresponding to the point in the requested data and to a stop time for outputting the requested data to a user;
- a header identifying a desired stream rate;
- a header identifying a current stream position; and
- a header identifying a current stream rate.
33. The computer program product of claim 23, wherein the request comprises:
- a header identifying a current session;
- a header identifying a time format; and
- a header identifying time values indicating start and stop times for outputting the requested data to a user.
Type: Application
Filed: May 24, 2012
Publication Date: Nov 28, 2013
Applicant: Rovi Corporation (Santa Clara, CA)
Inventor: Yuri Bulava (Tomsk City)
Application Number: 13/480,181
International Classification: G06F 15/16 (20060101);