Systems and methods for estimating available bandwidth and performing initial stream selection when streaming content
Systems and methods for determining available bandwidth and performing initial stream selection when commencing adaptive bitrate streaming using Hypertext Transfer Protocol (HTTP) in accordance with embodiments of the invention are disclosed. One embodiment of the invention includes measuring network bandwidth using a playback device, determining an initial network bandwidth estimate using the network bandwidth measurements, selecting an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates rates using the playback device based upon the maximum bitrates of the plurality of streams and the initial bandwidth estimate; and commencing streaming of encoded media by requesting portions of the encoded media from the initial streams using the playback device.
Latest DIVX, LLC Patents:
- Systems, Methods, and Media for Controlling Delivery Of Content
- Chunk Header Incorporating Binary Flags and Correlated Variable-Length Fields
- Systems and methods for providing variable speeds in a trick-play mode
- Federated digital rights management scheme including trusted systems
- Systems and Methods for Seeking Within Multimedia Content During Streaming Playback
This application is a continuation of U.S. patent application Ser. No. 13/341,838, filed Dec. 30, 2011, which claimed priority to U.S. Provisional Application Ser. No. 61/502,767, entitled “Client-Server Bandwidth Hinting for Adaptive Streaming Enhancements”, filed Jun. 29, 2011, the entirety of which are incorporated herein by reference.
FIELD OF THE INVENTIONThe present invention generally relates to streaming media and more specifically to selecting an initial stream from a number of streams encoded with different maximum bitrates.
BACKGROUNDThe term streaming media describes the playback of media on a playback device, where the media is stored on a server and continuously sent to the playback device over a network during playback. Typically, the playback device stores a sufficient quantity of media in a buffer at any given time during playback to prevent disruption of playback due to the playback device completing playback of all the buffered media prior to receipt of the next portion of media. Adaptive bit rate streaming or adaptive streaming involves detecting the present streaming conditions (e.g. the user's network bandwidth and CPU capacity) in real time and adjusting the quality of the streamed media accordingly. Typically, the source media is encoded at multiple bit rates and the playback device or client switches between streaming the different encodings depending on available resources. When a playback device commences adaptive bitrate streaming, the playback device typically starts by requesting portions of media from the lowest bitrate streams (where alternative streams are available). As the playback device downloads the requested media, the playback device can measure the available bandwidth. In the event that there is additional bandwidth available, the playback device can Witch to higher bitrate streams.
Adaptive streaming solutions typically utilize Hypertext Transfer Protocol (HTTP), published by the Internet Engineering Task Force and the World Wide Web Consortium as RFC 2616, to stream media between a server and a playback device. HTTP is a stateless protocol that enables a playback device to request a byte range within a file, HTTP is described as stateless, because the server is not required to record information concerning the state of the playback device requesting information or the byte ranges requested by the playback device in order to respond to requests received from the playback device.
In adaptive streaming systems, the source media is typically stored on a media server as a top level index file pointing to a number of alternate streams that contain the actual video and audio data. Each stream is typically stored in one or more container files. Different adaptive streaming solutions typically utilize different index and media containers. The Synchronized Multimedia Integration language (SMIL) developed by the World Wide Web Consortium is utilized to create indexes in several adaptive streaming solutions including IIS Smooth Streaming developed by Microsoft Corporation of Redmond, Wash., and Flash Dynamic Streaming developed by Adobe Systems incorporated of San Jose, Calif. HTTP Adaptive Bitrate Streaming developed by Apple Computer Incorporated of Cupertino, Calif. implements index files using an extended M3U playlist file (.M3U8), which is a text file containing a list of URIs that typically identify a media container file. The most commonly used media container formats are the MP4 container format specified in MPEG-4 Part 14 (i.e. ISO/IEC 14496-14) and the MPEG transport stream (TS) container specified in MPEG-2 Part 1 (i.e. ISO/IEC Standard 13818-1). The MP4 container format is utilized in IIS Smooth Streaming and Flash Dynamic Streaming. The TS container is used in HTTP Adaptive Bitrate Streaming.
SUMMARY OF THE INVENTIONSystems and methods in accordance with embodiments of the invention estimate available bandwidth and perform initial stream selection when commencing streaming of media. One embodiment of the method of the invention includes measuring network bandwidth using a playback device, determining an initial network bandwidth estimate using the network bandwidth measurements, selecting an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates rates using the playback device based upon the maximum bitrates of the plurality of streams and the initial bandwidth estimate; and commencing streaming of encoded media by requesting portions of the encoded media from the initial streams using the playback device.
In a further embodiment of the method. of the invention, the playback device performs the initial bandwidth estimate using the network bandwidth measurements.
Another embodiment of the method of the invention also includes providing the network bandwidth measurements to a performance server. In addition, the performance server determines the initial bandwidth estimate using the network bandwidth measurements.
In a still further embodiment of the method of the invention, measuring network bandwidth includes requesting a block of data and measuring the time taken to download the data.
In still another embodiment of the method of the invention, measuring network bandwidth includes associating a timestamp with the network bandwidth measurement.
In a yet further embodiment of the method of the invention, measuring network bandwidth includes associating a content distribution network identifier with the network bandwidth measurement.
In yet another embodiment of the method of the invention, measuring network bandwidth includes associating a server IP address with the network bandwidth measurement.
In a further embodiment again of the method of the invention, measuring network bandwidth includes associating a server hostname with the network bandwidth measurement.
In another embodiment again of the method of the invention, measuring bandwidth includes associating a video maximum bitrate with the network bandwidth measurement.
In a further additional embodiment of the method of the invention, determining an initial network bandwidth estimate using the network bandwidth measurements includes determining an average network bandwidth using the network bandwidth measurements.
In another additional embodiment of tile method of the invention, the average network bandwidth is determined by filtering the network bandwidth measurements based upon at least one piece of information associated with the network bandwidth measurements selected from the group consisting of time of day, day of week, content distribution network, server, and maximum bitrate.
In a still yet further embodiment of the method of the invention, determining an initial network bandwidth estimate using the network bandwidth measurements further includes determining the accuracy of the initial bandwidth estimate.
In still vet another embodiment of the method of the invention, the initial stream is selected for use in the adaptive bitrate streaming of the encoded media.
In a still further embodiment again of the method of the invention, the initial stream is selected by initially allocating available bandwidth to streams which have no alternative and allocating the remaining bandwidth to the selection from the plurality of streams of encoded media that are encoded at different maximum bitrates rates.
Still another embodiment again of the method of the invention, also includes selecting a stream from each of a number of sets of streams of encoded media that are encoded at different maximum bitrates.
In a still further additional embodiment, the user can instruct the playback device concerning allocation of bandwidth to at least one of the selections subject to the remaining available bandwidth.
A further embodiment of the invention includes a processor configured, via a client application, to request portions of files from a remote server. In addition, the client application further configures the processor to measure network bandwidth using a playback device, obtain an initial network bandwidth estimate using the network bandwidth measurements, select an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates rates using, the playback device based upon the maximum bitrates of the plurality of streams and the initial bandwidth estimate, and commence streaming of encoded media by requesting portions of the encoded media from the initial streams using the playback device.
In another embodiment, the client application further configures the processor to measure network bandwidth by requesting a block of data and measuring the time taken to download the data.
In a still further embodiment, the client application further configures the processor to associate a time stamp with the network bandwidth measurement.
In still another embodiment, the client application further configures the processor to associate a content distribution network identifier with the network bandwidth measurement.
In a yet further embodiment, the client application further configures the processor to associate a server IP address with the network bandwidth measurement.
In yet another embodiment, the client application further configures the processor to associate a server host name with the network bandwidth measurement.
In a further embodiment again, the client application further configures the processor to associate a video maximum bitrate with the network bandwidth measurement.
In another embodiment again, the client application further configures the processor to associate the playback device IP address with the network bandwidth measurement.
In a further additional embodiment, the client application further configures the processor to perform an initial bandwidth estimate using the network bandwidth measurements.
In another additional embodiment, the client application further configures the processor to determine an initial network bandwidth estimate using the network bandwidth measurements by determining an average network bandwidth using the network bandwidth measurements.
In a still further embodiment, the client application further configures the processor to determine the average network bandwidth by filtering the network bandwidth measurements based upon at least one piece of information associated with the network bandwidth measurements selected from the group consisting of time of day, day of week, content distribution network, server and maximum bitrate.
In still another embodiment, the client application further configures the processor to determine the accuracy of the initial bandwidth estimate.
In a yet further embodiment, the client application further configures the processor to provide the network bandwidth measurements to a performance server, and receive an initial bandwidth estimate from the performance server.
In yet another embodiment, the initial stream is selected for use in the adaptive bitrate streaming of the encoded media.
In a further embodiment again, the client application farther configures the processor to select the initial stream by initially allocating available bandwidth to streams which have no alternative and allocating the remaining bandwidth to the selection from the plurality of streams of encoded media that are encoded at different maximum bitrates rates.
In another embodiment again, the client application further configures the processor to select a stream front each of a number of sets of streams of encoded media that are encoded at different maximum bitrates.
A further additional embodiment includes a machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process including measuring network bandwidth using a playback device, obtaining an initial network bandwidth estimate using the network bandwidth measurements, selecting an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates rates using the playback device based upon the maximum bitrates of the plurality of streams and the initial bandwidth estimate, and commencing streaming of encoded media by requesting portions of the encoded media from the initial streams using the playback device.
In a still yet further embodiment, execution of the instructions by a processor causes the process to obtain an initial network bandwidth estimate by calculating an initial bandwidth estimate using the network bandwidth measurements.
In still yet another embodiment, execution of the instructions by a processor causes the process to obtain an initial network bandwidth estimate by transmitting the network bandwidth measurements to a server, and receiving an initial so bandwidth estimate from the server.
Another further embodiment of the invention includes recording performance data concerning playback of streamed content by a playback device, obtaining a maximum bitrate of a piece of content, comparing the maximum bitrate of the piece of content to recorded performance data for playback of streamed content by the playback device, and verify that the playback device can support playback of content based upon the comparison of the maximum bitrate of the piece of content and the recorded performance data for playback of streamed content by the playback device.
In still another further embodiment, the performance data includes information concerning the amount of time spent buffering streamed content.
In yet another further embodiment, comparing the maximum bitrate of the piece of content to recorded performance data for playback of streamed content by the playback device includes comparing an amount of time spent buffering streamed content encoded at a maximum bitrate corresponding to the maximum bitrate of the piece of content to a predetermined threshold.
In another further embodiment again, the performance data includes information concerning the amount of time spent streaming content streams encoded at different maximum bitrates during adaptive bitrate streaming.
In another further additional embodiment, comparing the maximum bitrate of the piece of content to recorded performance data for playback of streamed content by the playback device includes comparing the proportion of time spent streaming content streams encoded at a maximum bitrate corresponding to the maximum bitrate of the piece of content to a predetermined threshold.
Turning now to the drawings, systems and methods for estimating available bandwidth and performing initial stream selection when commencing streaming of media in accordance with embodiments of the invention are illustrated. In many embodiments, playback devices record performance information during the streaming session. When a playback device initiates a new streaming session, previously recorded performance information cart be utilized to estimate available bandwidth. The bandwidth estimate can then be used to select between streams encoded at different maximum bitrates for commencement of conventional or adaptive bitrate streaming. Where audio and/or subtitle tracks are only available at a single bitrate, the bitrate of any audio and/or subtitle streams is subtracted from the bandwidth estimate. Where a specific stream such as (but not limited to) a video stream is selected from a number at alternative streams encoded at different maximum bitrates, a video stream is selected having a maximum bitrate equal to or fess than the estimated remaining available bandwidth. Where alternative audio and video stream combinations are available, the system can default or rely on user selections to choose either a higher bitrate audio stream or a higher bitrate video stream.
In many embodiments, the historical performance information is utilized to estimate available bandwidth using a technique appropriate to a specific application. In a number of embodiments, the last network bandwidth measurement recorded by a playback device is utilized to generate a bandwidth estimate. In several embodiments, the network bandwidth measurements recorded by a playback device over a predetermined time period are utilized to generate a bandwidth estimate. In many embodiments, correlation between the time of day and the network bandwidth measurements recorded by the playback device can be utilized in determining an initial bandwidth estimate.
In various embodiments, the process of determining a bandwidth estimate also includes providing information concerning the accuracy of the estimate. In many embodiments, one or more accuracy factors are generated that can he utilized in selection of initial streams based upon a bandwidth estimate. Depending on the accuracy factor(s), a playback device can determine the margin to allow between the bandwidth requirements of selected streams and the estimated available bandwidth. In many embodiments, the accuracy factor is determined based upon the correlation between bandwidth estimates made with respect to the playback device and the actual streaming conditions subsequently experienced by the playback device.
In a number of embodiments, playback devices report network bandwidth measurements to a performance server and, the performance server makes recommendations concerning initial stream selection. In several embodiments, the playback devices can report back the identity of the content distribution network and/or specific server from which content was streaming at the time the network bandwidth measurement was taken. In addition to providing stream selection recommendations, the performance server can also provide recommendations concerning the specific content distribution network and/or servers from which to stream content to the playback device to achieve the best network performance. By aggregating network performance measurements from multiple playback devices, a performance server in accordance with embodiments of the invention can perform bandwidth estimations and recommendations concerning content distribution networks and/or servers utilizing bandwidth measurements taken from multiple playback devices within a geographic area that utilize the same Internet Service Provider (ISP).
The availability of network bandwidth measurements on a performance server can also enable other possible functionality. In several embodiments, playback devices can purchase content for streaming from online content stores and the price for the same content at different resolutions may vary. In a number of embodiments, the performance server utilizes recorded network bandwidth measurements to determine whether the playback device is likely to experience network bandwidth performance capable of supporting, the streaming of the content at a specific resolution. In this way, the recorded performance information can be utilized to make recommendations concerning the purchase of content at different resolutions. In several embodiments, the recommendation can also include an indication of the accuracy or likelihood that the recommendation is correct. In a number of embodiments, an accuracy factor can be generated based upon the total time spent rebuffering content streamed at the recommended level during previous streaming sessions. In many embodiments, an accuracy factor can be generated with respect to an adaptive bitrate streaming system based upon the proportion of time spent streaming below the recommended level during previous streaming sessions. In other embodiments, any of a variety of accuracy metrics can be utilized as appropriate to the requirements of a specific application.
In other embodiments, the performance server can the network bandwidth measurements in any of a variety of ways appropriate to a specific application including but not limited to enabling online content stores to offer premium content not only based upon resolution, but also based upon distribution from a specific content distribution network that has historically achieved higher network performance with respect to a specific playback device. Performance servers and processes for collecting network bandwidth measurements and estimating network bandwidth performance in accordance with embodiments of the invention are discussed further below.
Streaming System Architecture
Streaming systems in accordance with many embodiments of the invention include a server on which content is stored, playback devices that request streaming of content and that perform bandwidth measurements, and a performance server configured to aggregate network bandwidth measurements and to provide bandwidth estimates. A streaming system in accordance with an embodiment of the invention is illustrated in
During streaming of content, the playback devices 12 perform network bandwidth or data rate measurements. Each network bandwidth measurement can be timestamped and stored locally and/or sent to a performance server 18 that can store the network bandwidth measurement in a database 20. In many embodiments, the playback device sends network bandwidth measurements to the performance server continuously. In a number of embodiments, the playback device sends network bandwidth measurements to the performance server in batches.
When a playback device 12 initiates a new streaming session, previous measurements of network bandwidth made by a playback device can be utilized to estimate the available network bandwidth. Any of a variety of techniques appropriate to a specific application can be utilized to generate a network bandwidth estimate in accordance with embodiments of the invention. The generation of network bandwidth. estimates is discussed further below. Based upon the network bandwidth estimate, the playback device can determine the initial streams from which to commence requesting portions of encoded media based upon the maximum bitrates of the available streams. In several embodiments, the network performance server 18 generates the network bandwidth estimate and provides the network bandwidth estimate to the playback device. in a number of embodiments, the network performance server 18 generates a network bandwidth estimate and uses the estimate to provide initial stream selection recommendations to a playback device 12. In many embodiments, the playback device 12 generates the network bandwidth estimate based upon locally stored network bandwidth measurements. As is discussed .further below, the process of generating a network bandwidth estimate can also include determining the accuracy of the network bandwidth estimate. In a number of embodiments, the accuracy of a network bandwidth estimate is considered during the initial stream selection.
In several embodiments, playback devices 12 are able to purchase content for streaming via one or more online content stores 22. In many instances, the online content store will provide different pricing for content encoded at different resolutions. For example, a content store can charge a premium for content encoded at high definition resolution relative to content encoded at standard definition resolution. if insufficient network bandwidth is available to a playback device to stream the high definition content, the playback device may prefer to purchase the lower resolution content. In the context of conventional streaming, the playback device will experience frequent buffering interruptions by purchasing the premium content. In the context of adaptive bitrate streaming, the majority of the content received by the playback device likely will be at the lower resolution irrespective of whether the playback device purchases the premium content. In order to reduce the likelihood that a playback device 12 purchases premium content that cannot be supported by the available network bandwidth, the bandwidth available to the playback device can be estimated using the recorded network performance measurements and recommendations provided to the user concerning whether the premium content can be supported. In several embodiments, the process of generating a recommendation also involves determining the likelihood that the recommendation is correct. In a number of embodiments, the servers of the online content store 22 communicate with the performance server 18 to obtain a network performance estimate. In other embodiments, the playback device 12 generates the network performance estimate based upon locally stored records of past network bandwidth measurements.
Although a specific streaming system is illustrated in
Playback Devices
In the embodiment illustrated in
In a number of embodiments, the playback device is configured to perform adaptive bitrate streaming. In several embodiments, a top level index file is stored and/or generated by a server within the content distribution network 14 or a server associated with the online content store 22. The top level index file specifies the alternative streams available to the playback device and the maximum bitrate of each stream. Therefore, a performance server and/or a playback device can utilize the index file to perform initial stream selections based upon a bandwidth estimate generated in accordance with embodiments of the invention. Examples of adaptive bitrate streaming systems that can be utilized in accordance with embodiments of the invention include but are not limited to the adaptive bitrate streaming systems disclosed in U.S. Provisional Patent Application Ser. No. 61/430,110, “Systems and Methods For Adaptive Bitrate Streaming of Media Stored in Matroska Files Using Hypertext Transfer Protocol”, filed Jan. 5, 2011, the entirety of which is incorporated herein by reference.
Although a specific playback device architecture is illustrated in
Measuring Network Bandwidth
Any of a variety of processes can be utilized to measure network bandwidth arid decoder performance. A process for measuring network performance in accordance with an embodiment of the invention is illustrated in
Although a specific process for estimating network bandwidth is illustrated in
Estimating Network Bandwidth
Historical network bandwidth measurements can be utilized to select one or more initial streams when streaming content. A process for generating a bandwidth estimate using historical network bandwidth measurements in accordance with embodiments of the invention is illustrated in
Although a specific process is illustrated in
Selecting initial Steams
When an estimate of the available bandwidth or bitrate has been obtained, a playback device can select one or more initial streams and commence streaming of the encoded content. A process for selecting initial stream(s) in accordance with an embodiment of the invention is illustrated in
Although a specific process for selecting initial streams based upon an initial bandwidth estimate is discussed above with respect to
Premium Content Recommendations
Many content distribution systems charge a premium for higher resolution content, which is typically encoded at a higher maximum bitrate relative to lower resolution content. At the time of purchase, however, information is typically not available concerning the data rate of a network connection that can be established between the content distribution system and a playback device. In many instances, purchase of higher resolution content may result in an inferior playback experience due to repeated interruptions to buffer content. In adaptive bitrate streaming systems, insufficient network bandwidth may cause the majority of the video streamed to a playback device to be at a lower resolution than the resolution paid for by a user. Initial bandwidth estimates generated using stored network bandwidth measurements can be utilized to make recommendations concerning whether a user's playback device and network connection can support playback at a specific resolution.
A process for determining whether a playback device can support playback of a piece of content at a specific resolution in accordance with an embodiment of the invention is illustrated in
As is the case with the initial bandwidth estimates discussed above, the network bandwidth measurements utilized to determine whether the bandwidth requirements of the content are supported can be filtered based upon one or more pieces of additional information associated with the network measurements including (but not limited to) time of day, date, and the content distribution network and/or server streaming the content. In many embodiments, the determination concerning whether the network bandwidth requirements are supported can be accompanied by a confidence factor or accuracy metric. In several embodiments, the confidence factor or accuracy metric can represent the proportion of time that is likely to be spent buffering or streaming at a lower resolution or reflect variation in the network bandwidth measurement data utilized to determine whether the network bandwidth requirements of the content can be supported. When the network bandwidth requirements can be met, the process 90 can involve generating (98) a user notification that the playback requirements can be met and/or authorizing the transaction. In embodiments Where a performance server makes the determination, the determination can be integrated within the user interface of a content distribution system and/or the determination can be integrated into the purchase process. In the event a determination is made that the network bandwidth requirements of a specific content resolution exceed the data rate that is likely to be available to a playback device, then a notification is generated (100) informing the user that they should not proceed with purchasing and/or streaming the content.
Although a specific process is illustrated in
Content Distribution Network Recommendations
The data rate of a network connection experienced by a specific playback device can vary depending upon the content distribution network utilized to distribute content. In a number of embodiments, a content distribution system can offer premium content based not just on the resolution of the content but the content distribution network utilized to deliver the content. In many embodiments, a streaming system can provide a performance metric such as (but not limited to) an estimate of the amount of time the playback device will spend buffering during playback from different content distribution networks and can charge a premium based the higher playback performance anticipated when streaming from a premium content distribution network. In several embodiments, an adaptive bitrate streaming system can display a performance metric such as (but not limited to) an estimate of the proportion of time that a playback device will receive content encoded at the highest available resolution and charge a premium based upon the higher playback performance anticipated when streaming from a premium content distribution network.
Although the present invention has been described in certain specific aspects, many additional modifications and variations would be apparent to those skilled in the art. It is therefore to be understood that the present invention may be practiced otherwise than specifically described, including various changes in the implementation such as utilizing encoders and decoders that support features beyond those specified within a particular standard with which they comply, without departing from the scope and spirit of the present invention. Thus, embodiments of the present invention should be considered in all respects as illustrative and not restrictive.
Claims
1. A method of selecting an initial stream of encoded media from a plurality of streams of encoded media that are encoded at different maximum bitrates rates, the method comprising:
- retrieving historical network performance data comprising a plurality of network bandwidth measurements from a memory of a playback device;
- filtering the plurality of network bandwidth measurements based upon at least one piece of information associated with the network bandwidth measurements selected from the group consisting of time of day and day of week;
- determining, using the playback device and prior to commencing streaming of encoded media, an initial network bandwidth estimate using the filtered network bandwidth measurements, wherein the initial network bandwidth estimate is determined by performing statistical analysis using the filtered network bandwidth measurements;
- generating, using the playback device, an estimate of the accuracy of the initial network bandwidth estimate by (i) determining a ratio of previous bandwidth estimates to actual bandwidth or (ii) computing a standard deviation of the filtered network bandwidth measurements utilized to generate the initial bandwidth estimate;
- selecting an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates rates using the playback device for commencing streaming of encoded media based upon the maximum bitrates of the plurality of streams, the initial network bandwidth estimate, and the accuracy of the initial network bandwidth estimate; and
- commencing streaming of encoded media by requesting portions of the encoded media from the initial streams using the playback device.
2. The method of claim 1, wherein the statistical analysis comprises averaging the filtered network bandwidth measurements.
3. The method of claim 2, wherein the network bandwidth measurements are for content streamed from a particular content distribution network.
4. The method of claim 2, wherein the network bandwidth measurements occurred within a predetermined time period.
5. The method of claim 1, wherein the network bandwidth measurements occurred within a time of day or day of week.
6. The method of claim 1, wherein the initial network bandwidth estimate is determined by filtering the network bandwidth measurements based upon at least one piece of information associated with the network bandwidth measurements selected from the group consisting of time of day, day of week, content distribution network, server, and maximum bitrate.
7. A method of selecting an initial stream of encoded media from a plurality of streams of encoded media that are encoded at different maximum bitrates rates, the method comprising:
- retrieving historical network performance data comprising a plurality of network bandwidth measurements from a memory of a playback device;
- determining, using the playback device and prior to commencing streaming of encoded media, an initial network bandwidth estimate using the network bandwidth measurements, wherein the initial network bandwidth estimate is determined by performing statistical analysis using the network bandwidth measurements;
- selecting an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates rates using the playback device for commencing streaming of encoded media based upon the maximum bitrates of the plurality of streams and the initial network bandwidth estimate, wherein the initial stream is selected by initially allocating available bandwidth to streams at least one secondary stream which have has no alternative and allocating the remaining bandwidth to the selection selecting the initial stream from the plurality of streams of encoded media that are encoded at different maximum bitrates based on the remaining bandwidth, wherein the at least one secondary stream is at least one selected from the group consisting of an audio stream and a subtitle stream; and
- commencing streaming of encoded media by requesting portions of the encoded media from the initial streams using the playback device.
8. The method of claim 1, wherein the playback device performs the initial network bandwidth estimate using the filtered network bandwidth measurements.
9. The method of claim 1, further comprising:
- providing the network bandwidth measurements to a performance server; wherein filtering the network bandwidth measurements is performed by the performance server and the performance server determines the initial network bandwidth estimate using the network bandwidth measurements.
10. The method of claim 1, wherein the initial stream is selected for use in the adaptive bitrate streaming of the encoded media.
11. The method of claim 1, further comprising selecting a stream from each of a number of sets of streams of encoded media that are encoded at different maximum bitrates.
12. The method of claim 1, further comprising measuring current network bandwidth using the playback device.
13. The method of claim 12, further comprising determining the initial network bandwidth estimate using the measured current network bandwidth measurements.
14. A non-transitory machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process comprising:
- retrieving historical network performance data comprising a plurality of network bandwidth measurements;
- filtering the plurality of network bandwidth measurements based upon at least one piece of information associated with the network bandwidth measurements selected from the group consisting of time of day and day of week;
- determining, prior to commencing streaming of encoded media, an initial network bandwidth estimate using the filtered network bandwidth measurements, wherein the initial network bandwidth estimate is determined by performing statistical analysis using the filtered network bandwidth measurements;
- generating an estimate of the accuracy of the initial network bandwidth estimate by (i) determining a ratio of previous bandwidth estimates to actual bandwidth or (ii) computing a standard deviation of the filtered network bandwidth measurements utilized to generate the initial bandwidth estimate
- selecting an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates rates for commencing streaming of encoded media based upon the maximum bitrates of the plurality of streams, the initial network bandwidth estimate and the estimate of the accuracy of the initial network bandwidth estimate; and
- commencing streaming of encoded media by requesting portions of the encoded media from the initial streams.
15. The non-transitory machine readable medium of claim 14, wherein performing the statistical analysis comprises averaging the filtered network bandwidth measurements.
16. The non-transitory machine readable medium of claim 14, wherein the network bandwidth measurements are for content streamed from a particular content distribution network.
17. The non-transitory machine readable medium of claim 14, wherein the network bandwidth measurements occurred within a predetermined time period.
18. The non-transitory machine readable medium of claim 14, wherein the network bandwidth measurements occurred within a time of day or day of week.
19. The method of claim 7, further comprising generating an estimate of the accuracy of the initial network bandwidth estimate by (i) determining a ratio of previous bandwidth estimates to actual bandwidth or (ii) computing a standard deviation of the network bandwidth measurements utilized to generate the initial bandwidth estimate.
20. The method of claim 7, wherein the initial network bandwidth estimate is determined by filtering the network bandwidth measurements based upon at least one piece of information associated with the network bandwidth measurements selected from the group consisting of time of day, day of week, content distribution network, server, and maximum bitrate.
21. A method of selecting an initial stream of encoded media from a plurality of streams of encoded media that are encoded at different maximum bitrates, the method comprising:
- retrieving historical network performance data comprising a plurality of network bandwidth measurements;
- filtering the plurality of network bandwidth measurements based upon at least one piece of information associated with the network bandwidth measurements selected from the group consisting of time of day and day of week to identify at least one network bandwidth measurement;
- determining, using the playback device and prior to commencing streaming of encoded media, an initial network bandwidth estimate using the at least one network bandwidth measurement;
- generating, using the playback device, an estimate of the accuracy of the initial network bandwidth estimate by (i) determining a ratio of the bandwidth estimate to actual bandwidth or (ii) when the at least one network bandwidth measurement comprises a filtered plurality of network bandwidth measurements, computing a standard deviation of the filtered plurality of network bandwidth measurements utilized to generate the initial bandwidth estimate;
- selecting an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates rates using the playback device for commencing streaming of encoded media based upon the maximum bitrates of the plurality of streams, the initial network bandwidth estimate, and the accuracy of the initial network bandwidth estimate; and
- commencing streaming of encoded media by requesting portions of the encoded media from the initial streams using the playback device.
22. A non-transitory machine readable medium containing processor instructions, where execution of the instructions by a processor causes the processor to perform a process comprising:
- retrieving historical network performance data comprising a plurality of network bandwidth measurements;
- filtering the plurality of network bandwidth measurements based upon at least one piece of information associated with the network bandwidth measurements selected from the group consisting of time of day and day of week to identify at least one network bandwidth measurement;
- determining, using the playback device and prior to commencing streaming of encoded media, an initial network bandwidth estimate using the at least one network bandwidth measurement;
- generating, using the playback device, an estimate of the accuracy of the initial network bandwidth estimate by (i) determining a ratio of the bandwidth estimate to actual bandwidth or (ii) when the at least one network bandwidth measurement comprises a filtered plurality of bandwidth measurements, computing a standard deviation of the filtered plurality of network bandwidth measurements utilized to generate the initial bandwidth estimate;
- selecting an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates using the playback device for commencing streaming of encoded media based upon the maximum bitrates of the plurality of streams, the initial network bandwidth estimate, and the accuracy of the initial network bandwidth estimate; and
- commencing streaming of encoded media by requesting portions of the encoded media from the initial streams using the playback device.
23. A method of selecting an initial stream of encoded media from a plurality of streams of encoded media that are encoded at different maximum bitrates rates, the method comprising:
- retrieving historical network performance data comprising at least one network bandwidth measurement from a memory of a playback device;
- determining, using the playback device and prior to commencing streaming of encoded media, an initial network bandwidth estimate using the at least one network bandwidth measurement, wherein the initial network bandwidth estimate is determined by performing statistical analysis using the at least one network bandwidth measurement;
- selecting an initial stream from said plurality of streams of encoded media that are encoded at different maximum bitrates rates using the playback device for commencing streaming of encoded media based upon the maximum bitrates of the plurality of streams and the initial network bandwidth estimate, wherein the initial stream is selected by initially allocating available bandwidth to at least one secondary stream which has no alternative and selecting the initial stream from the plurality of streams of encoded media that are encoded at different maximum bitrates based on the remaining bandwidth, wherein the at least one secondary stream is at least one selected from the group consisting of an audio stream and a subtitle stream; and
- commencing streaming of encoded media by requesting portions of the encoded media from the initial streams using the playback device.
5400401 | March 21, 1995 | Wasilewski et al. |
5574785 | November 12, 1996 | Ueno et al. |
5600721 | February 4, 1997 | Kitazato |
5621794 | April 15, 1997 | Matsuda et al. |
5642338 | June 24, 1997 | Fukushima et al. |
5805700 | September 8, 1998 | Nardone et al. |
5813010 | September 22, 1998 | Kurano et al. |
5854873 | December 29, 1998 | Mori et al. |
5907658 | May 25, 1999 | Murase et al. |
5923869 | July 13, 1999 | Kashiwagi et al. |
6002834 | December 14, 1999 | Hirabayashi et al. |
6009237 | December 28, 1999 | Hirabayashi et al. |
6016381 | January 18, 2000 | Taira et al. |
6057832 | May 2, 2000 | Lev et al. |
6065050 | May 16, 2000 | DeMoney |
6266483 | July 24, 2001 | Okada et al. |
6282320 | August 28, 2001 | Hasegawa et al. |
6320905 | November 20, 2001 | Konstantinides |
6351538 | February 26, 2002 | Uz |
6373803 | April 16, 2002 | Ando et al. |
6385673 | May 7, 2002 | Demoney |
6415031 | July 2, 2002 | Colligan et al. |
6445877 | September 3, 2002 | Okada et al. |
6453115 | September 17, 2002 | Boyle |
6453116 | September 17, 2002 | Ando et al. |
6504873 | January 7, 2003 | Vehvilaeinen |
6512883 | January 28, 2003 | Shim et al. |
6594699 | July 15, 2003 | Sahai et al. |
6654933 | November 25, 2003 | Abbott et al. |
6671408 | December 30, 2003 | Kaku |
6690838 | February 10, 2004 | Zhou |
6724944 | April 20, 2004 | Kalevo et al. |
6751623 | June 15, 2004 | Basso et al. |
6813437 | November 2, 2004 | Ando et al. |
6871006 | March 22, 2005 | Oguz et al. |
6912513 | June 28, 2005 | Candelore |
6931531 | August 16, 2005 | Takahashi |
6957350 | October 18, 2005 | Demos |
6970564 | November 29, 2005 | Kubota et al. |
6983079 | January 3, 2006 | Kim |
7006757 | February 28, 2006 | Ando et al. |
7007170 | February 28, 2006 | Morten |
7020287 | March 28, 2006 | Unger |
7047309 | May 16, 2006 | Baumann et al. |
7151832 | December 19, 2006 | Fetkovich et al. |
7188183 | March 6, 2007 | Paul et al. |
7212726 | May 1, 2007 | Zetts |
7242772 | July 10, 2007 | Tehranchi |
7274861 | September 25, 2007 | Yahata et al. |
7295673 | November 13, 2007 | Grab et al. |
7349886 | March 25, 2008 | Morten et al. |
7352956 | April 1, 2008 | Winter et al. |
7382879 | June 3, 2008 | Miller |
7397853 | July 8, 2008 | Kwon et al. |
7400679 | July 15, 2008 | Kwon et al. |
7418132 | August 26, 2008 | Hoshuyama |
7457415 | November 25, 2008 | Reitmeier et al. |
7499930 | March 3, 2009 | Naka et al. |
7546641 | June 9, 2009 | Robert et al. |
7639921 | December 29, 2009 | Seo et al. |
7640435 | December 29, 2009 | Morten |
7711052 | May 4, 2010 | Hannuksela et al. |
7770200 | August 3, 2010 | Brooks |
7853980 | December 14, 2010 | Pedlow, Jr. et al. |
7864186 | January 4, 2011 | Robotham et al. |
7908393 | March 15, 2011 | Marr |
7945143 | May 17, 2011 | Yahata et al. |
7949775 | May 24, 2011 | Virdi |
8009575 | August 30, 2011 | Spain |
8099757 | January 17, 2012 | Riedl |
8131875 | March 6, 2012 | Chen |
8169916 | May 1, 2012 | Pai et al. |
8243924 | August 14, 2012 | Chen et al. |
8286213 | October 9, 2012 | Seo |
8312079 | November 13, 2012 | Newsome et al. |
8369421 | February 5, 2013 | Kadono et al. |
8516144 | August 20, 2013 | Hsu et al. |
8649669 | February 11, 2014 | Braness et al. |
8683066 | March 25, 2014 | Hurst et al. |
8782268 | July 15, 2014 | Pyle et al. |
8812662 | August 19, 2014 | Soroushian et al. |
8819116 | August 26, 2014 | Tomay et al. |
8832297 | September 9, 2014 | Soroushian |
8849950 | September 30, 2014 | Stockhammer et al. |
8918533 | December 23, 2014 | Chen et al. |
8997160 | March 31, 2015 | Hunt et al. |
9038116 | May 19, 2015 | Knox et al. |
9088527 | July 21, 2015 | Manor |
9124642 | September 1, 2015 | Choudhury et al. |
9313249 | April 12, 2016 | Soroushian et al. |
9420024 | August 16, 2016 | Pearson |
9491498 | November 8, 2016 | Brooks |
9521178 | December 13, 2016 | Freeman, II et al. |
9936267 | April 3, 2018 | Shivadas et al. |
20010021276 | September 13, 2001 | Zhou |
20010052077 | December 13, 2001 | Fung et al. |
20010052127 | December 13, 2001 | Seo et al. |
20020048450 | April 25, 2002 | Zetts |
20020067432 | June 6, 2002 | Kondo et al. |
20020135607 | September 26, 2002 | Kato et al. |
20020141503 | October 3, 2002 | Kobayashi et al. |
20020154779 | October 24, 2002 | Asano et al. |
20020164024 | November 7, 2002 | Arakawa et al. |
20020169971 | November 14, 2002 | Asano et al. |
20030002577 | January 2, 2003 | Pinder |
20030044080 | March 6, 2003 | Frishman et al. |
20030053541 | March 20, 2003 | Sun et al. |
20030063675 | April 3, 2003 | Kang et al. |
20030077071 | April 24, 2003 | Lin et al. |
20030135742 | July 17, 2003 | Evans |
20030142594 | July 31, 2003 | Tsumagari et al. |
20030206717 | November 6, 2003 | Yogeshwar et al. |
20040001594 | January 1, 2004 | Krishnaswamy et al. |
20040022391 | February 5, 2004 | Obrien |
20040028227 | February 12, 2004 | Yu |
20040037421 | February 26, 2004 | Truman |
20040047592 | March 11, 2004 | Seo et al. |
20040047607 | March 11, 2004 | Seo et al. |
20040076237 | April 22, 2004 | Kadono et al. |
20040081333 | April 29, 2004 | Grab et al. |
20040093494 | May 13, 2004 | Nishimoto et al. |
20040101059 | May 27, 2004 | Joch et al. |
20040107356 | June 3, 2004 | Shamoon et al. |
20050013494 | January 20, 2005 | Srinivasan et al. |
20050063541 | March 24, 2005 | Candelore |
20050076232 | April 7, 2005 | Kawaguchi |
20050144468 | June 30, 2005 | Northcutt |
20050177741 | August 11, 2005 | Chen et al. |
20050243912 | November 3, 2005 | Kwon et al. |
20050265555 | December 1, 2005 | Pippuri |
20060002425 | January 5, 2006 | Mane et al. |
20060013568 | January 19, 2006 | Rodriguez |
20060031564 | February 9, 2006 | Brassil et al. |
20060165163 | July 27, 2006 | Burazerovic et al. |
20070047645 | March 1, 2007 | Takashima |
20070067472 | March 22, 2007 | Maertens et al. |
20070083467 | April 12, 2007 | Lindahl et al. |
20070086485 | April 19, 2007 | Vega-Garcia et al. |
20070180051 | August 2, 2007 | Kelly et al. |
20080086570 | April 10, 2008 | Dey et al. |
20080101718 | May 1, 2008 | Yang et al. |
20080137847 | June 12, 2008 | Candelore et al. |
20090010622 | January 8, 2009 | Yahata et al. |
20090013195 | January 8, 2009 | Ochi et al. |
20090077143 | March 19, 2009 | Macy, Jr. |
20090106082 | April 23, 2009 | Senti et al. |
20090132599 | May 21, 2009 | Soroushian et al. |
20090144425 | June 4, 2009 | Marr |
20090178090 | July 9, 2009 | Oztaskent |
20090249081 | October 1, 2009 | Zayas |
20090282162 | November 12, 2009 | Mehrotra et al. |
20090300203 | December 3, 2009 | Virdi |
20090310819 | December 17, 2009 | Hatano |
20100142915 | June 10, 2010 | Mcdermott et al. |
20100235542 | September 16, 2010 | Visharam et al. |
20100306373 | December 2, 2010 | Wormley |
20100306810 | December 2, 2010 | Brooks et al. |
20110010466 | January 13, 2011 | Fan et al. |
20110058675 | March 10, 2011 | Brueck et al. |
20110066673 | March 17, 2011 | Outlaw |
20110096828 | April 28, 2011 | Chen et al. |
20110103374 | May 5, 2011 | Lajoie et al. |
20110135090 | June 9, 2011 | Chan et al. |
20110145858 | June 16, 2011 | Philpott et al. |
20110173345 | July 14, 2011 | Knox et al. |
20110179185 | July 21, 2011 | Wang et al. |
20110197261 | August 11, 2011 | Dong et al. |
20110246661 | October 6, 2011 | Manzari et al. |
20110296048 | December 1, 2011 | Knox et al. |
20110314130 | December 22, 2011 | Strasman |
20120005312 | January 5, 2012 | McGowan et al. |
20120042090 | February 16, 2012 | Chen et al. |
20120047542 | February 23, 2012 | Lewis et al. |
20120110120 | May 3, 2012 | Willig et al. |
20120141089 | June 7, 2012 | Hunt |
20120167132 | June 28, 2012 | Mathews et al. |
20120173751 | July 5, 2012 | Braness et al. |
20120177101 | July 12, 2012 | van der Schaar |
20120179834 | July 12, 2012 | van der Schaar et al. |
20120311174 | December 6, 2012 | Bichot et al. |
20120324519 | December 20, 2012 | Laughlin |
20120331167 | December 27, 2012 | Hunt |
20130007200 | January 3, 2013 | van der Schaar et al. |
20130007263 | January 3, 2013 | Soroushian et al. |
20130007297 | January 3, 2013 | Soroushian et al. |
20130013803 | January 10, 2013 | Bichot et al. |
20130080267 | March 28, 2013 | McGowan |
20140068096 | March 6, 2014 | Shivadas et al. |
20140140253 | May 22, 2014 | Lohmar et al. |
20140149557 | May 29, 2014 | Lohmar et al. |
20140365651 | December 11, 2014 | Soroushian et al. |
20150288530 | October 8, 2015 | Oyman |
20180288498 | October 4, 2018 | Shivadas et al. |
2237293 | July 1997 | CA |
1453319 | September 2004 | EP |
1283640 | October 2006 | EP |
2180664 | April 2010 | EP |
2360923 | August 2011 | EP |
20040039852 | May 2004 | KR |
20060106250 | October 2006 | KR |
2328040 | June 2008 | RU |
2000049762 | August 2000 | WO |
2000049763 | August 2000 | WO |
2003047262 | June 2003 | WO |
2004012378 | February 2004 | WO |
2004100158 | November 2004 | WO |
2005008385 | January 2005 | WO |
2005015935 | February 2005 | WO |
2009006302 | January 2009 | WO |
2009109976 | September 2009 | WO |
2011087449 | July 2011 | WO |
2011101371 | August 2011 | WO |
2011103364 | August 2011 | WO |
2013002828 | January 2013 | WO |
2013002834 | January 2013 | WO |
2013002835 | January 2013 | WO |
- International Preliminary Report on Patentability for International Application No. PCT/US2011/064684, dated Mar. 4, 2014, 5 pgs.
- International Preliminary Report on Patentability for International Application No. PCT/US2011/068156, Report dated Jan. 7, 2014, 9 Pgs.
- International Preliminary Report on Patentability for International Application No. PCT/US2011/068264, Search Completed Jan. 7, 2014, dated Jan. 16, 2014, 5 pgs.
- International Search Report and Written Opinion for Application PCT/US2011/068264, Search Completed Mar. 30, 2012, dated May 1, 2012, 6 pgs.
- International Search Report and Written Opinion for International Application No. PCT/US2011/068156, Search completed Apr. 7, 2012, dated May 1, 2012, 11 Pgs.
- International Search Report and Written Opinion for International Application PCT/US2011/064684, Search Completed Mar. 26, 2012, dated Apr. 10, 2012, 7 pgs.
- “MPEG-2, Part 1, ISO/IEC 13818-1”, Information technology—Generic Coding of Moving Pictures and Associated Audio: Systems, Nov. 13, 1994, 161 pgs.
- “MPEG-4, Part 14, ISO/IEC 14496-14”, Information technology—Coding of audio-visual objects, Nov. 15, 2003, 18 pgs.
- “Video Publishers Turning to HD Online Video to Boost Competitive Edge and Increase Revenues”, Business Wire, Inc., Apr. 22, 2009, 2 pages, retrieved from https://global.factiva.com/ga/default.aspx.
- Begen et al., “Watching Video over the Web Part 1: Streaming Protocols”, IEEE Internet Computing, Mar. 1, 2011, vol. 15, Issue 2, pp. 54-63.
- Fielding et al., “Hypertext Transfer Protocol—HTTP1.1”, Network Working Group, RFC 2616, Jun. 1999, 114 pgs.
- Gannes, “The Lowdown on Apple's HTTP Adaptive Bitrate Streaming”, GigaOM, Jun. 10, 2009, 12 pgs.
- Liu et al., “Rate adaptation for adaptive HTTP streaming”, MMSys '11 Proceedings of the second annual ACM conference on Multimedia systems, Feb. 23-25, 2011, San Jose, CA, USA, pp. 169-174.
- Riiser et al., “Video Streaming Using a Location-based Bandwidth-Lookup Service for Bitrate Planning”, ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 8, Issue 3, Article No. 24, Jul. 2012, 24 pages.
- Schulzrinne, “Real Time Streaming Protocol 2.0 (RTSP): draft-ietfmmusic-rfc2326bis-27”, MMUSIC Working Group of the Internet Engineering Task Force (IETF), Mar. 9, 2011, 296 page.
- Zambelli, “IIS Smooth Streaming Technical Overview”, Microsoft Corporation, Mar. 2009, 17 pages.
- Richardson, “H.264 and MPEG-4 Video Compression”, Wiley, 2003, 306 pgs. (Presented in 2 parts).
- Sima et al., “An Efficient Architecture for Adaptive Deblocking Filter of H.264 AVC Video Coding”, IEEE Transactions on Consumer Electronics, vol. 50, No. 1, Feb. 2004, pp. 292-296.
- Spanos et al., “Performance Study of a Selective Encryption Scheme for the Security of Networked, Real-Time Video”, Proceedings of the Fourth International Conference on Computer Communications and Networks, IC3N'95, Sep. 20-23, 1995, Las Vegas, NV, pp. 2-10.
- Srinivasan et al., “Windows Media Video 9: overview and applications”, Signal Processing: Image Communication, 2004, 25 pgs.
- Stockhammer, “Dynamic Adaptive Streaming over HTTP—Standards and Design Principles”, Proceedings of the second annual ACM conference on Multimedia, Feb. 2011, pp. 133-145.
- Timmerer et al., “HTTP Streaming of MPEG Media”, Proceedings of Streaming Day, 2010, 4 pgs.
- Tiphaigne et al., “A Video Package for Torch”, Jun. 2004, 46 pgs.
- Trappe et al., “Key Management and Distribution for Secure Multimedia Multicast”, IEEE Transaction on Multimedia, vol. 5, No. 4, Dec. 2003, pp. 544-557.
- Van Deursen et al., “On Media Delivery Protocols in the Web”, 2010 IEEE International Conference on Multimedia and Expo, Jul. 19-23, 2010, 6 pgs.
- Ventura, “Streaming of Multimedia Learning Objects”, AG Integrated Communication System, Mar. 2003, 101 pgs.
- Waggoner, “Compression for Great Digital Video”, 2002, 184 pgs.
- Watanabem et al., “MPEG-2 decoder enables DTV trick plays”, esearcher System LSI Development Lab, Fujitsu Laboratories Ltd., Kawasaki, Japan, Jun. 2001, 2 pgs.
- Wiegand, “Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG”, Jan. 2002, 70 pgs.
- Willig et al., U.S. Appl. No. 61/409,285, filed Nov. 2, 2010, 43 pgs.
- Yang et al., “Projection-Based Spatially Adaptive Reconstruction of Block-Transform Compressed Images”, IEEE Transactions on Image Processing, vol. 4, No. 7, Jul. 1995, pp. 896-908.
- Yang et al., “Regularized Reconstruction to Reduce Blocking Artifacts of Block Discrete Cosine Transform Compressed Images”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 3, No. 6, Dec. 1993, pp. 421-432.
- Yu et al., “Video deblocking with fine-grained scalable complexity for embedded mobile computing”, Proceedings 7th International Conference on Signal Processing, Aug. 31-Sep. 4, 2004, pp. 1173-1178.
- Zakhor, “Iterative Procedures for Reduction of Blocking Effects in Transform Image Coding”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 2, No. 1, Mar. 1992, pp. 91-95.
- Information Technology—MPEG Systems Technologies—Part 7: Common Encryption in ISO Base Media File Format Files (ISO/IEC 23001-7), Apr. 2015, 24 pgs.
- ISO/IEC 14496-12 Information technology—Coding of audio-visual objects—Part 12: ISO base media file format, Feb. 2004 (“MPEG-4 Part 12 Standard”), 62 pgs.
- ISO/IEC 14496-12:2008(E) Informational Technology—Coding of Audio-Visual Objects Part 12: ISO Base Media Filea Format, Oct. 2008, 120 pgs.
- ISO/IEC FCD 23001-6 MPEG systems technologies Part 6: Dynamic adaptive streaming over HTTP (DASH), Jan. 28, 2011, 86 pgs.
- Microsoft Corporation, Advanced Systems Format (ASF) Specification, Revision 01.20.03, Dec. 2004, 121 pgs.
- MPEG-DASH presentation at Streaming Media West 2011, Nov. 2011, 14 pgs.
- Pomelo, LLC Tech Memo, Analysis of Netflix's Security Framework for ‘Watch Instantly’ Service, Mar.-Apr. 2009, 18 pgs.
- Server-Side Stream Repackaging (Streaming Video Technologies Panorama, Part 2), Jul. 2011, 15 pgs.
- Text of ISO/IEC 23001-6: Dynamic adaptive streaming over HTTP (DASH), Oct. 2010, 71 pgs.
- Universal Mobile Telecommunications System (UMTS), ETSI TS 126 233 V9.1.0 (Jun. 2011) 3GPP TS 26.233 version 9.1.0 Release 9, 18 pgs.
- Universal Mobile Telecommunications Systems (UMTS); ETSI TS 126 244 V9.4.0 (May 2011) 3GPP TS 26.244 version 9.4.0 Release 9, 58 pgs.
- “Apple HTTP Live Streaming specification”, Aug. 2017, 60 pgs.
- “Data Encryption Decryption using AES Algorithm, Key and Salt with Java Cryptography Extension”, Available at https://www.digizol.com/2009/10/java-encrypt-decrypt-jce-salt.html, Oct. 200, 6 pgs.
- “Delivering Live and On-Demand Smooth Streaming”, Microsoft Silverlight, 2009, 28 pgs.
- “HTTP Based Adaptive Streaming over HSPA”, Apr. 2011, 73 pgs.
- “HTTP Live Streaming”, Mar. 2011, 24 pgs.
- “HTTP Live Streaming”, Sep. 2011, 33 pgs.
- “Information Technology—Coding of Audio Visual Objects—Part 2: Visual”, International Standard, ISO/IEC 14496-2, Third Edition, Jun. 1, 2004, pp. 1-724. (presented in three parts).
- “Java Cryptography Architecture API Specification & Reference”, Available at https://docs.oracle.com/javase/1.5.0/docs/guide/security/CryptoSpec.html, Jul. 25, 2004, 68 pgs.
- “Java Cryptography Extension, javax.crypto.Cipher class”, Available at https://docs.oracle.com/javase/1.5.0/docs/api/javax/crypto/Cipher.html, 2004, 24 pgs.
- “JCE Encryption—Data Encryption Standard (DES) Tutorial”, Available at https://mkyong.com/java/jce-encryption-data-encryption-standard-des-tutorial/, Feb. 25, 2009, 2 pgs.
- “Live and On-Demand Video with Silverlight and IIS Smooth Streaming”, Microsoft Silverlight, Windows Server Internet Information Services 7.0, Feb. 2010, 15 pgs.
- “Microsoft Smooth Streaming Protocol specification”, Jul. 22, 2013, 56 pgs.
- “OpenDML AVI File Format Extensions Version 1.02”, OpenDMLAVI MJPEG File Format Subcommittee. Last revision: Feb. 28, 1996. Reformatting: Sep. 1997, 42 pgs.
- “Single-Encode Streaming for Multiple Screen Delivery”, Telestream Wowza Media Systems, 2009, 6 pgs.
- “The MPEG-DASH Standard for Multimedia Streaming Over the Internet”, IEEE MultiMedia, vol. 18, No. 4, 2011, 7 pgs.
- “Windows Media Player 9”, Microsoft, Mar. 23, 2017, 3 pgs.
- Abomhara et al., “Enhancing Selective Encryption for H.264/AVC Using Advanced Encryption Standard”, International Journal of computer Theory and Engineering, Apr. 2010, vol. 2, No. 2, pp. 223-229.
- Alattar et al., “Improved selective encryption techniques for secure transmission of MPEG video bit-streams”, In Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348), vol. 4, IEEE, 1999, pp. 256-260.
- Antoniou et al., “Adaptive Methods for the Transmission of Video Streams in Wireless Networks”, 2015, 50 pgs.
- Apostolopoulos et al., “Secure Media Streaming and Secure Transcoding”, Multimedia Security Technologies for Digital Rights Management, 2006, 33 pgs.
- Asai et al., “Essential Factors for Full-Interactive VOD Server: Video File System, Disk Scheduling, Network”, Proceedings of Globecom '95, Nov. 14-16, 1995, 6 pgs.
- Beker et al., “Cipher Systems, The Protection of Communications”, 1982, 40 pgs.
- Bocharov et al., “Portable Encoding of Audio-Video Objects, The Protected Interoperable File Format (PIFF)”, Microsoft Corporation, First Edition Sep. 8, 2009, 30 pgs.
- Bulterman et al., “Synchronized Multimedia Integration Language (SMIL 3.0)”, W3C Recommendation, Dec. 1, 2008, https://www.w3.org/TR/2008/REC-SMIL3-20081201/, 321 pgs. (Presented in five parts).
- Cahill et al., “Locally Adaptive Deblocking Filter for Low Bit Rate Video”, Proceedings 2000 International Conference on Image Processing, Sep. 10-13, 2000, Vancouver, BC, Canada, 4 pgs.
- Candelore, U.S. Appl. No. 60/372,901, filed Apr. 16, 2002, 5 pgs.
- Chaddha et al., “A Frame-work for Live Multicast of Video Streams over the Internet”, Proceedings of 3rd IEEE International Conference on Image Processing, Sep. 19, 1996, Lausanne, Switzerland, 4 pgs.
- Cheng, “Partial Encryption for Image and Video Communication”, Thesis, Fall 1998, 95 pgs.
- Cheng et al., “Partial encryption of compressed images and videos”, IEEE Transactions on Signal Processing, vol. 48, No. 8, Aug. 2000, 13 pgs.
- Cheung et al., “On the Use of Destination Set Grouping to Improve Fairness in Multicast Video Distribution”, Proceedings of IEEE INFOCOM'96, Conference on Computer Communications, vol. 2, IEEE, 1996, 23 pgs.
- Collet, “Delivering Protected Content, An Approach for Next Generation Mobile Technologies”, Thesis, 2010, 84 pgs.
- Diamantis et al., “Real Time Video Distribution using Publication through a Database”, Proceedings SIBGRAPI'98. International Symposium on Computer Graphics, Image Processing, and Vision (Cat. No. 98EX237), Oct. 1990, 8 pgs.
- Dworkin, “Recommendation for Block Cipher Modes of Operation: Methods and Techniques”, NIST Special Publication 800-38A, 2001, 66 pgs.
- Fang et al., “Real-time deblocking filter for MPEG-4 systems”, Asia-Pacific Conference on Circuits and Systems, Oct. 28-31, 2002, Bail, Indonesia, 4 pgs.
- Fecheyr-Lippens, “A Review of HTTP Live Streaming”, Jan. 2010, 38 pgs.
- Fukuda et al., “Reduction of Blocking Artifacts by Adaptive DCT Coefficient Estimation in Block-Based Video Coding”, Proceedings 2000 International Conference on Image Processing, Sep. 10-13, 2000, Vancouver, BC, Canada, 4 pgs.
- Huang, U.S. Pat. No. 7,729,426, U.S. Appl. No. 11/230,794, filed Sep. 20, 2005, 143 pgs.
- Huang et al., “Adaptive MLP post-processing for block-based coded images”, IEEE Proceedings—Vision, Image and Signal Processing, vol. 147, No. 5, Oct. 2000, pp. 463-473.
- Huang et al., “Architecture Design for Deblocking Filter in H.264/JVT/AVC”, 2003 International Conference on Multimedia and Expo., Jul. 6-9, 2003, Baltimore, MD, 4 pgs.
- Jain et al., U.S. Appl. No. 61/522,623, filed Aug. 11, 2011, 44 pgs.
- Jung et al., “Design and Implementation of an Enhanced Personal Video Recorder for DTV”, IEEE Transactions on Consumer Electronics, vol. 47, No. 4, Nov. 2001, 6 pgs.
- Kalva, “Delivering MPEG-4 Based Audio-Visual Services”, 2001, 113 pgs.
- Kang et al., “Access Emulation and Buffering Techniques for Steaming of Non-Stream Format Video Files”, IEEE Transactions on Consumer Electronics, vol. 43, No. 3, Aug. 2001, 7 pgs.
- Kim et al., “A Deblocking Filter with Two Separate Modes in Block-based Video Coding”, IEEE transactions on circuits and systems for video technology, vol. 9, No. 1, 1999, pp. 156-160.
- Kim et al., “Tree-Based Group Key Agreement”, Feb. 2004, 37 pgs.
- Laukens, “Adaptive Streaming—A Brief Tutorial”, EBU Technical Review, 2011, 6 pgs.
- Legault et al., “Professional Video Under 32-bit Windows Operating Systems”, SMPTE Journal, vol. 105, No. 12, Dec. 1996, 8 pgs.
- Li et al., “Layered Video Multicast with Retransmission (LVMR): Evaluation of Hierarchical Rate Control”, Proceedings of IEEE INFOCOM'98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century, Cat. No. 98, vol. 3, 1998, 26 pgs.
- List et al., “Adaptive deblocking filter”, IEEE transactions on circuits and systems for video technology, vol. 13, No. 7, Jul. 2003, pp. 614-619.
- Massoudi et al., “Overview on Selective Encryption of Image and Video: Challenges and Perspectives”, EURASIP Journal on Information Security, Nov. 2008, 18 pgs.
- McCanne et al., “Receiver-driven Layered Multicast”, Conference proceedings on Applications, technologies, architectures, and protocols for computer communications, Aug. 1996, 14 pgs.
- Meier, “Reduction of Blocking Artifacts in Image and Video Coding”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 9, No. 3, Apr. 1999, pp. 490-500.
- Nelson, “Smooth Streaming Deployment Guide”, Microsoft Expression Encoder, Aug. 2010, 66 pgs.
- Newton et al., “Preserving Privacy by De-identifying Facial Images”, Carnegie Mellon University School of Computer Science, Technical Report, CMU-CS-03-119, Mar. 2003, 26 pgs.
- O'Brien, U.S. Appl. No. 60/399,846, filed Jul. 30, 2002, 27 pgs.
- O'Rourke, “Improved Image Decompression for Reduced Transform Coding Artifacts”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 5, No. 6, Dec. 1995, pp. 490-499.
- Park et al., “A postprocessing method for reducing quantization effects in low bit-rate moving picture coding”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 9, No. 1, Feb. 1999, pp. 161-171.
Type: Grant
Filed: Apr 11, 2018
Date of Patent: Sep 21, 2021
Assignee: DIVX, LLC (San Diego, CA)
Inventors: Kourosh Soroushian (San Diego, CA), Jason A. Braness (San Diego, CA)
Primary Examiner: William H. Wood
Application Number: 15/950,988
International Classification: G06F 15/173 (20060101); H04L 12/26 (20060101); H04N 21/239 (20110101); H04N 21/2343 (20110101); H04N 21/442 (20110101); H04L 12/801 (20130101); H04L 29/06 (20060101); H04N 21/2662 (20110101); H04L 1/00 (20060101);