SUPPORTING TRANSPORT DIVERSITY AND TIME-SHIFTED BUFFERS FOR MEDIA STREAMING OVER A NETWORK
A device for processing media data includes one or more processors configured to receive a session description protocol (SDP) message including an attribute that defines a time-shifted buffer (TSB) depth, determine an amount of memory for the TSB based on a value of the attribute, allocate the determined amount of memory to the TSB, and store at least a portion of media data associated with the SDP message in the TSB. The value for the attribute may signal the depth of the TSB in terms of playback time in seconds. The attribute may leverage the extensibility of SDP messages through, for instance, “a=” lines. For instance, the TSB depth attribute may correspond to an “a=tsb-length:” attribute.
Latest QUALCOMM Incorporated Patents:
- User equipment (UE)-initiated discontinuous reception (DRX) medium access control (MAC) control element (MAC-CE)
- Techniques for time alignment of measurement gaps and frequency hops
- Configuration for legacy voice support in 5G
- Configuring beam management based on skipped transmissions of signals associated with beam management
- Distributed device management for positioning
This application claims the benefit of U.S. Provisional Application Ser. No. 61/752,456, filed Jan. 15, 2013, and U.S. Provisional Application Ser. No. 61/809,871 filed Apr. 8, 2013, the entire contents of each of which are hereby incorporated by reference.
TECHNICAL FIELDThis disclosure relate to communication systems, and more particularly, to delivery of content via a network.
BACKGROUNDDigital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, digital cameras, digital recording devices, digital media players, video gaming devices, video game consoles, cellular or satellite radio telephones, video teleconferencing devices, and the like. Digital video devices implement video compression techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263 or ITU-T H.264/MPEG-4, Part 10, Advanced Video Coding (AVC), and extensions of such standards, to transmit and receive digital video information more efficiently.
Video compression techniques perform spatial prediction and/or temporal prediction to reduce or remove redundancy inherent in video sequences. For block-based video coding, a video frame or slice may be partitioned into blocks. Each block can be further partitioned. Blocks in an intra-coded (I) frame or slice are encoded using spatial prediction with respect to neighboring blocks. Blocks in an inter-coded (P or B) frame or slice may use spatial prediction with respect to neighboring blocks in the same frame or slice or temporal prediction with respect to other reference frames.
After video data has been encoded, the video data may be packetized for transmission or storage. The video data may be assembled into a video file conforming to any of a variety of standards, such as the International Organization for Standardization (ISO) base media file format and extensions thereof, such as AVC.
Wireless communication networks are widely deployed to provide various communication services such as voice, video, packet data, messaging, broadcast, etc. These wireless networks may be multiple-access networks capable of supporting multiple users by sharing the available network resources. Examples of such multiple-access networks include Code Division Multiple Access (CDMA) networks, Time Division Multiple Access (TDMA) networks, Frequency Division Multiple Access (FDMA) networks, Orthogonal FDMA (OFDMA) networks, and Single-Carrier FDMA (SC-FDMA) networks.
The 3rd Generation Partnership Project (3GPP) Long Term Evolution (LTE) represents a major advance in cellular technology as an evolution of Global System for Mobile communications (GSM) and Universal Mobile Telecommunications System (UMTS). The LTE physical layer (PHY) provides a highly efficient way to convey both data and control information between base stations, such as an evolved Node Bs (eNBs), and mobile devices, such as UEs. In prior applications, a method for facilitating high bandwidth communication for multimedia has been single frequency network (SFN) operation. SFNs utilize radio transmitters, such as, for example, eNBs, to communicate with subscriber UEs. In unicast operation, each eNB is controlled so as to transmit signals carrying information directed to one or more particular subscriber UEs. The specificity of unicast signaling enables person-to-person services such as, for example, voice calling, text messaging, or video calling.
SUMMARYIn general, this disclosure describes techniques for streaming media data over a network. For example, this disclosure describes techniques for receiving media data via various types of transports, e.g., any of unicast, broadcast, and/or multicast. For instance, a redirector/proxy unit may cause a streaming client to retrieve media data from the Internet directly via unicast, or when a broadcast or multicast service is available, from a broadcast or multicast middleware unit. Alternatively, the redirector/proxy unit may retrieve the media data on behalf of the streaming client when the broadcast or multicast service is not available.
This disclosure also describes techniques related to buffering retrieved media data. For instance, retrieved media data may be stored in a time-shifted buffer (TSB). This disclosure describes techniques for signaling a size of the TSB, e.g., in terms of playback time for the media content, such that an appropriate amount of memory can be allocated for the TSB. In this manner, the media data can be played back at a later time and/or played back using a trick mode, e.g., fast forward or rewind.
In one example, a method of retrieving media data includes, by a proxy unit: obtaining mapping information that maps an identifier for media data to a resource location based on a service for retrieving the media data, wherein the service defines a type of transport for transporting the media data, receiving a request for the media data from a streaming client, determining whether the service is available, and, when the service is available, causing the streaming client to receive the media data from a unit that receives the media data using the service from the resource location, based on the mapping information.
In another example, a device for retrieving media data includes a proxy unit configured to obtain mapping information that maps an identifier for media data to a resource location based on a service for retrieving the media data, wherein the service defines a type of transport for transporting the media data, receive a request for the media data from a streaming client, determine whether the service is available, and, when the service is available, cause the streaming client to receive the media data from a unit that receives the media data using the service from the resource location, based on the mapping information.
In another example, a device for retrieving media data includes means for obtaining mapping information that maps an identifier for media data to a resource location based on a service for retrieving the media data, wherein the service defines a type of transport for transporting the media data, means for receiving a request for the media data from a streaming client, means for determining whether the service is available, and means for causing, when the service is available, the streaming client to receive the media data from a unit that receives the media data using the service from the resource location, based on the mapping information.
In another example, a computer-readable storage medium has stored thereon instructions that, when executed, cause a processor to obtain mapping information that maps an identifier for media data to a resource location based on a service for retrieving the media data, wherein the service defines a type of transport for transporting the media data, receive a request for the media data from a streaming client, determine whether the service is available, and cause, when the service is available, the streaming client to receive the media data from a unit that receives the media data using the service from the resource location, based on the mapping information.
In another example, a method of processing media data includes receiving a session description protocol (SDP) message including an attribute that defines a time-shifted buffer (TSB) depth, determining an amount of memory for the TSB based on a value of the attribute, allocating the determined amount of memory to the TSB, and storing at least a portion of media data associated with the SDP message in the TSB.
In another example, a device for processing media data includes one or more processors configured to receive a session description protocol (SDP) message including an attribute that defines a time-shifted buffer (TSB) depth, determine an amount of memory for the TSB based on a value of the attribute, allocate the determined amount of memory to the TSB, and store at least a portion of media data associated with the SDP message in the TSB.
In another example, a device for processing media data includes means for receiving a session description protocol (SDP) message including an attribute that defines a time-shifted buffer (TSB) depth, means for determining an amount of memory for the TSB based on a value of the attribute, means for allocating the determined amount of memory to the TSB, and means for storing at least a portion of media data associated with the SDP message in the TSB.
In another example, a computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to receive a session description protocol (SDP) message including an attribute that defines a time-shifted buffer (TSB) depth, determine an amount of memory for the TSB based on a value of the attribute, allocate the determined amount of memory to the TSB, and store at least a portion of media data associated with the SDP message in the TSB.
In general, this disclosure describes techniques for supporting various types of transport mechanisms for streaming media data, such as audio and/or video data, through a network. For example, an application service client (which may also be referred to as a streaming client) may be configured to interact with a proxy unit to retrieve media data either according to a unicast protocol or a broadcast (or multicast) protocol. In some examples, the proxy unit may determine whether the media data can be retrieved using the broadcast protocol, e.g., based on whether a client device is within an area of coverage of a service provider for delivering media data using the broadcast protocol, and select either the unicast protocol or the broadcast protocol based on whether the client device is within the area of coverage. The client device may execute the application service client and/or include the proxy unit. In some examples, the client device may execute the application service client, and the proxy unit may be included in a device separate from the client device.
The techniques of this disclosure may be used in conjunction with various application-layer protocols for streaming media data. For example, the techniques of this disclosure may be used in conjunction with Dynamic Adaptive Streaming over HTTP (DASH), which utilizes HTTP to stream media data. As another example, the techniques of this disclosure may be used in conjunction with Real-time Transport Protocol (RTP) or Real-Time Streaming Protocol (RTSP). In these and other instances, an application service client (e.g., an RTP client, an RTSP client, or a DASH client) may be transport-agnostic, in the sense that the application service client need not be aware of a transport mechanism used to retrieve media data. For instance, the application service client need not be aware of whether an underlying transport mechanism corresponds to a unicast protocol or a broadcast or multicast protocol.
As discussed in greater detail below, the proxy unit (which may also be referred to as a redirector/proxy unit) may be configured to receive a request from an application service client, where the request specifies certain media data. The proxy unit may determine whether the media data can be retrieved using a particular transport mechanism, e.g., a broadcast protocol or a unicast protocol. The proxy unit may then cause the application service client to receive the media data using one of the transport mechanisms (based on availability, preference, reliability, and/or other factors). For example, if the broadcast protocol is more preferable than the unicast protocol, and the broadcast protocol is available, the proxy unit may cause the application service client to receive the media data via the broadcast protocol, whereas if the broadcast protocol is not available, the proxy unit may cause the application service client to receive the media data via the unicast protocol.
As one example, with respect to DASH, media data may be available from one or more servers, e.g., a broadcast server and a unicast server, as a broadcast and/or a unicast service. A DASH client may receive a modified media presentation description (MPD) for the media data that indicates that the media data is available from a proxy unit (rather than the server(s)). When the DASH client sends a request for media data to the proxy unit, the proxy unit may determine whether a unicast protocol or a broadcast protocol is available for retrieving the requested media data. If the broadcast protocol is available, a broadcast receiving unit (e.g., a multimedia broadcast multicast services (MBMS) or evolved MBMS (eMBMS) middleware unit) may receive the media data, and the proxy unit may cause the DASH client to send a request for the media data to the broadcast receiving unit. On the other hand, if the broadcast protocol is not available, the proxy unit may cause the DASH client to send a request for the media data to a unicast server, to retrieve the media data using unicast. Alternatively, the proxy unit may retrieve the media data from the unicast server, and then provide the retrieved media data to the DASH client. In some examples, the unicast server and the broadcast server may correspond to the same server.
As another example, with respect to RTP or RTSP, an RTP client (which, alternatively, may correspond to an RTSP client) may submit DESCRIBE, SETUP, and PLAY commands to a proxy unit. In response to the DESCRIBE command, the proxy unit may provide a modified session description protocol (SDP) message to the RTP client. The modified SDP message may specify an address of the proxy unit as the address from which the media data is available. This modified SDP message may cause the RTP client to send the SETUP and PLAY commands to the proxy unit. When the proxy unit determines that the broadcast protocol is available, the proxy unit may send SETUP and PLAY commands to a broadcast receiving unit (e.g., an MBMS or eMBMS middleware unit), which may receive the media data and forward the media data to the RTP client. On the other hand, when the proxy unit determines that the broadcast protocol is not available, the proxy unit may retrieve the media data from a unicast server, and then provide the retrieved media data to the RTP client.
In some examples, components for performing the techniques of this disclosure may include an application service client, a proxy unit, and a broadcast receiving unit. In various examples, a client device may include any or all of these components, alone or in any combination. Alternatively, the client device may include the application service client, and the proxy unit and/or the broadcast receiving unit may be included in one or more devices that are separate from the client device.
Moreover, this disclosure also describes techniques related to signaling data for a time-shifted buffer (TSB). A client device (also referred to herein as “user equipment”) may allocate memory to form a TSB to be used for holding media data in support of various trick modes, such as fast forward, rewind, replay, or the like. In general, a trick mode may refer to a playback mode in which media data is played back in an order or rate other than a defined output order. For instance, in fast forward, certain media data may be skipped. As another example, in rewind, an output order for media data may be inverted, and certain media data may also be skipped. To skip media data, e.g., with respect to video data, only pictures coded using an intra-coding mode could be displayed, skipping inter-coded pictures.
In accordance with the techniques of this disclosure, data may be signaled in, e.g., an SDP message, for instantiating a TSB. For example, a TSB attribute may be signaled, with a value defining, in seconds, an amount of media data that can be stored in the TSB. A client device may calculate an amount of memory needed for the TSB based on the value of the TSB attribute. This calculation may involve, for video data as an example, a frame rate of video data in the media data. The client device may calculate a memory size for the TSB based on an average amount of data per picture, a number of pictures in a period of time (based on frame rate), and a length of time as defined by the TSB attribute. Likewise, the client device may further determine an amount of memory needed for audio data, text data, or other data or media for the period of time. Thus, the client device may allocate this data to the TSB for storing media data to perform trick modes. Furthermore, the client device may perform trick modes for data received via a broadcast protocol, such as MBMS or eMBMS.
In other words, in some examples, the techniques of this disclosure include a middleware architecture for supporting features of Real-time Protocol/Real-time Streaming Protocol (RTP/RTSP) streaming over evolved Multimedia Broadcast Multicast Service (eMBMS) network. These features include time-shifted buffer and transport diversity (e.g., unicast to broadcast switching or vice-versa for content delivery). For the time-shifted buffer (TSB) feature, the architecture may support extensions of the session description protocol (SDP) to include data signaling a buffer depth. As one example of the transport diversity feature, this disclosure describes proposes an architecture where applications consuming the content need not be aware of the details of transport switching from unicast to broadcast or vice-versa, as this switching may be managed in the middleware level.
In HTTP streaming, frequently used operations include GET and partial GET. The GET operation requests the retrieval of a whole file associated with a given uniform resource identifier (URI) (e.g., URL or URN). The partial GET operation requests the retrieval of a byte range (subset) of a resource. Thus, movie fragments may be provided for HTTP streaming, because a GET or partial GET operation can retrieve one or more individual movie fragments. In a movie fragment, there can be several track fragments of different tracks. In HTTP streaming, a media presentation may be a structured collection of data that is accessible to the client. The client may request and download media data information to present a streaming service to a user.
In the example of streaming DASH data using HTTP, there may be multiple representations for video and/or audio data of multimedia content. The manifest of such representations may be defined in a Media Presentation Description (MPD) data structure. A media presentation may correspond to a structured collection of data that is accessible to an HTTP streaming client device. The HTTP streaming client device may request and download media data information to present a streaming service to a user of the client device. A media presentation may be described in the MPD data structure, which may include dynamic updates of the MPD.
A media representation may contain a sequence of one or more periods. Periods may be defined by a Period element in the MPD. Each period may have an attribute start in the MPD. The MPD may include a start attribute and an availablityStartTime attribute for each period. For live services, the sum of the start attribute of the period and the MPD attribute availableStartTime may specify the availability time of the period in UTC format, in particular the first Media Segment of each representation in the corresponding period. For on-demand services, the start attribute of the first period may be 0. For any other period, the start attribute may specify a time offset between the start time of the corresponding Period relative to the start time of the first Period. Each period may extend until the start of the next Period, or until the end of the media presentation in the case of the last period. Period start times may be precise. They may reflect the actual timing resulting from playing the media of all prior periods.
Each period may contain one or more representations for the same media content. A representation may be one of a number of alternative encoded versions of audio, video or other data. The representations may differ by encoding types, e.g., by bitrate, resolution, and/or codec for video data and bitrate, language, and/or codec for audio data. The term representation may be used to refer to a section of encoded audio or video data or other media or data corresponding to a particular period of the multimedia content and encoded in a particular way.
Representations of a particular period may be assigned to a group indicated by a group attribute in the MPD. Representations in the same group are generally considered alternatives to each other. For example, each representation of video data for a particular period may be assigned to the same group, such that any of the representations may be selected for decoding to display video data of the multimedia content for the corresponding period. The media content within one period may be represented by either one representation from group 0, if present, or the combination of at most one representation from each non-zero group, in some examples. Timing data for each representation of a period may be expressed relative to the start time of the period.
A representation may include one or more segments. Each representation may include an initialization segment, or each segment of a representation may be self-initializing. When present, the initialization segment may contain initialization information for accessing the representation. In general, the initialization segment does not contain media data. A segment may be uniquely referenced using an identifier, such as a uniform resource identifier (URI) (e.g., uniform resource locator (URL) or uniform resource name (URN)). The MPD may provide the identifiers for each segment. In some examples, the MPD may also provide byte ranges in the form of a range attribute, which may correspond to the data for a continuous byte range of segment within a resource (e.g., file) accessible by referencing the corresponding URI.
Each representation may also include one or more media components, where each media component may correspond to an encoded version of one individual media type, such as audio, video, or timed text (e.g., for closed captioning). Media components may be time-continuous across boundaries of consecutive media segments within one representation.
The techniques described herein may be used for the wireless networks and radio technologies mentioned above as well as other wireless networks and radio technologies. For clarity, certain aspects of the techniques are described below for LTE, and LTE terminology is used in much of the description below.
RTP/RTSP streaming is a common protocol for supporting real-time streaming services. The 3GPP specification for eMBMS services describes the architecture for RTP streaming over Long Term Evolution (LTE) networks. However, this disclosure, recognizes two problems with RTP/RTSP, and discusses techniques that may be used to overcome these problems.
The Time-shifted buffer (TSB) is a feature that may be used to store a portion of media content in User Equipment (UE). The buffer allows users to move the playback point of media content forward or backward. In this process, the UE needs to have been informed of the buffer depth, which provides an indication of how much time worth of content the UE needs to accumulate at the device to allow for trick mode playback, e.g., fast-forward and rewind of playback. In one implementation of a VLC media player, the TSB feature is enabled by providing the buffer depth as an argument to the VLC player at startup. The process is not automatic, and instead requires intervention of a user to provide the buffer depth argument and consequently enable the TSB. This disclosure describes techniques for supporting a TSB in an eMBMS-enabled middleware layer, such as a multicast services device client (MSDC).
The session description protocol (SDP) describes endpoint information of a streaming service and its media stream (session) related parameters, collectively known as a “session profile.” Therefore, if a streaming service is available in different transport methods, multiple profiles that describe different transport/endpoint parameters for the service are needed. For example, a broadcast real-time RTP-based service can refer to a multicast IP address and port for its media streams, whereas the unicast version of the service may refer to a unicast IP address and port. There may be different representations of the media streams for broadcast and unicast delivery methods. In LTE networks, the MSDC provides delivery of streaming services to eMBMS-enabled applications. To support switching between unicast and other (e.g., broadcast or multicast) transport methods, it needs to utilize multiple session profiles, the choice of which will be based mainly on whether a UE is out of coverage or in coverage of broadcast. Whenever a UE moves to broadcast coverage area, the MSDC may use the broadcast-enabled version of the SDP profile to set up a connection and consume content. However, in the reverse scenario, when a UE moves out of broadcast coverage, MSDC may need to use the unicast SDP and set up a unicast connection with the remote RTSP server to consume unicast content. To keep the transitions between unicast and broadcast hidden from users or applications, this disclosure describes a mechanism for seamless transition.
This disclosure describes, among other techniques, techniques related to supporting the TSB and techniques for supporting transport switching, which may be used alone, together, and/or in any combination with other techniques described in this disclosure. For enabling the TSB feature in RTP-based streaming over eMBMS, this disclosure describes techniques for using the extension of the SDP mechanism to signal the time-shifted buffer depth value. For the transport switching problem, this disclosure describes a unified approach for an SDP description that includes both unicast and broadcast descriptions. Also, in order to make the transitions between broadcast and unicast seamless to the applications/users, this disclosure describes a middleware-based solution where the middleware handles the redirection of content using unified SDP, hiding the process from the users.
In accordance with aspects of the subject matter of this disclosure, there are provided methods, apparatus, and an architecture for deployment within a client device or collection of devices, to provide a layer of abstraction wherein multiple different types of transport mechanisms/protocols (e.g., eMBMS, digital video broadcast (DVB), etc.) may be employed to, e.g., deliver meta-data and digital content to applications of a client. Applications need not be cognizant of the details of each individual transport. The disclosure may include the option of a configurable set of policies which may be used to determine the choice, location, or timing of availability of data and meta-data that are delivered to the applications.
Applications that access web content on the Internet may identify the content (or its constituent parts) using a universal resource identifier (URI), which may be a universal resource locator (URL) that utilizes the HTTP or HTTP secure (HTTPS) schemes. For example, in the context of MPEG-DASH, URL references containing the HTTP or HTTPS schemes may be resolved using the HTTP or HTTPS protocols. In addition, HTTP may be used to fetch data referenced by the URLs. The DASH standard may utilize non-HTTP URLs, and the disclosure may relate to cases where arbitrary requests (including HTTP-based requests) from clients are handled in a flexible manner that provides an option to cause requests to be directed to a different location, at a different time, and/or instruct the requesting client to access alternative content.
Techniques of this disclosure may involve the instantiation of a re-director function that handles client requests for content. The re-director (also referred to herein as a redirector/proxy) may be able to apply processing to the material identified in a client request, invoke a method or algorithm to compare the result of this processing against a table of matching criteria, and indicate to the client an alternative location, access method, timing information, or content (e.g., in a file) so that the client may know where to perform a subsequent request. In another aspect, the re-director may perform the request for the content on the client's behalf and fetch the requested content. In this case, the re-director may act in a proxy or recursive function.
A re-director/proxy 104 may receive the application's 101 request via an application service client 102 and process or parse the request to determine the requested content. The re-director/proxy 104 may compare the result against a table (or other suitable type of data structure) containing matching criteria to determine if a match has occurred. If a match occurs, the re-director/proxy 104 may determine a redirection target. The target may be delivered to the application 101. In another aspect, the re-director/proxy 104 may act in a recursive fashion and the application 101 may not need to process the redirection target. For example, the re-director/proxy 104 may fetch the content based on the redirection target on behalf of the application 101.
The re-director/proxy 104 may be co-located with the application 101 on the same device, or the re-director/proxy 104 may be located on a separate device. In the case of a co-located re-director/proxy 104, the application 101 may communicate with the re-director/proxy 104 through HTTP, application programming interface (API), inter-process communication (IPC), etc. In the case of the re-director/proxy 104 located on a separate device, the application 101 may communicate with the re-director (via the application service client 102) through HTTP, etc.
Subsequent to the determination of a “match” (or non-match), a re-direction target may be provided to the application 101. The target may indicate alternative content object(s) to access in lieu of content object(s) requested by the application 101. There may be various methods by which the re-direction target may be indicated to the application 101.
Signaling between components of the system 100 may be performed using an API or IPC. This aspect, the API or IPC mechanism may be utilized to provide communication between the re-director/proxy 104 and the application 101. In the case of using the API, the re-director/proxy 104 may invoke methods or procedures implemented within the application 101 or application library to indicate the redirect target.
Signaling between components of the system 100 may be performed through meta-data re-writing. In this aspect, the meta-data (e.g., a DASH MPD) may be re-written so as to indicate alternative references for objects. For example, a URI present in the original meta-data may be re-written to form an alternative URI for accessing equivalent or replacement content.
Signaling between components of the system 100 may be performed using indications in a communication protocol. In this aspect, an application layer communications protocol (e.g., HTTP) may be utilized in such a way as to communicate signaling information to the application 101. In one variant, the HTTP response code category of “re-direct” may be used (e.g., this may correspond to codes of the form 3xx, where x may be a number from zero through nine). In one subcase of this variant, e.g., the code 300 (indicating multiple choices available) may be used, in which the re-director/proxy 104 provides a vector of references (e.g., URIs) from which an application 101 may select one or more for use.
The matching function may give the result of comparing object references originated from the application 101 (e.g., a request URI) with object references present in a data structure (e.g., a file, database, matching table, etc.).
By way of non-limiting examples, in one aspect, matching criteria may be expressed as URI prefixes and a single preferred match occurs is indicated by the match with the longest prefix (e.g., as determined by the number of bits or bytes in common). In another aspect, each potential match may be assigned a corresponding priority number, and the entry with the largest (or smallest) priority number may be selected.
In another aspect, matching criteria may be expressed as regular expressions, and the preferred match may be determined using the largest number of matching characters. In another variation, regular expressions may be used to express the matching criteria and the potential match with the largest (or smallest) priority number may be selected.
The matching algorithm or contents of the data structure discussed above may be based on policies that may be pre-determined or pre-configured. In another aspect, the policies may be dynamically added or deleted from a logical policy database. The policy database may be implemented using various data structures and is not limited to any particular type of data structure or database implementation.
Policies may be used to determine the operation of the matching algorithm in a number of ways including, for example, prioritizing or de-prioritizing matching criteria from one source over other(s) (e.g., one transport type may be favor/disfavored over (s)); removing matching criteria as a function of source, time, redirection target, and/or matching criteria value; adding matching criteria as a function of source or time or redirection target or matching criteria value; and/or modifying matching criteria as a function of source or time or redirection target or matching criteria value.
The example system 100 may include a transport middleware 110 for requesting content based on the redirected location. The transport middleware 110 may include a number of transport mechanisms/protocol such as Transport A 110A through Transport R 110R. Each transport mechanism (Transport A 110A through Transport R 110R) may have a corresponding module capable of acquiring a list and details of file information and producing, e.g., a summarizing URI or common URI prefix. These modules may convey this information via a communication mechanism (e.g., IPC, API, or protocol) to the Controller 106 which may apply policy and resolution, and which may in turn program the re-director/proxy 104 subject to the policy. The transport mechanisms (Transport A 110A through Transport R 110R) may include, e.g., eMBMS, DVB-T2, DVB-S, other DVB-family broadcast technologies, or the like. Each transport middleware may include a cache for storing content or other data. For example, the transport middleware may cache a starting segment of content so that content delivery to the application 101 may appear instantaneous to the user. The transport middleware 110 may request the content from a source such as a content server 120. Additionally or alternatively, the client may request the content over a network, including the Internet 122. For the re-director/proxy 104 configured for proxy or “recursive” operation, the re-director/proxy 104 may request content via the transport middleware 110 or via a network or the Internet 122. The re-director/proxy 104 may be pre-configured to operate in one of the re-direction or proxy roles. The role of the re-director/proxy 104 may be determined at run-time, e.g., based on a set of rules.
The application 101, which may or may not be related to application 101 in
In this example, more than one transport may be available and various content may be available over the various transports at different times. In particular, various services may be available, where the various services may each define different respective types of transports, such as broadcast, multicast, unicast, or the like, for transporting media data. For example, the eMBMS and/or one or more of the DVB-family of transports may be available. Further, the available media content (e.g., files) may be expressed using, e.g., an application service description. One example of an application service description is an MBMS user service description (USD) metadata fragment, such as an MPD fragment in the case of DASH content delivered as an MBMS User Service, in describing the various available media components. Another example of an application service description is an Apple HLS (HTTP Live Streaming) manifest file, in the case of HLS content delivered as MBMS User Service. Each transport mechanism may have a corresponding module capable of acquiring the list and details of file information and producing a summarizing URI or common URI prefix. These modules may convey this information via a communication mechanism (e.g., IPC, API or other protocol) to the controller 106 which may apply policy and resolution, and which in turn programs the re-director/proxy 104 subject to the policy.
Content server 310 may broadcast (e.g., using LTE RAN, DVB-T, etc.) a service listing (e.g., USD, ESG), which may be received by the transport middleware 110. An appropriate module (e.g., one of Transport A 110A through Transport R 110R) parses the service listing and processes the information into a form which is not transport specific. The transport middleware 110 may convey the results (e.g., an identifier of a service and data defining a list of available files, also referred to as a file availability list) to the controller 106 (324). The controller 106 may process the file availability list in conjunction with the access policy to produce a set of mappings (325). The mappings may include which URIs (or URI prefixes) should be re-directed to which server (e.g., files available over MBMS may be available from the server instantiated within or associated with the MBMS middleware).
Once the application service client 102 is invoked, the application service client 102 may issue a request (e.g., an HTTP request) via the configured proxy address to obtain, e.g., the MPD contents (326). The request may be communicated to the re-director/proxy 104. The re-director/proxy 104 may perform the matching algorithm to determine matching criteria or, if a match does not occur, another indicator (e.g., a default re-direction target, error indication, or copy of the original request). Redirector/proxy 104 may then provide a re-direction target or error to the application service client 102, e.g., using HTTP (327A). The application service client 102, having observed a code (e.g., HTTP code) indicating re-direction (e.g., 3xx type code) responds depending on the code type. For example, code types other than 300 include an HTTP “Location:” header that may indicate the alternate location for a resource. Upon receiving such an indication, the application service client 102 may re-try its request using the alternate location indicated. In this example, the re-director/proxy 104 indicates a particular URI should be directed to the server integrated within the MBMS middleware. Accordingly, the application service client 102 may submit a request to the transport middleware 110 to obtain, e.g., the MPD (327B) in response to such a redirection code.
Once the application service client 102 has acquired the MPD information, the application service client 102 may determine which media segments to access (e.g., which “representation” in the case of MPEG-DASH) and issue an HTTP-based request (328) to the redirector/proxy 104. Using the mechanism described above (e.g., the type of transport associated with the service identified by the service ID), the re-director/proxy 104 may provide a redirect target (329) to the application service client 102. The target may be a default value, a copy of the request (indicating the application service client 102 should handle the request), or a reference to a different server. The targets then serve as the locations used by the application service client 102 to obtain media segments. That is, assuming that the content is available using the service, the application service client 102 may submit a content request to the transport middleware 110 (330A), and in response, the transport middleware 110 may deliver the content to the application service client 102 (331A). For content available through other sources (e.g., when the service is not available), the application service client 102 may obtain the content via other methods (e.g., from other servers, storage/cache or the Internet). For example, the application service client 102 may send a content request to content server 310 (330B), and, in response, the content server 310 may deliver the requested content to the application service client 102 (331B).
In some cases, the URIs requested from the application service client 102, at step 326 or step 328, may be unknown to the re-director/proxy 104. In such cases, errors (e.g., HTTP code 404) may be generated to indicate to the application service client 102 that it may try to request a different segment or stop using the re-director/proxy 104 (e.g., use a direct Internet access request instead). Alternatively, the re-director 104 may use a location header equivalent to the incoming request to suggest non-proxy operation. Another aspect may involve the re-director/proxy 104 acting as a proxy or web proxy, in which case it may access the Internet or another network or cache on behalf of the application service client 102, e.g., as illustrated below in example process flow 400 of
For the 300-type HTTP error codes, the application service client 102 may be presented with a vector of choices that may not be present in the “Location:” header. In this case, the application service client 102 may choose among the multiple choices provided and, depending on the particulars of the media encoding format, re-initialize its decoding state. The scenario may arise, for example, if a re-direction occurs in the middle of a playback scenario to a representation encoded in a way other than that which the application service client 102 has been processing so far.
Whichever process the application service client 102 uses to select its next media request(s)/access(es), the application service client 102 may initiate the request(s)/access(es) based on the re-directed information. Subsequent requests/access may pass through the re-director initially, providing the opportunity for the re-director to intervene and effectively modify the location (and other characteristics) from which segments (e.g., meta-data and/or media segments stored as files) are retrieved.
The application 101 may be a provisioning application. Although described with respect to application 101 of
In this example, more than one transport mechanism may be available and various media may be available over the various transports at different times. The various types of transports may be defined by respective services. For example, the eMBMS and/or one or more of the DVB family of transports may be available. Further, the available media files may be expressed using, e.g., the MPD fragment of the eMBMS USD, and via DVB electronic service guide (ESG) for DVB transport. Each transport mechanism, represented by the transport middleware 110, may have a corresponding module capable of acquiring a list of available services and details of file information and for producing a summarizing URI or common URI prefix. These modules may convey this information via a communication mechanism (e.g., IPC, API or other protocol) to the controller 106 which may apply policy and resolution, and which in turn programs the re-director/proxy 104 subject to the policy.
Content server 310 may broadcast a service listing (e.g., USD, ESG) and received by the transport middleware 110 (403). An appropriate module (e.g., one of Transport A 110A through Transport R 110R) parses the service listing and processes the information into a form which is not transport specific. The transport middleware 110 may convey the results to the controller 106 (404). The controller 106 may process the file availability list in conjunction with the access policy to produce a set of mappings (405). The mappings may include which URIs (or URI prefixes) should be re-directed to which server instantiated (e.g., files or media available over MBMS may be available from the server instantiated within or associated with the MBMS middleware).
Once the application service client 102 is invoked, the application service client 102 may issue a request (e.g., an HTTP request) via the configured proxy address to obtain, e.g., the MPD contents (406). The request may be communicated to the re-director/proxy 104. The re-director/proxy 104 may perform the matching algorithm to determine matching criteria or, if a match does not occur, another indicator (e.g., a default re-direction target, error indication, or copy of the original request). In the case of an error, the error may be provided to the application service client 102. If a match occurs, the re-director/proxy 104 may act as a proxy and retrieve the content on behalf of the application service client 102. The targets then serve as the locations used by the re-director/proxy 104, acting as a proxy, to obtain media segments. That is, the redirector/proxy 104 may submit a content request to the transport middleware 110 (407A), and the transport middleware may deliver the content to the redirector/proxy 104 (408A). For content available through other sources, the re-director/proxy 104, acting as a proxy, may obtain the content via other servers, from storage/files or the Internet. That is, the redirector/proxy 104 may submit a content request to content server 310 (407B), and the content server 310 may deliver the requested content to the redirector/proxy (408B).
As illustrated, in one example, the apparatus 600 may include an electrical component or module 602 for receiving a request for content. The apparatus 600 may include an electrical component or module 604 for determining whether a match exists for the request based on a set of rules. The apparatus 600 may include an electrical component or module 606 for sending a re-direction to the multimedia client in response to determining a match exists.
In related aspects, the apparatus 600 may optionally include a processor component 610 having at least one processor, in the case of the apparatus 600 configured as a network entity. The processor 610, in such case, may be in operative communication with the components 602-606 or similar components via a bus 612 or similar communication coupling. The processor 610 may effect initiation and scheduling of the processes or functions performed by electrical components or modules 602-606.
In further related aspects, the apparatus 600 may include a network interface component 614 for communicating with other network entities. The apparatus 600 may optionally include a component for storing information, such as, for example, a memory device/component 616. The computer readable medium or the memory component 616 may be operatively coupled to the other components of the apparatus 600 via the bus 612 or the like. The memory component 616 may be adapted to store computer readable instructions and data for performing the activity of the components 602-606, and subcomponents thereof, or the processor 610. The memory component 616 may retain instructions for executing functions associated with the components 602-606. While shown as being external to the memory 616, it is to be understood that the components 602-606 can exist within the memory 616.
As discussed above with respect to
Initially, policy information may be provided to the system including DASH client 802, playback application 804, HTTP redirector/proxy 818, controller 814, and MSDC 808. Such policy information may be stored within policy database 816 (830). This policy information may influence the action of controller 814. Controller 814 may be implemented in hardware, software, firmware, or any combination thereof. It is presumed that, when implemented in software and/or firmware, requisite hardware may is also provided, such as one or more hardware-based processing units for executing instructions of software corresponding to controller 814.
Initially, playback application 804 may provision identifying information, such as information identifying a proxy endpoint (e.g., host name or address, protocol, and protocol port number) (832). This information may be provided by an optional provisioning application. DASH client 802 may be configured with the identifying information (834).
In the example of
MSDC 808 processes the USD-specific information into a form which is not transport-specific and conveys the results to controller 814 (840). Controller 814 may then process the file availability list in conjunction with the access policy to produce a set of mappings (842). The mappings indicate which URIs (or URI prefixes) should be re-directed to which server instantiated (e.g., files available over MBMS may be available from server 812, instantiated within or associated with MSDC 808). In other words, HTTP redirector/proxy 818 may obtain mapping information that maps an identifier for media data to a resource location based on a service for retrieving the media data, wherein the service defines at least one of a plurality of types of transports (e.g., broadcast, unicast, or multicast) for transporting the media data. For example, the service may define broadcast or multicast transport types for transporting the media data, in which case MSDC 808 may receive the data. MSDC 808 may store received media data in cache 810, for subsequent retrieval by, e.g., DASH client 802, as discussed below.
Once invoked, DASH client 802 may issue an HTTP request via the configured proxy address (to HTTP redirector/proxy 818) to obtain the MPD contents (844). Likewise, HTTP redirector/proxy 818 may receive the HTTP request from DASH client 802. The HTTP request may be a request for media data. HTTP redirector/proxy 818 may perform the matching algorithm to determine matching criteria or, if a match does not occur, another indicator (e.g., a default re-direction target, error indication, or copy of the original request). Thus, using the matching criteria on the mapping information, HTTP redirector/proxy 818 may determine whether a particular service is available, e.g., a service defining broadcast or multicast for transporting the requested media data. HTTP redirector/proxy 818 may then provide a re-direction target or error to DASH client 820 using HTTP (846), based on the matching algorithm.
DASH client 802, having observed an HTTP code indicating a re-direction (e.g., a 3xx type HTTP code) may respond depending on the code type. For types other than 300, an HTTP “Location:” header may indicate the alternate location for a resource. Upon receiving such an indication, DASH client 802 may re-try its request using the indicated alternative location. In this example, HTTP re-director/proxy 818 may indicate that a particular URI should be directed to the server integrated within MSDC 808, from which DASH client 802 may obtain the MPD (848).
Once DASH client 802 has acquired the MPD information, DASH client 802 may determine what media files to access (e.g., which “representation” in the case of MPEG-DASH) and issue one or more HTTP-based requests to retrieve the determined media files (852). Using the mechanism described above, HTTP redirector/proxy 818 provides a redirect target to DASH client 802 (854). The target may be a default value, a copy of the request (indicating the client should handle the request), or a reference to a different server. The targets then serve as the locations used by DASH client 802 to obtain media segments, e.g., from MSDC 808 (856). In this manner, when a service defining, e.g., broadcast or multicast for transporting media is available, HTTP redirector/proxy 818 may cause DASH client 802 to receive the requested media data from a unit (e.g., MSDC 808) that receives the media data using the service from the resource location indicated in the mapping information. Alternatively, the redirection target may correspond to a network location available via some network, including Internet 806.
In some cases, the URIs requested from the client (steps 844/852) may be unknown to HTTP redirector/proxy 818. In such cases, errors (e.g., HTTP code 404) could be generated to indicate to DASH client 802 that DASH client 802 should try to request a different segment or change to not use HTTP redirector/proxy 818 (i.e., instead use a direct network or Internet access request). Alternatively, HTTP redirector/proxy 818 may use a Location: header equivalent to the incoming request to suggest non-proxy operation. One further variant would involve HTTP redirector/proxy 818 acting as a conventional web proxy, in which case HTTP redirector/proxy 818 may access Internet 806 on behalf of DASH client 802.
In this manner, the techniques of
Furthermore, if the resource location corresponds to a network location (e.g., a location within or accessible via Internet 806), HTTP redirector/proxy 818 may be configured to determine whether media data specified in a request matches media data in mapping information, and when the media data specified in the request matches the media data in the mapping information, send a redirect message to the application service client specifying the network location to which the identifier for the media data is mapped in the mapping information to cause the application service client to retrieve the media data from the network location.
Likewise, HTTP redirector/proxy 818 represents an example of a proxy unit configured to obtain mapping information that maps an identifier for media data to a resource location based on a service for retrieving the media data, wherein the service defines at least one of a plurality of types of transports (e.g., broadcast, multicast, or unicast) for transporting the media data, receive a request for the media data from an application service client, determine whether the service is available, and, when the service is available, cause the application service client to receive the media data from a unit that receives the media data using the service from the resource location, based on the mapping information.
In the example of
In addition, as described in greater detail below, MSDC 1010 may receive an SDP message including an attribute that defines a time-shifted buffer (TSB) depth for TSB 1012 of
In order to signal the TSB depth, the techniques of this disclosure may be used to leverage the extension mechanism of SDP. SDP “a=” (attribute) lines may be used for providing extensions to the general session description. The following semantics may be used to signal data related to the TSB, as one example:
-
- TSB-attribute=“a=tsb-length:” Value
- Value=token
- Value will have seconds as unit
As an example, the following SDP snippet mentioned in Table 1 describes data for the TSB. The buffer depth in this example is 60 seconds, or 1 minute. This implies that the MSDC (e.g., the TSB of the MSDC illustrated in
When data for the TSB is provided in the SDP, the MSDC may allocate an equivalent amount of buffer of the size mentioned in the SDP, and may allow the RTSP/RTP client to fast-forward or rewind playback, with respect to the buffer duration.
One example technique for achieving transport diversity is based upon the solution proposed by U.S. Provisional Application Ser. No. 61/752,456, “ARCHITECTURE SUPPORTING TRANSPORT DIVERSITY,” to Fall et al., filed, Jan. 15, 2013, attorney docket no. 131286P1 (hereinafter, the “Fall provisional application”), the entire contents of which are hereby incorporated by reference. The Fall provisional application proposes common client architecture for supporting diverse transport protocols. The techniques of this disclosure for supporting transport diversity in RTP may be used to extend the techniques described in the Fall provisional application, which relates to DASH-based delivery of contents.
Unlike the DASH protocol, where media representations in Media Presentation Description (MPD) may change based on transport delivery methods, RTP protocol conventionally has no way to differentiate the transport diversity via a single SDP profile directly. This disclosure describes to example techniques to achieve transport diversity.
As one example, eMBMS broadcast definitions of services allow mechanisms to select between unicast and broadcast transport mechanisms. For example, a deliveryMethod element in an eMBMS service definition schema may contain an SDP profile that describes broadcast delivery sessions, while an AlternativeAccessDelivery element may contain references to an RTSP URL for unicast access or PSS SDP file that denotes unicast SDP session information. When a UE is under broadcast network coverage, eMBMS middleware may consume broadcast content using broadcast SDP session information. Otherwise, the middleware may pass the content by connecting to the unicastAccessURI in AlternateAccessDelivery. An example of the eMBMS user service description (USD) schema snippet is shown in
An alternative example to the deliveryMethod carrying broadcast SDP and unicast access URI is to have a unified single SDP that contains both unicast and broadcast session descriptions in different media streams. Since any session description in SDP can contain multiple media stream definitions (for example, one for an audio track and another for a video track), a mechanism may be used to group broadcast and unicast media streams into different sets to provide a unified SDP profile. This can be achieved by a grouping method in SDP. An example of the grouping mechanism is described as follows:
-
- Media stream identification
- Identifies media streams within groups
- Media-attribute=“a=mid:” Identification-tag
- Identification-tag=token
- Group attribute
- Identifies unicast/broadcast media streams
- Group-attribute=“a=group:” Semantics (identification-tag)*
- Semantics=“BROADCAST”|“UNICAST”
- Media stream identification
The following snippet in Table 2 provides an example in which values are provided for group and media stream identifier attributes. In the example below, the “a=group:” lines denote which media streams are used for broadcast and which media streams are used for unicast. In this example, broadcast media streams are denoted by “a=mid:” values 1 and 2 and unicast are denoted by “a=mid:” values 3 and 4, respectively. Therefore, middleware may connect to IP address 224.1.1.2 and ports 30000 and 30002 for broadcast content, and to IP address 131.10.1.2 and ports 26890 and 26892 for unicast streams.
In this manner,
In the examples of
Playback application 1204 may correspond substantially to application 101 (
In this example, the policy manager is provisioned with policies (1230). The eMBMS Middleware (or MSDC) 1208 receives USD descriptions (1232, 1234) for the list of eMBMS services. The MSDC 1208 then passes the SDP profile information for the RTP streaming service and the broadcast coverage notification to the controller 1214 (1236), which in turn matches the SDP information with the list of policies and provides mapping information to the RTSP redirector/proxy 1218 (1238). Mapping information contains data indicating, for each scenario (broadcast or unicast coverage), which SDP profile connection-endpoints to use. In this manner, the RTSP redirector may obtain mapping information that maps an identifier for media data to a resource location based on a service for retrieving the media data. The resource location may correspond to a network address, e.g., an address available via LTE RAN 1206. The service may define at least one of a plurality of types of transports for transporting the media data, e.g., broadcast or multicast.
The MSDC 1208, meanwhile, passes a list of services to the playback application 1204 (1240). The playback application 1204 may correspond to application 101 (
RTSP redirector/proxy 1218 may determine whether the service for retrieving media data is available, e.g., whether a service that defines broadcast or multicast as a transport is available. RTSP redirector/proxy 1218 may determine whether the service is available based at least in part on whether RTSP redirector/proxy 1218, or MSDC 1208, is within a service coverage area. The techniques of
Meanwhile, during the active broadcast session of the streaming service, a network operator sends RTP content over LTE RAN 1206 (1254). MSDC collects RTP content for the service from broadcast connection endpoint (labeled eMBMS rX 1220 in
In this manner, the techniques of
In this manner, the techniques of this disclosure may use an SDP extension mechanism (attribute) to provide TSB indication for eMBMS broadcast delivery of RTP content. This disclosure also defines an example architecture to support seamless transition between broadcast and unicast transport, and to provide RTP media content delivery mechanisms within a UE. Moreover, this disclosure describes techniques for grouping of multiple RTP-based media streams for unicast and broadcast delivery of content within an SDP message.
RTSP redirector/proxy 1218 represents an example of a proxy unit that may be configured to obtain mapping information that maps an identifier for media data to a resource location based on a service for retrieving the media data, wherein the service defines at least one of a plurality of types of transports (e.g., broadcast, multicast, or unicast) for transporting the media data, receive a request for the media data from an application service client, determine whether the service is available, and, when the service is available, cause the application service client to receive the media data from a unit that receives the media data using the service from the resource location, based on the mapping information.
In accordance with certain techniques of this disclosure, an application service client, such as a DASH client (e.g., the DASH client of
Each instance of broadcast maps to a unique Representation delivered over the MBMS bearer. Its baseURL will be compared to a portion of the Segment URL used by the DASH client to request Segments—specifically, the initial portion of the Segment URL starting from the URI scheme and extending to and inclusive of the identifier of the Representation to which the Segment belongs, to determine whether that Representation is being requested.
For example, assume the Segment URL issued by the DASH client to be “http://example.com/per-3/rep-512/seg-99.3gp”, corresponding to a request for Segment 99 of Representation 512 (Representation@id=‘512’) in Period 3 (Period@id=‘3’). The portion of this URL of concern for the purpose of matching with the basesURL in the USD is “http://example.com/per-3/rep-512. In the event that this Representation is also available over broadcast, an instance of mediaPresentationDescription2.broadcast will be present in the USD with baseURL given by “http://example.com/per-3/rep-512”, identical to the portion interest in the request URL. For instance, a match in the portion of interest for the request URL to the baseURL attribute of the USD's broadcast element denotes broadcast transport of the Representation to which the requested Segment belongs.
Similarly, each of the zero or more unicast Representations, in this example, is identified in the USD by a unique baseURL attribute of the mediaPresentationDescription2.unicast element. A matching pattern in the same portion of the request URL as discussed above to the unicast baseURL implies that this Representation is available over unicast delivery. The same Representation may be available over both, one-only, or neither of the transport modes.
The entity to which the DASH client submits the Segment request may be a proxy unit (or to an MBMS or eMBMS client). For purposes of example, the proxy unit is described below as performing these techniques, but it should be understood that the MBMS or eMBMS of, e.g.,
The proxy unit may retrieve a policy that indicates preferences among types of transports. For example, the policy may indicate that if the request is for a Representation delivered over unicast, as long as the device is located in a broadcast coverage area, only a broadcast-delivered Representation should be accessible to the DASH client. Such a broadcast Representation may be the same Representation as original requested, if the baseURL appears in the USD's identicalContent element, and can be substituted for the requested Representation, albeit delivered over a different transport mode.
Alternatively, the broadcast Representation may be an alternative Representation known to be delivered over broadcast, and is considered a switchable Representation, since the baseURL for the alternative Representation appears in the list of entries of the USD element switchableContent. In this case, the proxy unit may send a message back to the DASH client to cause the DASH client to request the same Segment belonging to an alternative Representation which is delivered over broadcast. For instance, the proxy unit may send a 300-type HTTP response message to the DASH client, which corresponds to a redirect message, and the redirect message may specify a resource URL corresponding to the alternative representation from the list contained in switchableContent.
As another example, if the DASH client requested a Segment known to the proxy unit to be delivered over broadcast but broadcast reception is not available (e.g., due to a client device being out of a coverage area for broadcast transport), the proxy unit may send a 300-type HTTP response message that includes a URL of a Segment corresponding to a unicast transport of the same, or a different Representation if that same Representation is not available over unicast transport, but appears as an entry in switchableContent.
As another example, if the DASH client requested a Segment known to the proxy unit to be delivered over broadcast and broadcast reception is available, the proxy unit will proxy the request to the local content cache, and deliver the retrieved Segment back to the DASH client.
Alternatively, the proxy unit may respond with a 400-type error message, which may cause the DASH client to re-submit a request using a different Segment URL. The proxy unit may communicate a different transport protocol in other ways as well, e.g., via an application programming interface (API), inter-process communication (IPC), sending a modified MPD that omits the selected base URL, or the like.
The redirect or error message from the proxy unit may cause the DASH client to select a different transport mode. In some examples, more than one Representation may be available on the redirected transport mode, in which case the DASH client may select from among one of those available Representations. As an example, if the DASH client had attempted to select a broadcast Representation (as given by an instance of the broadcast element of
In some instances, an application service client, such as the DASH client, may be tasked with managing unicast-broadcast transitions. This disclosure describes, with respect to
In one example, user equipment (UE) may be located within an MBMS coverage area, and the DASH client may request a particular Representation that the USD indicates is unavailable via broadcast delivery. Service provider policy may dictate that whenever the UE is in MBMS coverage, only broadcast-delivered Representations of the same program can be accessed. In this situation, the DASH client may be limited to selecting (or redirected or instructed to select) from among the alternative Representation(s) available over broadcast delivery.
In another example, UE may be located within MBMS coverage, and the DASH client may request a Representation that is known to be available over broadcast delivery. In this situation, the desired Representation can be directly accessed by the UE.
In another example, UE may be outside MBMS reception area, and the DASH client may request a Representation that is known to be only available over broadcast delivery. In this situation, the DASH client may be limited to selecting (or redirected or instructed to select) from among the alternative Representation(s) available over unicast delivery, in the form of switchable content(s).
In another example, UE may be outside MBMS reception area, and the DASH client may request a Representation that is known to be also available over broadcast delivery. In this situation, the DASH client may be able to receive the same Representation over unicast, in the form of identical content.
Additional information that may be carried in the USD includes the indication of the service area(s) over which each Representation is delivered, and/or the identification of the SDP that describes the FLUTE session carrying that Representation.
This disclosure describes techniques by which a mediaPresentationDescription2 child element may be added under userServiceDescription to carry additional transport parameters. Previously, it was proposed that MPD-specific parameters, such as Period ID, Adaptation Set ID, and Representation ID be contained in mediaPresentationDescription2, which may identify those Representations available over unicast delivery. These same parameters, along with URI reference to a Session Description fragment, may identify each Representation that can be delivered over broadcast, as well as the mapping to the FLUTE session carrying the Segments of that Representation.
One issue that the techniques of this disclosure may overcome is with regards to enabling the use of HTTP/1.1 as the protocol interface between the DASH client and the MBMS client for Segment request/response, while maintaining clean layering separation in protocol processing. For instance, in previous techniques, the MBMS client has to process MPD-specific information to be able to correlate the DASH client request for Segments of a particular Representation to an actually available and permitted (e.g. in accordance to service provider policy) Representation Segments by transport mode (broadcast or unicast). In the event that a requested Representation cannot be provided, in order for the MBMS client to use HTTP Redirection (via 3xx response codes as defined in Fielding et al., “Hypertext Transfer Protocol—HTTP/1.1,” Network Working Group, RFC 2616, June 1999 available at http://www.ietf.org/rfc/rfc2616.txt) to inform the DASH client of one or more alternative Representations, the MBMS client will have to compose a complete Segment URI for each alternative resource. Such a layering violation on the part of the MBMS client can be eliminated through the use of the techniques of this disclosure.
The techniques of this disclosure eliminate the need for the MBMS client (or proxy unit) to be aware of or have to process (DASH-specific) MPD information. The MBMS client merely performs data/pattern matching, based on the presence of new metadata in the USD, with the Segment request URIs generated by the DASH client, in determining whether the requested Segment is available over broadcast, unicast, neither or both transport modes, or by some other fashion (e.g., cache). This is possible because the portion of the request URI that uniquely identifies the Representation to which the requested Segment belongs is also conveyed in the USD. In addition, the MBMS client can determine whether the Representation requested by the DASH client (which is agnostic to the transport mode) is available over broadcast, unicast, neither or both transport modes, or by some other fashion (e.g., cache), by the location of the matching data pattern in the USD. In conjunction with any other relevant rules and conditions, such as coverage condition (whether the UE is in unicast and/or broadcast service area), service provider policy, etc., and assuming that the substitution method for returning the alternative resource URI is allowed based on the USD attribute replacementAllowed=‘true’. The MBMS client can use HTTP/1.1 mechanisms to mediate Segment requests to the appropriate resource—e.g., a local content cache in the UE, or an external HTTP server, without a protocol layering violation.
As can be seen in the examples of
In this example, each of the one or more broadcast Representations is identified in the USD by a unique baseURL attribute of the broadcast element. Each instance of broadcast maps to a unique Representation delivered over the MBMS bearer. Its baseURL will be compared to a portion of the Segment URI used by the DASH client to request Segments—specifically, the initial portion of the Segment URI starting from the URI scheme and extending to and inclusive of the Representation-ID (value of Representation@id in the MPD). For example, assume the Segment URI issued by the DASH client is the URL “http://example.com/per-3/rep-512/seg-99.3gp”, corresponding to a request for Segment 99 of Representation 512 (Representation@id=‘512’) in Period 3 (Period@id=‘3’). The portion of this URL of concern for the purpose of matching with USD data is “http://example.com/per-3/rep-512. In the event that this Representation is also available over broadcast, an instance of mediaPresentationDescription2.broadcast will be present in the USD with baseURL given by “http://example.com/per-3/rep-512”, identical to the portion interest in the request URI.
In the event the MBMS client determines that only alternative Representation(s) to what is requested by the DASH client can be accessed, the replacementAllowed attribute of mediaPresentationDescription2 may indicate to the MBMS client whether and how to provide such notification to the DASH client via the HTTP Redirect (3xx status code) method, e.g., as defined in RFC 2616.
For instance, if replacementAllowed=‘true’, it may be assumed that the DASH content and MPD are authored in such a way that allows the MBMS client to provide, to the DASH client, alternative resource URI(s) via ‘303 See Other’ redirection, regardless of the transport mode of the alternative Representation. Specifically, each alternative URI may be formed by replacing the portion of interest in the Segment URI as described above with the baseURL in the USD corresponding to the alternative Representation, while retaining the Segment number in the original request.
On the other hand, if replacementAllowed=‘false’, then such replacement method for generating an alternative resource URI and providing that to the DASH client may not be allowed. The resulting technique for causing an alternative Representation to be requested and delivered may be implementation dependent (e.g., the MBMS client may return a 4xx error code accompanied with the alternative Representation signaled by the baseURL value, and rely on the DASH client to formulate an alternative request). An example call-flow showing interactions between the MBMS client and the DASH client, based on HTTP ‘303’ redirection, is described below with respect to
Similarly, each of the zero or more unicast Representations may be identified in the USD by a unique baseURL attribute of the mediaPresentationDescription2.unicast element. A matching pattern in the same portion of the request URI, as discussed above, to the unicast baseURL implies that this Representation is available over unicast delivery. Note that the same Representation may be available over both, one-only, or neither of the transport modes.
Otherwise, via the sessionDescription element, each broadcast Representation may be linked to a Session Description fragment or SDP document pertaining to the FLUTE session carrying that Representation. In addition, the serviceArea element, if present, may denote the MBMS service area(s) over which the broadcast Representation(s) are available.
Furthermore, in this example, it is assumed that the MPD includes the following contents:
-
- MPD@type=‘dynamic’
- Period@id=‘3’
- Period. SegmentTemplate@media=‘http://example.com/per-3/rep-$RepresentationID$/seg-$Number$0.3gp’
- Representation@id=‘512’ . . .
- Representation@id=‘256’ . . .
- Representation@id=‘128’ . . .
Given these example MPD parameter values, the DASH client attempting to request Segment no. 99 for Representation 512 in Period 3 may issue the following request URI: http://example.com/per-3/rep-512/seg-99.3gp. The call flow of
In other words, this disclosure describes certain techniques for the use of USD-based signaling of DASH transport in support of transport diversity. It may provide one or more improvements over a previous proposal described in Tdoc S4-130051, “Rationale for USD Indication of DASH Delivery Mode and Illustrative Implementation” from Qualcomm Inc. For instance, a layering violation may be entirely avoided in this method, because the MBMS client does not have to understand or process MPD information. Instead, the MBMS client may perform data pattern matching of known transport semantics against the URIs of Segments requested by the DASH client to determine whether the requested Segments are requested to be delivered via broadcast and/or unicast, and whether that request can be satisfied as is, or needs to be redirected to the same or an alternate Representation available using another transport mode.
Such determination may be based on factors such as whether UE is located inside or outside of MBMS coverage, service provider policy, if any, governing accessibility of Representations by transport mode, and possibly dependent on other conditions or rules. Example network architecture and call flow diagram for DASH content delivery via MBMS with unicast fallback were provided to illustrate end-to-end content delivery featuring the use of HTTP protocol interface between the DASH client and MBMS client. The adherence to protocol layering should provide architectural benefits in extensibility and simplicity of UE implementation in support of DASH-in-MBMS services.
The use of HTTP Redirect via the 3xx status code by the MBMS client to restrict the DASH client access to an alternative resource (Representation) is predicated on the mediaPresentationDescription2's replacementAllowed attribute having the value ‘true’. In such event, it is assumed that the DASH content and MPD are authored in such a way that allows the MBMS client to provide to the DASH client alternative resource URI(s) via ‘303 See Other’ redirection, regardless of the transport mode of the alternative Representation. Specifically, each alternative URI is formed by replacing the portion of interest in the Segment URL as described above with the baseURL in the USD corresponding to the alternative Representation, while retaining the Segment number in the original request. If the value of replacementAllowed=‘false’, then such replacement method for generating an alternative resource URI and providing that to the DASH client is not allowed.
The resulting techniques to cause an alternative Representation to be requested and delivered may be implementation dependent. For example, the MBMS client may return an 4xx error code accompanied with or without an alternative Representation as signaled by the baseURL value in the entity body of the HTTP response, and rely on the DASH client to form an alternative request. Here, the presence of the baseURL identifying an alternative Representation is available in the response may be used as a hint to the DASH client with the intelligence to directly request that Representation as follow-up. Alternatively, the baseURL “hint” may not be provided in the 4xx response, or the DASH client may lack the intelligence to utilize such hint in generating a new request for another Representation which may or may not correspond to the allowed Representation from the MBMS client perspective.
With respect to the example of
The MPD in this example may specify the following data, where BaseURLs correspond to base portions of URLs for accessing Segments:
BaseURL1=http://www.cnn.com/512/, RepresentationId=“512”;
Template=seg$Number$0.3 gp,
BaseURL2=http://www.cnn.com/256/, RepresentationId=“256”;
Template=seg$Number$0.3gp,
BaseURL3=http://www.cnn.com/128/, RepresentationId=“128”;
Template=seg$Number$0.3gp.
In the example of
With respect to the example of
The MPD in this example may specify the following data, where BaseURLs correspond to base portions of URLs for accessing Segments:
BaseURL1=http://www.cnn.com/512/, RepresentationId=“512”;
Template=seg$Number$0.3gp,
BaseURL2=http://www.cnn.com/256/, RepresentationId=“256”;
Template=seg$Number$0.3gp,
BaseURL3=http://www.cnn.com/128/, RepresentationId=“128”;
Template=seg$Number$0.3gp.
In the example of
With respect to the example of
The MPD in this example may specify the following data, where BaseURLs correspond to base portions of URLs for accessing Segments:
BaseURL1.1=http://www.cnn.com/512/,
BaseURL1.2=http://localhost/512/, RepresentationId=“512”;
Template=seg$Number$0.3gp,
BaseURL2=http://www.cnn.com/256/, RepresentationId=“256”;
Template=seg$Number$0.3gp,
BaseURL3=http://www.cnn.com/128/, RepresentationId=“128”;
Template=seg$Number$0.3gp.
In the example of
With respect to the example of
The MPD in this example may specify the following data, where BaseURLs correspond to base portions of URLs for accessing Segments:
BaseURL1=http://www.cnn.com/;
RepresentationId=“512”; Template=$RepresentationId$/seg$Number$0.3 gp, RepresentationId=“256”; Template=$Representagtion$/seg$Number$0.3gp, RepresentationId=“128”; Template=$Representagtion$/seg$Number$0.3gp.In the example of
With respect to the example of
The MPD in this example may specify the following data, where BaseURLs correspond to base portions of URLs for accessing Segments:
BaseURL1=http://localhost/512/, RepresentationId=“512”;
Template=seg$Number$0.3gp,
BaseURL2=http://www.cnn.com/256/, RepresentationId=“256”;
Template=seg$Number$0.3gp,
BaseURL3=http://www.cnn.com/128/, RepresentationId=”“128”;
Template=seg$Number$0.3gp.
In the example of
With respect to the example of
The MPD in this example may specify the following data, where BaseURLs correspond to base portions of URLs for accessing Segments:
BaseURL1=http://www.cnn.com/1024/, RepresentationId=“1024”;
Template=seg$Number$0.3gp,
BaseURL2=http://www.cnn.com/512/, RepresentationId=“512”;
Template=seg$Number$0.3gp,
BaseURL3=http://www.cnn.com/256/, RepresentationId=“256”;
Template=seg$Number$0.3gp,
BaseURL4=http://www.cnn.com/128/, RepresentationId=”“128”;
Template=seg$Number$0.3gp.
In the example of
Initially, MSDC 1010 may receive a session description protocol (SDP) message (2400). The SDP message may include an attribute that defines a time-shifted buffer (TSB) depth. Accordingly, MSDC 1010 may determine a value for the attribute (also referred to as a TSB depth attribute) in the SDP message (2402). The value of the TSB depth attribute may define the depth of the TSB, e.g., in terms of seconds of playback for media data to be stored in the TSB. For example, the value of the attribute may define a maximum playback time, in seconds, that can be stored in the TSB.
MSDC 1010 may therefore determine an amount of memory to be allocated to form the TSB from the signaled depth (2404). For example, assuming that the depth of the TSB is signaled in terms of seconds of playback for media data, and assuming that a frame rate is signaled for video data, MSDC may determine a number of video frames to be stored in the TSB, as the product of the frame rate (expressed in frames per second) and the number of seconds of media data to be stored. MSDC 1010 may then determine an amount of memory for the TSB based on the product of an average bitrate per frame and the number of frames. MSDC 1010 may use similar concepts for determining memory size for audio data, timed text data, and the like.
MSDC 1010 may then allocate the determined amount of memory to form the TSB (2406). Thus, as MSDC 1010 receives media data, MSDC 1010 may store the received media data in the TSB (2408). A playback application may use this buffered media data for time-shifted application, such as for watching at a later time or for performing a trick mode, such as fast forward or rewind. Of course, the buffered data may also be used for substantially real time playback.
In this manner, the method of
Those of ordinary skill in the art will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
Those of ordinary skill will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media, or communication media including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. In this manner, computer-readable media generally may correspond to (1) tangible computer-readable storage media which is non-transitory or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.
By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are instead directed to non-transitory, tangible storage media. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor,” as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated hardware and/or software modules configured for encoding and decoding, or incorporated in a combined codec. Also, the techniques could be fully implemented in one or more circuits or logic elements.
The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chip set). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, various units may be combined in a codec hardware unit or provided by a collection of interoperative hardware units, including one or more processors as described above, in conjunction with suitable software and/or firmware.
The detailed description set forth above, in connection with the appended drawings, is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form in order to avoid obscuring such concepts.
Various examples have been described. These and other examples are within the scope of the following claims.
Claims
1. A method of processing media data, the method comprising:
- receiving a session description protocol (SDP) message including an attribute that defines a time-shifted buffer (TSB) depth;
- determining an amount of memory for the TSB based on a value of the attribute;
- allocating the determined amount of memory to the TSB; and
- storing at least a portion of media data associated with the SDP message in the TSB.
2. The method of claim 1, wherein receiving comprises receiving the SDP message in accordance with one of real-time transport protocol (RTP) and real-time streaming protocol (RTSP).
3. The method of claim 1, wherein the attribute comprises an “a=tsb-length:” attribute.
4. The method of claim 1, wherein the value of the attribute defines a length of the TSB.
5. The method of claim 4, wherein the value for the attribute defines the length in terms of seconds of playback for media data stored in the TSB.
6. The method of claim 1, wherein determining the amount of memory comprises determining the amount of memory based at least in part on a frame rate for media data associated with the SDP.
7. The method of claim 1, further comprising extracting at least a portion of the stored media data from the TSB to perform trick mode playback.
8. The method of claim 7, wherein the trick mode comprises one of fast forward and rewind.
9. The method of claim 1, wherein the SDP message comprises an SDP message for one of a broadcast SDP session and a unicast SDP session.
10. The method of claim 1, wherein the SDP message comprises a unicast session description and a broadcast session description.
11. The method of claim 10, wherein the SDP message includes attributes associated with identifiers of various sets of media data, wherein the attributes indicate whether the associated media data correspond to a broadcast media stream or a unicast media stream.
12. The method of claim 11, wherein the attributes comprise respective “a=group:” attributes.
13. The method of claim 10, wherein the SDP message includes attributes that distinctly identify each media stream.
14. The method of claim 13, wherein the attributes that distinctly identify each media stream comprise “a=mid:” identification tags, each having unique identifier values.
15. The method of claim 1, wherein allocating comprises allocating the determined amount of memory of a multicast services device client (MSDC) to the TSB.
16. A device for processing media data, the device comprising one or more processors configured to receive a session description protocol (SDP) message including an attribute that defines a time-shifted buffer (TSB) depth, determine an amount of memory for the TSB based on a value of the attribute, allocate the determined amount of memory to the TSB, and store at least a portion of media data associated with the SDP message in the TSB.
17. A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:
- receive a session description protocol (SDP) message including an attribute that defines a time-shifted buffer (TSB) depth;
- determine an amount of memory for the TSB based on a value of the attribute;
- allocate the determined amount of memory to the TSB; and
- store at least a portion of media data associated with the SDP message in the TSB.
Type: Application
Filed: Jan 13, 2014
Publication Date: Jul 17, 2014
Applicant: QUALCOMM Incorporated (San Diego, CA)
Inventors: Chaitali Gupta (San Diego, CA), Nagaraju Naik (San Diego, CA), Carlos Marcelo Dias Pazos (Carlsbad, CA), Ralph Akram Gholmieh (San Diego, CA)
Application Number: 14/153,869
International Classification: H04N 5/91 (20060101); H04L 29/06 (20060101);
