SYSTEM AND METHOD FOR SHARING MOBILE VIDEO AND AUDIO CONTENT
Systems and methods for broadcasting media content include recording the media content by a mobile device and monitoring the recording to determine whether a threshold has been reached. The recorded content is segmented into chunks that can be encoded, compressed, and uploaded to a server for broadcasting to a specified list of recipients. The recorded chunks can be deleted from the mobile device once a confirmation is received from the server that the chunk was received.
This application is a continuation of U.S. application Ser. No. 15/257,407, entitled “SYSTEM AND METHOD FOR SHARING MOBILE VIDEO AND AUDIO CONTENT,” filed on Sep. 6, 2016, which claims the benefit under 35 U.S.C. §119(e) to U.S. Provisional Application No. 62/214,300, entitled “SYSTEM AND METHOD FOR MOBILE VIDEO AND AUDIO CONTENT BROADCAST IN NETWORKS WITH NON-CONTIGUOUS CONNECTIONS,” filed on Sep. 4, 2015, and to U.S. Provisional Application No. 62/259,359, entitled “SYSTEM AND METHOD FOR COMPRESSING AND SHARING MOBILE VIDEO AND AUDIO CONTENT,” filed on Nov. 24, 2015, the contents of each of which are incorporated by reference herein in their entirety.
FIELD OF THE INVENTIONThe present invention relates to systems and methods for broadcasting compressed video and audio content from a mobile device.
BACKGROUND OF THE INVENTIONBroadcasting video and/or audio content originating from a mobile device has many challenges. For example, cellular and Wi-Fi networks are not always and continuously reliable. Most applications and devices that record and/or stream media files use standard protocols, which result in capturing continuous streams of data. When a mobile device attempts to broadcast or transmit video and/or audio files, which are large and require long transmit times, network connection problems can result in failed streams, unacceptable latency, and interrupted user experience.
Because the media files captured by mobile device are large, the file size and bandwidth of the streamed media file are subject to resolution quality and recorded length. For example, a two minute video at a 720p resolution can absorb about 250 MB of space and a thirty minute video at a 480p resolution can absorb about 3 GB of space.
SUMMARY OF THE INVENTIONA method for broadcasting media content from a mobile device can include the steps of recording the media content, monitoring, by the mobile device, the recording of the media content to determine whether a threshold has been reached, and segmenting the recorded content into at least one chunk, when the threshold has been reached. The method can also include determining to encode the at least one chunk, and encoding, by the mobile device, the at least one chunk, when the mobile device determines to encode the at least one chunk. The method can also include compressing the at least one chunk and placing the at least one compressed chunk into an upload queue. The method can also include transmitting, by the mobile, to a server the at least one chunk from the upload queue, receiving, by the mobile device, from the server a confirmation that the at least one transmitted chunk has been successfully received by the server, and deleting the at least one transmitted chunk from the mobile device, after receiving the confirmation from the server.
According to other embodiments, a method for delivering media content recorded on a mobile device can include the steps of receiving, by a server, from a first mobile device at least one encoded chunk of a media content recording, receiving, by the server, from the first mobile device metadata associated with the at least one encoded chunk of a media content recording, and transmitting to the first mobile device a confirmation that the at least one encoded chunk has been successfully received by the server. The method can also include storing the metadata to a database in communication to the server, notifying, by the server, at least one recipient of the media content that new media content is available, and storing, by the server, the at least one chunk to at least one of a content delivery network (CDN) or a streaming service.
According to other embodiments, a method for delivering media content recorded on a mobile device can include the steps of receiving, by a server, from a first mobile device at least one encoded chunk of a media content recording, receiving, by the server, from the first mobile device metadata associated with the at least one encoded chunk of a media content recording, and transmitting to the first mobile device a confirmation that the at least one encoded chunk has been successfully received by the server. The method can also include storing the metadata to a database in communication to the server, notifying, by the server, at least one recipient of the media content that new media content is available, reconstructing the media content from the at least one chunks, and storing the reconstructed media content to at least one of a CDN or a streaming service.
The present invention is directed to systems and methods for broadcasting, recording, and/or sharing a mobile video and/or audio content in environments without a continuous network connection. A mobile device user can capture, stream and share uninterrupted video or audio content to a selected audience using segmented files instead of standard streaming protocols. According to aspects of the disclosure, the audience can choose to either stream the broadcasted video or audio content or view the content on demand. The disclosed systems and methods can support adaptive bitrate switching protocols for desktop, laptop and mobile devices.
Most providers of streaming video/audio content support live or real-time broadcasting of video/audio with standard streaming protocols, for example, Periscope, Meerkat, UStream, LiveStream. Real-time video and audio broadcasting can create various issues for the broadcasters and the viewers of the broadcasted content. For example, a broadcaster is required to have a strong network connection throughout the entire broadcast to avoid failed steams, constant buffering, and large latency, and to provide uninterrupted video for the viewers. Frequently, the video or audio quality is severely compromised, at least in parts of the broadcast, because of its dependency on a continuous network connection with standard streaming protocols. As a result, the resolution of a broadcasted video is reduced to compensate for poor bandwidth. These problems are even more noticeable when the broadcaster is a mobile device user, because, in many situations, the broadcast originates at a venue, e.g., concert, that has poor network connection, there is interference from the venue structure, and cell congestion because of other mobile device users.
The disclosed systems and methods can remove the dependency to have a continuous network connection and can eliminate buffering in its entirety when broadcasting a high-quality video to viewers. According to aspects of the disclosure, the disclosed systems and methods use segmented files to capture blocks of the video or audio content and then send the segmented files upstream to a server for an uninterrupted delivery to the audience. The disclosed methods do not depend on standard streaming protocols and systems, and, therefore can guarantee delivery of higher quality content since there is no longer a dependency on a continuously reliable network. Accordingly, the disclosed systems and methods can guarantee that the content is delivered without data loss.
According to aspects of the disclosure, before each chunk is transmitted to the server the chunk is compressed. For example, the chunks can be compressed into a video container that is represented, for example, by a.mov or mp4 format. The compression process can use a codec, such as the native H.264 codec. In addition, each chunk can be analyzed before being processed over a fixed or variable bitrate reduction. The disclosed method can also remove unnecessary data and artifacts. The results can be significant; for example, a 250 MB video file at a 720p resolution can be reduced to 80 MB (⅓ the size) and a 3 GB video file at 480p resolution can be reduced to 500 MB or ⅙ of the size while maintaining the same resolution output.
According to aspects of the disclosure, each chunk can have a number in sequential order with metadata. For example, when the first recording starts, a sequential number can be prepended to the name of the chunk, starting at digit “00000000000.” This can then followed by incrementing the digits to the next number, e.g., “000000001” when a new chuck is generated. The mobile device can transmit the lower numbered chunks to the server before the higher numbered ones. If the server receives the chunks out of order, it can process the chunks, store them, and can wait for the lower numbered chunks to be processed so that the received chunks can be delivered in order.
According to aspects of the disclosure, the metadata can indicate which number in the sequence corresponds to the last chunk that the server should expect. Accordingly, the server can know when the final element has been received. For example, when the user ends their broadcast, a small REST request can be sent to the server to indicate that this session stops when a certain sequence number is received. In addition, metadata can be added to the data sent from the upload queue that can indicate to the server that the received chunk is the last chuck. The disclosed method can also provide checks that can inform the server that the broadcast is complete, even if the server has not received the final chunk of the recorded broadcast. This can be useful, for example, when the final chunk arrives at a later date, e.g., the next time the user opens the application on his cell phone or the mobile device is connected to a network. When the server receives the final chunk(s), it can append the final chunks to the ones already received.
According to aspects of the disclosure, the mobile device user may select to broadcast video and/or audio content that has already been recorded and is stored in the mobile device, for example in the mobile device's memory, or on media connected to the mobile device, for example, on an external hard drive 122. The method can determine whether the existing media will be segmented 124 before being uploaded into the upload queue, or whether the existing media will be uploaded in full 126.
According to aspects of the disclosure, the server can reconstruct the recording from its chunks before it is delivered or streamed to a target mobile device, for example using command line tools, such as FFmpeg that can perform various conversions. Alternatively, the content can be delivered or streamed to the target mobile device as chunks.
According to aspects of the disclosure, the chunks can be transmitted to the server before the user finishes recording of the content. Therefore, the target user can start receiving chunks from the CDN storage server or can stream the content from the streaming service even before the final chunk arrives at the server. As the chunks come in to the server, the method can detect the rate at which they come in, and accordingly deliver them to the target mobile device to achieve reduced latency. Therefore, a broadcast can continue for hours and the end user can watch as it is happening. The end user does not have to wait for the broadcast to finish to be viewed.
A mobile user has an average of 32-64 GB of storage capacity on their respective mobile devices. This capacity is often not enough for the content, e.g., videos, audio, photographs, generated by an average user. For example, a single 2-minute video can absorb 250 MB of space alone leading most users to quickly reach maximum storage capacity on their devices. Most users delete videos from their device to create available space, having to sacrifice personal moments to capture new ones. As a result, users can use cloud based storage services for purchase, such as Apple iCloud or Dropbox, to increase their storage capacity. However, it can be increasingly challenging to manage a cloud-based library or sharing video files with others users. For example, iCloud makes as a backup a copy of designated data on all devices registered to a particular user, and doesn't necessarily remove it from the device.
The disclosed systems and methods can automatically store the generated content in the cloud as an archive or library for all videos broadcasted, recorded or shared. As each chunk is successfully stored, that file can be removed from the user's device, therefore taking up as little space as possible on a user's device while being recorded. For example, a one-hour video broadcast can only use a maximum of 5 MB of memory. As the broadcast is recorded, it can be segmented into chunks that are uploaded to a server or a cloud service. When the chunks are successfully uploaded, they can be removed entirely upon completion. This process essentially means an endless amount of recording at a time regardless of memory space. Content is stored in the cloud and users need not manually delete content from their device as it is automatically removed and archived. This means that a phone loss or damage or software issues will not result in loss of content.
A typical user behavior when sharing a recorded video from a mobile device is to point and shoot and then attach the file to an SMS or Email Message and then identify the user(s) to share the video. A majority of users can run into file size issues when sending the file through traditional messaging systems, which will force the content creator to either trim the video or forego sending it entirely. The disclosed systems and methods solve the problem of sending large files, because the data is sent in chunks during the recording and/or after the recording. As a result, there are no limits to the length of the content to send.
Another problem with the above user behavior, is to identify all the recipients of the video in a message window before the moment is lost or forgotten. The disclosed systems and methods enable the broadcaster to pre-select viewers or a group of viewers, such as family, co-workers, etc., to access the video during its broadcast or access the video-on-demand through a shared network of archived videos. All videos broadcasted through the system can automatically be recorded and archived within a broadcaster's profile and accessible by anyone designed to share the video. Each recipient can view the content from the single source as opposed to receiving the media content in its entirety.
According to alternative embodiments,
When the user stops recording or after a predetermined delay since one or more chunks have been created, the method prepares the generated chunks for transmission 112. Specifically, the method determines whether to encode the data 413. If the method determines that the chunks will be encoded before transmission 415, the generated chunks enter an encoding queue, where they are encoded 417. According to embodiments, the encoding queue can use, for example, an Fast Forward Moving Picture Experts Group (FFMPEG) encoder, and/or can wrap the audio and video content in an MPEG-TS container with H.264 codec for video and Advanced Audio Coding (AAC) codec for audio. The final output of an encoding process can be a MPEG-TS file (.ts) suitable for HTTP Live Streaming (HLS) playback. The encoding queue can track the duration of each file and can build the appropriate m3u8 file for HLS playback. According to embodiments, the encoding queue can receive a variable number or a constant number of chunks for encoding at one time.
According to aspects of the disclosure, before each chunk is transmitted to the server the chunk can be compressed. For example, the chunks can be compressed into a video container that is represented, for example, by a .mov or mp4 format. The compression process can use a codec, such as the native H.264 codec. In addition, each chunk can be analyzed before being processed over a fixed or variable bitrate reduction. The disclosed method can also remove unnecessary data and artifacts. The results can be significant; for example, a 250 MB video file at a 720p resolution can be reduced to 80 MB (⅓ the size) and a 3 GB video file at 480p resolution can be reduced to 500 MB or ⅙ of the size while maintaining the same resolution output.
If the method determines that the chunks will not be encoded 419 or after the chunks have been encoded 421, the chunks can be placed into an upload queue 114. The chunks can then be transmitted to the server 116. According to embodiments, the chunks are transmitted to one of many servers (422, 424, 426), for example, the chunks are sent to the closest server relative to the user.
When a chunk is successfully received by the server, the server sends back a confirmation 118 to the mobile device. For example, the confirmation can be a successful REST Header response as well as a JSON object: {“status”: 1}. Once the confirmation is received by the mobile device, the chunk can be deleted from the mobile device 120. According to embodiments, the chunk can be automatically deleted from the mobile device without any user interaction. The location of each chunk in the upload queue can be know, and therefore, the chunk can be identified and can be removed after the receipt of the confirmation from the server.
Device 102 can also sent to the server metadata that are associated with the recorded content 428. For example, the metadata can include information about the recipients of the recorded content. Other metadata can include the title of the broadcast, e.g., a title entered by the user of the mobile device recording the content, the location of the mobile device, for example, in latitude and longitude, the orientation of the device, and date and time information. According to embodiments, the device can initiate a broadcast first metadata, e.g., with or without a list of contacts that would receive the content. During the broadcast, the user can select to add or remove recipients. When this happens, device 102 can send update metadata that correspond to the user selection. According to embodiments, the data and/or the metadata are transmitted via a REST request to the selected server.
According to aspects of the disclosure, each chunk can have a number in sequential order with metadata. For example, when the first recording starts, a sequential number can be prepended to the name of the chunk, starting at digit “00000000000.” This can then followed by incrementing the digits to the next number, e.g., “000000001” when a new chuck is generated. The mobile device can transmit the lower numbered chunks to the server before the higher numbered ones. If the server receives the chunks out of order, it can process the chunks, store them, and can wait for the lower numbered chunks to be processed so that the received chunks can be delivered in order.
According to aspects of the disclosure, the metadata can indicate which number in the sequence corresponds to the last chunk that the server should expect. Accordingly, the server can know when the final element has been received. For example, when the user ends their broadcast, a small REST request can be sent to the server to indicate that this session stops when a certain sequence number is received. In addition, metadata can be added to the data sent from the upload queue that can indicate to the server that the received chunk is the last chuck. The disclosed method can also provide checks that can inform the server that the broadcast is complete, even if the server has not received the final chunk of the recorded broadcast. This can be useful, for example, when the final chunk arrives at a later date, e.g., the next time the user opens the application on his cell phone or the mobile device is connected to a network. When the server receives the final chunk(s), it can append the final chunks to the ones already received.
According to aspects of the disclosure, the mobile device user may select to broadcast video and/or audio content that has already been recorded and is stored in the mobile device, for example in the mobile device's memory, or on media connected to the mobile device, for example, on an external hard drive 122. The method can determine whether the existing media will be segmented (124) before being uploaded into the upload queue, or whether the existing media will be uploaded in full 126.
According to aspects of the disclosure, the server can reconstruct the recording from its chunks before it is delivered or streamed to a target mobile device. Alternatively, the content can be delivered or streamed to the target mobile device(s) as chunks.
According to aspects of the disclosure, the chunks can be transmitted to the server before the user, e.g., content creator, finishes recording of the content. Therefore, the recipient(s) can start receiving chunks from the CDN storage server or can stream the content from the streaming service even before the final chunk arrives at the server. As the chunks come into the server, the method can detect the rate at which they come in, and accordingly deliver them to the target mobile device to achieve reduced latency. Therefore, a broadcast can continue for hours and the end user can watch as it is happening. According to embodiments, the recipient can start viewing the content before the broadcast to finishes.
Claims
1. A method for broadcasting media content comprising:
- recording, by a mobile device, the media content;
- segmenting, by the mobile device, the recorded content into at least one chunk, when a threshold has been reached;
- encoding, by the mobile device, the at least one chunk, upon a determination to encode the at least one chunk;
- compressing, by the mobile device, the at least one chunk;
- placing, by the mobile device, the at least one compressed chunk into an upload queue;
- transmitting, by the mobile device, to a server the at least one chunk from the upload queue;
- receiving, by the mobile device, from the server a confirmation that the at least one transmitted chunk has been successfully received by the server; and
- deleting the at least one transmitted chunk from the mobile device, after receiving the confirmation from the server.
2. The method of claim 1, further comprising:
- transmitting, by the mobile device, to the server first metadata associated with the at least one chunk;
- wherein the first metadata includes at least one of a list of recipients for the media content, a title for the media content, a location associated with the media content, time information, and date information.
3. The method of claim 2, further comprising:
- transmitting, by the mobile device, to the server second metadata associated with the at least one chunk;
- wherein the second metadata includes at least one change in the list of recipients.
4. The method of claim 1, wherein the threshold is at least one of a media content size and a duration.
5. The method of claim 1, wherein the threshold can dynamically change during the recording of the media content.
6. The method of claim 1, further comprising selecting the server from a plurality of servers, based on the proximity of the server to the mobile device.
7. The method of claim 1, wherein the confirmation from the server comprises a successful REST Header response.
8. The method of claim 1, wherein the at least one chunk is encoded with a Fast Forward Moving Picture Experts Group (MPEG) encoder.
9. A method for delivering media content comprising:
- receiving, by a server, from a first mobile device at least one encoded chunk of a media content recording;
- receiving, by the server, from the first mobile device metadata associated with the at least one encoded chunk of a media content recording;
- transmitting, by the server, to the first mobile device a confirmation that the at least one encoded chunk has been successfully received by the server;
- storing, by the server, the metadata to a database in communication with the server;
- notifying, by the server, at least one recipient of the media content that new media content is available; and
- storing, by the server, the at least one chunk to at least one of a content delivery network (CDN) or a streaming service.
10. The method of claim 9, wherein the at least one recipient is specified in the metadata.
11. The method of claim 9, wherein the server notifies the at least one recipient through at least one of an email, short message service (SMS) message, and a push notification.
12. The method of claim 9, wherein the confirmation comprises a successful REST Header response.
13. A method for delivering media content comprising:
- receiving, by a server, from a first mobile device at least one encoded chunk of a media content recording;
- receiving, by the server, from the first mobile device metadata associated with the at least one encoded chunk of a media content recording;
- transmitting, by the server, to the first mobile device a confirmation that the at least one encoded chunk has been successfully received by the server;
- storing, by the server, the metadata to a database in communication to the server;
- notifying, by the server, at least one recipient of the media content that new media content is available;
- reconstructing the media content from the at least one chunks; and
- storing, by the server, the reconstructed media content to at least one of a content delivery network (CDN) or a streaming service.
14. The method of claim 13, wherein the at least one recipient is specified in the metadata.
15. The method of claim 13, wherein the server notifies the at least one recipient through at least one of an email, short message service (SMS) message, and a push notification.
16. The method of claim 13, wherein the confirmation comprises a successful REST Header response.
Type: Application
Filed: Sep 8, 2017
Publication Date: Dec 28, 2017
Inventors: Vincent TUSCANO (Santa Monica, CA), Raymond LEE (Los Angeles, CA), Vadim LAVRUSIK (San Mateo, CA)
Application Number: 15/698,878