Time shifting by simultaneously recording and playing a data stream
A video stream may include portions that are stored while other portions are being displayed. In one embodiment, a portion of the video stream may be stored in digital storage media at one instance while in the next instance another portion of the stream is being read out of the storage media.
This is a continuation-in-part of U.S. patent application Ser. No. 08/996,535, filed Dec. 23, 1997.
BACKGROUNDThis present invention relates to the recording and playing back of a video stream. A video stream includes any combination of audio and/or video data streams.
Video streams have typically been recorded on analog media such as a video cassette. A video cassette recorder (VCR) is used to record the video stream on the video cassette. The video stream may come via a broadcast signal, via cable, via satellite signal, or from another video playback device. Once the video stream has been recorded, the VCR is used to rewind the recording medium and play what was recorded. However, due to the nature of the analog medium, once the VCR has started recording, it is not possible to play back the portion of the video stream that has already been recorded until the recording session is terminated.
For example, imagine that a person sets up a VCR for recording a one hour show because he knows that he will miss the first 15 minutes of the show. When he arrives home 15 minutes into the show, he will have to wait for the entire show to be recorded before he can start watching the program from the beginning. A way of being able to view the show from the beginning without having to wait for the recording session to terminate is desirable.
SUMMARYA method of enabling a video stream to be stored and displayed at the same time including allowing portions of the video stream to be alternately written to and read from a storage device. The next portion to be written to the storage device is stored in a temporary buffer while another portion is being read from said storage device.
DESCRIPTION OF THE DRAWINGS
The digital video stream from the video input 102 is optionally compressed at compression unit 104. In one embodiment, the video is already compressed, such as would be the case with an Moving Picture Experts Group 2 (MPEG 2) specification (available from the International Standards Organization Standard IS01172) compliant video signal, and no further compression is needed. The video stream is then stored in the storage unit 108. A buffer unit 106 may be used as temporary storage for providing larger sequential blocks of video data to the storage unit 108. In one embodiment, the storage unit 108 is a random access memory that allows relatively quick access to any portion of the stored video stream. A hard disk is an example of a random access memory.
The video stream is played back by reading the video stream from the storage unit 108. If the video stream was compressed in compression unit 104, then a decompression unit 110 decompresses the retrieved video stream. The video stream is provided from a video output port 120, to a monitor or other display device to provide sound and/or video to a user.
A removable storage unit 122 may also be included in video record and playback system 100, Examples of removable storage units are a writeable compact disk read only memory (CD-ROM), writeable digital video disk (DVD), a flash memory, or another hard disk. The removable storage unit 122 allows a user to transfer a recording of a video stream stored in storage unit 108 to the removable storage unit 122 and then to transfer the unit 122 to another system at a different location.
In one embodiment, a processor 130 controls the operations of the video record and playback system 100. The compression unit 104 and decompression unit 110 may be implemented in hardware, or the compression and decompression functions of units 104 and 110 may be performed by the processor 130. Processor 130 receives instructions from firmware/memory 140, using technology that is well known.
In this way, the record and playback functions are decoupled from one another. The user may now begin watching a recorded TV show from the beginning, i.e., prior to the show being completely recorded.
In one embodiment, the random access storage unit is a hard disk. The retrieval of the time-shifted video signal from the hard disk is performed at a first physical location (or sector) of the hard disk, and the storing to the hard disk of the incoming video stream is performed at a different physical location (or sector) on the hard disk. Because it takes more time to jump back and forth between different sectors of the hard disk than to read and write to sequential locations in the same sector, data may be buffered to minimize the number of accesses to and from the hard disk using buffer 106. This increases the amount of data transferred per access.
Additionally, because of time constraints for reading and writing to the hard disk, data may be compressed and decompressed to speed transfers to and from the hard disk.
In block 302, the video stream is stored in a random access storage unit. The video stream is optionally compressed and/or buffered prior to storage. In block 304, the video stream is retrieved from the random access storage unit. Buffering and/or decompression may be performed before providing the retrieved video stream to the video output port 120.
The next portion of the video stream is stored (block 306) as described in block 302. At block 308, the next portion of the video stream is retrieved as described in block 304. This process is repeated until either the recording or playback cycle is terminated.
Due to the nature of the random access storage unit being capable of easily recording over itself, the random access storage unit may act as a temporary buffer for recording the latest portion, or X number of minutes, of an incoming video stream, where X is set up based upon the size of the storage unit. In one embodiment, X could be set up to be the entire storage unit. As newer portions of the video stream are received, they overwrite the older portions of the video stream saved in the random access storage unit. In this manner, the temporary buffering of the video stream acts as a circular buffer. In one embodiment, the processor 130 maintains pointers to the beginning and ending points of the temporary buffer. The processor 130 reassigns the pointers as newer portions of the video stream are received and/or older portions of the video stream are overwritten.
A user may initiate a playback cycle following block 506. For example, this may occur when the user wishes to re-view a video clip that he just saw. In one embodiment, the user stops recording to the temporary buffer and plays back the last portion of the temporary buffer.
However, it may be more desirable to the user to be able to continue recording as shown at block 508. A record and playback cycle (as described with respect to
At block 510, after the record and playback cycle is completed, all or part of the temporary buffer may be saved. Since the temporary buffer stores the latest X minutes of the video stream prior to the record and playback cycle, all or part of the temporary buffer may be allocated to the portion of the video stream saved during the record and playback cycle. Other portions of the video stream may then be deleted from the storage unit, or they may be marked as overwriteable and used as a new temporary buffer.
For example, playback of the video stream may have an overall rate faster than the rate of the incoming video stream if the playback is fast forwarded, or if segments of the playback are skipped altogether. When the time delay of the time-shifted video stream being played back falls below a certain threshold, the video and playback system 100 will cease providing the time-shifted video stream from the storage unit. Instead, the incoming video stream will be provided to the video output port 120 directly. In one embodiment, a bypass 142, as shown in
When this happens, the user has caught up to the “live” broadcast, i.e., the incoming video stream. The user may terminate the recording cycle, if he wishes. Alternatively, the user can put the video record and playback system 100 back into the temporary buffering mode in which only the latest portion of the video stream is stored.
At block 704 the incoming video stream is recorded but is not displayed to the monitor. Instead the playback is paused at the point at which the user indicated that the incoming video stream be suspended. When the user is ready to view the video stream again, he can signal the video record and playback system 100 to un-suspend the video stream so that it plays back from the point at which the video stream was suspended, as shown in block 706.
The user may then view the video stream time shifted by the amount of time that he suspended the incoming video stream, or he may fast forward (or rewind) through the time-shifted video stream. When playback of the time-shifted video stream catches up to the point at which the incoming video stream is being recorded, the record and playback system 100 may display the incoming video stream directly from incoming video stream without retrieving the video stream from the storage unit, as described with respect to
Employing a random access storage unit for storage of the video stream facilitates jumping to various points within the video stream. One way of jumping is by retrieving different time slices of the video stream. For example, an image frame from the video stream can be retrieved from the storage unit at 1 minute intervals from a current position of the video stream. In one embodiment, an image frame at the current playback position+1 minute, current playback position+2 minutes, and so forth are retrieved and displayed on the TV screen.
After the image frames are displayed, the user is able to select one of the frames as a new starting point to which to begin an operation, such as a playback or record operation. By using such a story boarding method, it is easy for a user to quickly jump to a desired location within the video stream.
In one embodiment, the record and playback system 100 is able to detect a black screen or fade out, such as those which accompany the beginning or end of a commercial. This is useful in editing a video stream.
Having the video stream stored on a random access storage unit such as a hard disk allows for easy editing of the video stream. Individual frames of the video stream may be deleted or replaced. For example, a commercial may be replaced by a fade-to-black sequence.
Once the video stream on the storage unit has been edited, it can be stored to a more permanent medium such as a writeable CD-ROM, flash memory, or another hard disk via the removable storage unit 122.
Referring now to the embodiment shown in
A computer system 1000 in accordance with one embodiment of the present invention, shown in
The bus 1012 is also coupled to a bridge 1016 which couples a hard disk drive 1018. The software 1020, 1022, 1024, 1026, 1028, and 1030 may be stored on the hard disk 1018. The bridge 1016 is also coupled to another bus 1032. The bus 1032 may be coupled to a serial input/output (SIO) device 1034. The device 1034 is in turn coupled to conventional components such as a mouse 1036, a keyboard 1038, and an infrared interface 1040. Also connected to the bus 1032 is a basic input/output system (BIOS) 1042.
An exemplary display 1100, shown in
Referring next to
If a particular television program is selected (even if the program is only selected for viewing) on the EPG, an identifier for that particular program is stored, as indicated at block 1202, and automatic recording of the program begins (block 1204). In this way the stored version of the program is available to implement the various features described herein even if the storage function was not particularly requested. Because the information is stored in an overwriting fashion, no harm is done by recording the information even if the information turns out not to be useful.
Referring to
Continuing in
Referring to block 1208, the pointer focus is identified to determine which quadrant will be zoomed. That quadrant is then scaled as indicated at block 1210. Because the television program is continually being stored, the scaling can be implemented without any loss of continuity of the television picture. After the user has completed looking at the zoomed depiction, the user can return to the program at the point where he or she departed to view the zoomed depiction. Thereafter the displayed depiction viewed by the user may be time delayed from the actually broadcast program. However the user is able to view the program in its entirety even if slightly time delayed.
Next a query is made at diamond 1212 to determine whether a pause function has been selected. If so, the playback is stopped as indicated at block 1214 but recording continues unabated as shown in block 1216.
The next check determines whether the play feature has been selected (block 1218). If so the recorded content is played thereby terminating the live display. However, while the recorded content is displayed, recording continues as indicated in block 1222 so that the recorded content may be eventually displayed without disrupting the continuity of the program.
A query finds out if the fast forward feature has been implemented as indicated in diamond 1224. If so, a fast playback is implemented as shown in block 1226. Once the playback equals the current broadcast or program content (diamond 1228), the flow returns to display current broadcast or program content. The fast forward may progress at 300 to 400% of normal playback speed.
Moving on to
A catchup feature is checked (diamond 1230) and, if this feature has been selected, an accelerated playback option is implemented (as indicated in block 1238). In the accelerated playback, the playback may be slightly increased in speed, for example from 105 to 115% of actual speed, so the user can easily follow the program but can make up for lost time to get back into the program as broadcast. Once the recorded program catches up to the real-time program as indicated in diamond 1240, the catchup feature is completed and the user returns to real-time display.
Control over the record and playback functions may be implemented through a normal mouse function, for example by clicking on icons as indicated in
As indicated in the depiction of an exemplary computer system 1000 (
A button 1306 is provided to implement the replay function, a button 1308 may implement the pause feature, and an enter button 1310 may be provided to allow entry of various information including a particular television program station. An on/off button 1314 may also be provided. Fast forward may be implemented by button 1316, an electronic program guide may be called by pressing button 1304. The catch up feature may be called by the button 1318, and a 10 second reply may be implemented by the button 1320. A conventional numeric keyboard 1322 may be provided to input additional information including selected television channels.
Techniques for storing the video stream onto the storage device 1410 and for reading the information out of the storage device are summarized in Table 1 below and illustrated in
(X = no action)
periods referred to herein as time steps. In a first time step, shown in
As the buffer 1404 fills up, the buffer 1408 moves to the top of the stack (as indicated by the dashed arrow) and incoming video is stored in buffer 1408. As indicated in block 1504 in
In time step 3 the buffer 1408 has filled up and the contents of buffers 1404 and 1408 are written to the storage device 1410 in a single write operation. During the same time step, buffer 1406 moves to the top of the stack and becomes the storage buffer for incoming video. This is illustrated in blocks 1508, 1510 and 1512 in
In time step 4, the buffer 1404 moves back to the top of the stack to store incoming video since its previous content has already been saved in the storage device 1410. This is indicated in block 1514 of
The content from the storage device 1412 is then read into buffers 1412 and 1414.
In time step 5, the buffer 1408 moves to the top of the stack to store incoming video, and buffer 1414 moves to the top of the output stack and transfers data to the video output 1418. The contents of the buffers 1404 and 1406 are then written to the storage device 1410.
The time steps 1 to 5 illustrate a complete cycle from input to output. The remaining sequence of steps repeat starting at step 5 for the input of data and time steps 6 through 9 repeat for the output of data.
Referring now solely to
In time step 7, the contents of the sixth buffer (which is buffer 1416) are sent to the output 1418. No other output operations occur.
In time step 8, the contents from the storage device 1410 are read into the fifth and sixth buffers (buffers 1412 and 1416) as indicated in block 1612. Also the contents of the fourth buffer (buffer 1414) are passed to the output 1418 as indicated in block 1614.
Finally, during time step 9 the contents of the fourth buffer (buffer 1412) are passed to the output 1418, completing the cycle.
Using these algorithms, the storage device is provided with enough time, through the operation of the buffers, to write the incoming video while supplying enough data simultaneously to the output display. Since the memory is used to make sure no content is dropped at the input end and the display is never starved at the output end, continuous, even display can be provided without losing information. In addition reading and writing larger chunks of data at one time minimizes the amount of storage device head movement, thereby allowing the storage device to keep up with the read and write requests.
The minimum size of the individual buffers 1404 to 1408 and 1412 to 1416 (“B”) is dependent on a number of factors including the input/output data rate “D”, for example in megabytes per second, the effective bandwidth of the storage device when reading or writing “S”, for example in megabytes per second, and the average seek time for the storage device per transaction “t”, in microseconds. The time that it takes to fill up one memory buffer (B divided by D) is advantageously greater than the time it takes to read or write two memory buffers (2B divided by S) plus the average seek time (t):
Solving for the minimum buffer size in the above equation yields the following equation which gives the minimum buffer size:
B≧((DS)/(S−2D))/(t/1000)
A viewer can pause a season premier of a television while the station is still broadcasting in order to get up and answer the telephone. The user can resume watching the show after interruption as if the program were on video tape. In addition, while watching a sports event the user can rewind back to a controversial pass, replay the play in slow motion, stop at the exact instance when the catch was made and zoom in on the receiver. Also, the user can apply an image sharpening filter to see if both of the receiver's feet touched the ground before the receiver went out of bounds.
Timeshifting by simultaneously recording and playing back a data stream enables the following type of benefits/features for an end-user. While watching the season premiere of a television show, the viewer can literally “pause” the program in order to get up and answer the phone or get a snack from the refrigerator. After the interruption, the viewer can resume watching again without having missed anything. If the viewer came home 15 minutes late for the show but had started recording the program from the beginning, the viewer can begin watching right away instead of waiting for the show to be over and then rewinding the cassette tape. Also the viewer can replay scenes during a “live” broadcast (e.g. season premiere of a show or sporting event) in case the viewer missed a line or an exciting play. In addition, while watching a sports event the user can rewind back to a controversial play, replay the play in slow motion, and stop at the exact instance when the catch was made. To get a better view, the viewer can zoom in on a portion of the screen and apply image sharpening filters to magnify one part of the screen.
Thus, a method of simultaneously recording and playing back a time-shifted video stream is disclosed. The specific arrangements and methods described herein are merely illustrative of the principles of this invention. For example, the same method may be used to store and retrieve other types of data streams besides video streams. Numerous modifications in form and detail may be made without departing from the scope of the described invention. Although this invention has been shown in relation to a particular embodiment, it should not be considered so limited. Rather, the described invention is limited only by the scope of the appended claims.
Claims
1-25. (canceled)
26. A method comprising:
- selectively displaying a video stream from storage or without storing said video stream, depending on a time delay.
27. The method of claim 26 including:
- when the time delay is greater than a threshold, displaying a video stream from the storage; and
- when the time delay is less than the threshold, displaying the video stream without storing said stream.
28. The method of claim 26 including allowing portions of the video stream to be alternately written to and read from the storage device and storing in a temporary buffer the next portion to be written to the storage device while another portion is being read from the storage device.
29. The method of claim 26 further including allowing one or more portions of the video stream to be read from the storage device to retrieve the video stream with one or more time delays that are user-specified.
30. The method of claim 26 wherein storing the video stream further comprises allowing the video stream to be compressed prior to writing the video stream to the storage device.
31. The method of claim 28 wherein displaying the video stream from the storage device when the time delay is greater than the predetermined threshold comprises decompressing the video stream after retrieving the video stream from the storage device.
32. An article comprising a medium storing instructions that, if executed, enable a computer to:
- selectively display a video stream from storage or without storing said video stream, depending on a time delay.
33. The article of claim 32 further storing instructions that, if executed, enable the computer to:
- when the time delay is greater than a threshold, display the video stream from the storage; and
- when the time delay is less than the threshold, display the video stream without storing said stream.
34. The article of claim 33 further storing instructions that, if executed, enable the computer to allow portions of the video stream to be alternately written to and read from the storage device and storing in a temporary buffer the next portion to be written to the storage device while another portion is being read from the storage device.
35. The article of claim 33 further storing instructions that, if executed, enable the computer to allow one or more portions of the video stream to be read from the storage device to retrieve the video stream with one or more time delays that are user-specified.
36. The article of claim 33 further storing instructions that, if executed, enable the computer to compress the video stream prior to writing the video stream to the storage device.
37. The article of claim 34 further storing instructions that, if executed, enable the computer to allow portions of the video stream to be displayed from the storage device when the time delay is greater than the predetermined threshold comprises decompressing the video stream after retrieving the video stream from the storage device.
38. A system comprising:
- a storage to store a video stream;
- a controller to selectively display a video stream from said storage or without storing said video stream, depending on a time delay.
39. The system of claim 38 said controller to display the video stream from the storage when the time delay is greater than a threshold, and when the time delay is less than a threshold, said controller to display the video stream without storing said stream
40. The system of claim 38 including a display.
41. The system of claim 38, said controller including a temporary buffer and wherein said storage stores instructions that enable the controller to allow portions of the video stream to be alternately written to and read from the storage and store in its temporary buffer the next portion to be written to the storage while another portion is being read from the storage.
42. The system of claim 40 including said controller to allow one or more portions of the video stream to be read from the storage to retrieve the video stream with one or more time delays that are user specified.
43. The system of claim 41 wherein said controller to allow the video stream to be compressed prior to writing the video stream to the storage.
44. The system of claim 41 wherein said storage decompresses the video stream after retrieving the video stream from the storage when the time delay is greater than the predetermined threshold.
Type: Application
Filed: Jul 5, 2005
Publication Date: Nov 3, 2005
Inventors: Dennis O'Connor (Chandler, AZ), Mark Chuang (Cupertino, CA)
Application Number: 11/174,906