SCENE-BASED VARIABLE COMPRESSION
Technology is disclosed for enabling scene-based variable compression (“the technology”). In some embodiments, the technology can receive an indication of a compression level for a first content type, wherein the compression level specifies a tolerance level for lossy compression; receive a request for content, the content having at least two portions, wherein a first portion has first content of the first content type and the second portion has second content of a second content type, a first compression method associated with the indicated compression level and a second compression method associated with a different compression level. The technology can transmit the content, wherein the first portion of the content was compressed using the first compression method and the second portion was compressed using the second compression method.
Latest EMPIRE TECHNOLOGY DEVELOPMENT LLC Patents:
This Application is a continuation application under 35 U.S.C. §120 of U.S. patent application Ser. No. 15/047,739, filed on Feb. 19, 2016, which is a continuation of U.S. patent application Ser. No. 13/389,432, filed on Feb. 7, 2012, now U.S. Pat. No. 9,338,257, which in turn is a U.S. National Stage filing under 35 U.S.C. §371 of International Application No. PCT/US11/28985, filed on Mar. 18, 2011. The disclosures of U.S. patent application Ser. No. 15/047,739, U.S. patent application Ser. No. 13/389,432, and International Application No. PCT/US11/28985 are hereby incorporated herein by reference in their entireties.
BACKGROUNDComputer networks enable vast amounts of data (e.g., content) to be transferred from one computing device to another computing device. As an example, a client computing device may access multimedia content stored at a server computing device via an intranet or the Internet. Because network bandwidth is limited, various compression methods are used to compress content that is to be transferred between computing devices. As an example, an international standards organization moving picture experts group (“MPEG”) has proposed compression methods for multimedia content.
Compression methods use a fewer number of bits to store data than the number of bits that represent the uncompressed data. Compressed data can thus require less storage space to store and reduced network bandwidth to transmit the compressed data as compared to the equivalent data prior to compression (“uncompressed data”).
Compression methods can be lossy or lossless. When a lossy compression method is used to compress data, the compressed data generally cannot be used during expansion to reproduce the originally compressed data with complete fidelity. In contrast, when a lossless compression method is used to compress data, the compressed data can be used to reproduce the originally compressed data with complete fidelity. Thus, which compression method is used to
SUMMARYTechnology is disclosed for enabling scene-based variable compression (“the technology”). The technology can receive an indication of a compression level for a first content type, wherein the compression level specifies a tolerance level for lossy compression; receive a request for content, the content having at least two portions, wherein a first portion has first content of the first content type and the second portion has second content of a second content type, a first compression method associated with the indicated compression level and a second compression method associated with a different compression level. The technology can then transmit the content to respond to the request, wherein the first portion of the content was compressed using the first compression method and the second portion was compressed using the second compression method.
The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.
In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the Figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly contemplated herein.
Technology is disclosed for enabling scene-based variable compression (“the technology”). In various embodiments, the technology enables delivery of content that is variably compressed based on scene-based preferences that are previously received from a requester of the content. A client (e.g., a client computing device or a user of the client computing device) can indicate preferences for various content types. As an example, the client can indicate that a wide-angle view of a sporting event should not be compressed, but a news program can be heavily compressed. A server can then store the preferences indicated by the client. When the client subsequently requests content (e.g., content that is to be downloaded or streamed), the server can then respond to the request by selecting content that is compressed according to the client's previously provided preferences. The content can be dynamically compressed (e.g., just before it is delivered to satisfy requests) or previously compressed and stored. When a server responds to client requests for content, the server can determine what content types correspond to the content either by detecting scenes dynamically (e.g., near the time the content is to be provided to the client) or by retrieving metadata associated with the portions of the requested content.
The content can include multiple scenes, each corresponding to a different content type. As an example, a sports show may include a portion of a game (e.g., a professional basketball game), followed by an analysis. The game portion may include multiple wide-angle parts and multiple close-up parts. The analysis portion may be similar to a news program. If the client that requests the content has specified preferences not to compress wide-angle shots, compress close-up shots somewhat, and compress news programs substantially, then the server may provide the sports show to the client with each part compressed accordingly.
A client can specify preferences for compression based at least on various scene-based factors, e.g., the gender or identity of the present speaker, the content type (e.g., violence, sex, nature, dance, news, etc.), and so forth. The preferences the client specifies may additionally indicate a preferred or desired compression level or a compression method for at least some of the content types. The technology may select various compression methods for the indicated compression levels. Thus, the technology is capable of providing content that responds to client indications of scene-based compression in a way that varies with content types. When the client specifies a particular compression level, the technology, in some embodiments, may vary compression parameters for a particular compression method (e.g., a specific lossy compression method) to achieve more or less compression, and each such variance can be referred to herein as a different compression method.
In various embodiments, initial scene recognition may initially be performed by a client computing device. As an example, the client computing device may initially receive content compressed using a first compression method (e.g., a lossy compression method). The client computing device may employ a scene recognition algorithm to recognize that the content contains a particular scene that a user of the client has previously indicated to be something the user wants to see with higher fidelity than would be possible when content is compressed using the first compression method. Upon recognizing that the content contains such a scene, the client computing device may indicate to the server computing device to use a second compression method (e.g., a lossless or “less lossy” compression method) that provides higher fidelity than the first compression method. The server computing device may then provide the content that is compressed using the second compression method. Later, when the client computing device determines that the received content includes a different scene that the user has not indicated should be received at higher fidelity than would be possible when content is compressed using the first compression method, the client computing device may request the server computing device to revert to using the first compression method (or a different compression method).
In various embodiments, the user may be able to indicate that the user wants to see some content types with lower fidelity than would occur when the content is compressed using the first compression method. In such embodiments, the client computing device may employ the scene recognition algorithm to recognize that the content contains a particular scene that the user has previously indicated to be something the user wants to see with lower fidelity than would be possible when content is compressed using the first compression method. Upon recognizing that the content contains such a scene, the client computing device may indicate to the server computing device to use a third compression method (e.g., a lossless or “more lossy” compression method) that provides lower fidelity than the first compression method. The server computing device may then provide the content that is compressed using the third compression method. Later, when the client computing device determines that the received content includes a different scene that the user has not indicated should be received at lower fidelity than would be possible when content is compressed using the third compression method, the client computing device may request the server computing device to revert to using the first compression method (or a different compression method).
In various embodiments, different servers can be used, e.g., one to store client preferences, a second to store content, and a third to respond to client requests. The technology will now be described with reference to the figures.
The routine 100 may further receive the indication of the compression level from a client computing device that requests the content. The routine 100 may further compress the first portion of the content using the first compression method. The first portion of the content may have a first size and a compressed size of the first portion of the content may be smaller than a result of using the second compression method to compress the first portion of the content. The routine 100 may further dynamically select, e.g., in nearly real time, a compression method for each portion of the content based on a content type associated with that portion of the content. A content type may be indicated in metadata associated with at least one of the portions of content. Alternatively, a content type may be automatically detected, e.g., based on a scene. The content type may be at least one of scenery, sex of a speaker, identity of a speaker, sports, wildlife, news, or sexual. As an example, when the first content type is news and the second content type is sports, the first compression method may be more lossy than the second compression method. As another example, when the first content type is a wide-angle view and the second content type is a close-up view, the first compression method may be less lossy than the second compression method.
Those skilled in the art will appreciate that the logic illustrated in
The routine 200 then continues in various embodiments either at block 212 or at block 214. At block 212, the routine 200 optionally compresses portions of the content, e.g., based on indicated compression levels for content types of portions of the content. Alternatively or in addition, the routine 200 can retrieve previously compressed content (not illustrated). In various embodiments, when a portion has been previously compressed at the preferred or desired compression level, the technology may simply retrieve the previously compressed portion instead of compressing it again. The routine 200 then continues at block 214.
At block 214, the routine 200 transmits the content, e.g., to the client that requested the content. The routine 200 then continues at block 216, where it returns.
Although
Depending on the desired configuration, processor 904 may be of any type including but not limited to a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. Processor 904 may include one or more levels of caching, such as a level one cache 910 and a level two cache 912, a processor core 914, and registers 916. An example processor core 914 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 918 may also be used with processor 904, or in some implementations, memory controller 918 may be an internal part of processor 904.
Depending on the desired configuration, system memory 906 may be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. System memory 906 may include an operating system 920, one or more applications 922, and program data 924. Application 922 may include a compressor component 926 that is arranged to compress data using one or more compression methods. Program data 924 may include content 928 (e.g., multimedia content) that clients may request, as is described herein. In some embodiments, application 922 may be arranged to operate with program data 924 on operating system 920 such that rotation of displayed information is enabled or disabled, e.g., depending on an orientation of the display. This described basic configuration 902 is illustrated in
Computing device 900 may have additional features or functionality, and additional interfaces to facilitate communications between basic configuration 902 and any required devices and interfaces. For example, a bus/interface controller 930 may be used to facilitate communications between basic configuration 902 and one or more data storage devices 932 via a storage interface bus 934. Data storage devices 932 may be removable storage devices 936, non-removable storage devices 938, or a combination thereof. Examples of removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. Example computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data.
System memory 906, removable storage devices 936 and non-removable storage devices 938 are examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which may be used to store the desired information and which may be accessed by computing device 900. Any such computer storage media may be part of computing device 900.
Computing device 900 may also include an interface bus 940 for facilitating communication from various interface devices (e.g., output devices 942, peripheral interfaces 944, and communication devices 946) to basic configuration 902 via bus/interface controller 930. Example output devices 942 include a graphics processing unit 948 and an audio processing unit 950, which may be configured to communicate to various external devices such as a display or speakers via one or more A/V ports 952. Example peripheral interfaces 944 include a serial interface controller 954 or a parallel interface controller 956, which may be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I/O ports 958. An example communication device 946 includes a network controller 960, which may be arranged to facilitate communications with one or more other computing devices 962 over a network communication link via one or more communication ports 964.
The network communication link may be one example of a communication media. Communication media may typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media. A “modulated data signal” may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term computer readable media as used herein may include both storage media and communication media.
Computing device 900 may be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a personal data assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions. Computing device 900 may also be implemented as a personal computer including both laptop computer and non-laptop computer configurations.
From the foregoing, it will be appreciated that various embodiments of the present disclosure have been described herein for purposes of illustration, and that various modifications may be made without departing from the scope of the present disclosure. Accordingly, the various embodiments disclosed herein are not intended to be limiting, with the true scope being indicated by the following claims.
Claims
1. A method, comprising:
- determining a respective content type for each portion of at least two portions of content;
- determining, based on the respective content type, a respective compression level for each portion of the at least two portions of the content;
- compressing, based on the respective compression level, each portion of the at least two portions of the content, wherein the compressing each portion comprises: compressing a first portion, of the at least two portion of the content, using a first compression level, wherein the first portion corresponds to a first content type; and compressing a second portion, of the at least two portions of the content, using a second compression level, wherein the second portion corresponds to a second content type; and
- sending the compressed at least two portions of the content to a user device.
2. The method of claim 1, wherein the determining the respective compression level for each portion of the at least two portions of the content comprises:
- obtaining, from the user device, a first indication of the first compression level for the first content type and a second indication of the second compression level for the second content type.
3. The method of claim 1, further comprising:
- determining a respective compression method for the first compression level and the second compression level, wherein:
- the compressing the first portion using the first compression level comprises compressing the first portion using a first compression method that corresponds to the first compression level, and
- the compressing the second portion using the second compression level comprises compressing the second portion using a second compression method that corresponds to the second compression level.
4. The method of claim 3, wherein the first compression method and the second compression method includes one of a lossless compression method and a lossy compression method.
5. The method of claim 1, wherein the determining the respective content type for each portion of the at least two portions of the content comprises checking metadata associated with each portion of the at least two portions of the content.
6. The method of claim 1, wherein the determining the respective content type for each portion of the at least two portions of the content comprises dynamically evaluating the content.
7. The method of claim 6, wherein the dynamically evaluating the content comprises employing one or more of: facial detection, location recognition, nudity recognition, and voice recognition.
8. The method of claim 1, wherein the determining the respective compression level for each portion of the at least two portions of the content comprises selecting the respective compression level, from a plurality of compression levels, for each portion of the at least two portions of the content, dynamically in real-time.
9. The method of claim 1, wherein the first portion of the content includes a wide-angle view, and the second portion of the content includes a close-up view, and wherein the first compression level is lossier than the second compression level.
10. The method of claim 1, wherein the compressing the second portion of the content using the second compression level comprises retrieving a previously compressed second portion of the content from a storage, and wherein the previously compressed second portion of the content is compressed using the second compression level.
11. The method of claim 1, further comprising:
- obtaining an indication of a third compression level for the first portion of the content that has been sent to the user device;
- compressing the first portion of the content using the third compression level; and
- re-sending the first portion of the content, which has been compressed using the third compression level, to the user device, wherein the third compression level has different fidelity than the first compression level.
12. A system, comprising:
- one or more processors configured to perform or cause to be performed: determine a respective content type for each portion of at least two portions of content; determine, based on the respective content type, a respective compression level for each portion of the at least two portions of the content; compress, based on the respective compression level, each portion of the at least two portions of the content,
- wherein to compress each portion, the one or more processors are configured to perform or cause to be performed: compress a first portion, of the at least two portion of the content, by use of a first compression level, wherein the first portion corresponds to a first content type; and compress a second portion, of the at least two portions of the content, by use of a second compression level, wherein the second portion corresponds to a second content type; and send the compressed at least two portions of the content to a user device; and
- a memory communicatively coupled to the one or more processors, wherein the memory is configured to store one or more of: the content, the compressed at least two portions of the content, the respective compression levels, and the respective content types.
13. The system of claim 12, further comprising:
- a communication interface communicatively coupled to the one or more processors and to the memory, wherein the communication interface is configured to obtain the compressed at least two portions of the content from the one or more processors or from the memory, and to transmit the compressed at least two portions of the content to the user device.
14. The system of claim 12, wherein the content type includes at least one of: scenery, gender of a speaker, identity of the speaker, sports, wildlife, news, or sexual.
15. The system of claim 12, wherein to determine the respective compression level for each portion, the one or more processors are configured to perform or cause to be performed:
- obtain, from the user device, a first indication of the first compression level for the first content type and a second indication of the second compression level for the second content type.
16. The system of claim 12, wherein the first compression level corresponds to a first compression method, and the second compression level corresponds to a second compression method, and wherein the first compression method and the second compression method includes one of a lossless compression method and a lossy compression method.
17. The system of claim 12, wherein to determine the respective content type of each portion of the at least two portions of the content, the one or more processors are configured to perform or cause to be performed at least one or more of:
- check metadata associated each portion of the at least two portions of the content; and
- dynamically evaluate the content by use of one or more of: facial detection, location recognition, nudity recognition, and voice recognition.
18. The system of claim 12, wherein the content includes multimedia content to download or to stream on the user device, wherein the first portion of the content corresponds to a first scene of the multimedia content, and wherein the second portion of the content corresponds to a second scene of the multimedia content.
19. The system of claim 12, wherein the memory is further configured to store one or more of:
- associations between a plurality of compression levels and a plurality of compression methods, and
- a table that comprises information of client preferences to compress one or more particular content types.
20. A non-transitory computer readable medium that stores instructions which, in response to execution by one or more processors, cause the one or more processors to perform or control performance of operations comprising:
- determine a respective content type for each portion of at least two portions of content;
- determine, based on the respective content type, a respective compression method for each portion of the at least two portions of the content;
- compress, based on the respective compression method, each portion of the at least two portions of the content, wherein the operation to compress each portion comprises at least one operation to: compress a first portion, of the at least two portion of the content, by use of a first compression method, wherein the first portion corresponds to a first content type; and compress a second portion, of the at least two portions of the content, by use a second compression method, wherein the second portion corresponds to a second content type; and
- send the compressed at least two portions of the content to a user device, wherein: the first compression method and the second compression method includes one of: a lossless compression method and a lossy compression method, and the first compression method and the second compression method correspond respectively to a first compression level and a second compression level.
21. The non-transitory computer readable medium of claim 20, wherein the content includes multimedia content to download or to stream on the user device, wherein the first portion of the content corresponds to a first scene of the multimedia content, and wherein the second portion of the content corresponds to a second scene of the multimedia content.
22. The non-transitory computer readable medium of claim 21, wherein the multimedia content includes content that correspond to one or more of violence, news, sports, dance, sexual, or wildlife.
23. The non-transitory computer readable medium of claim 20, wherein the first compression method and the second compression method are specified by a requestor of the content.
24. The non-transitory computer readable medium of claim 20, wherein the first portion of the content and the second portion of the content respectively comprise a first scene of a show and a second scene the show, and wherein the compression of the first portion includes a compression of the first scene and the compression of the second portion includes a compression of the second scene so that the show is compressed differently scene-by-scene.
Type: Application
Filed: Nov 17, 2017
Publication Date: Mar 15, 2018
Applicant: EMPIRE TECHNOLOGY DEVELOPMENT LLC (WILMINGTON, DE)
Inventor: Shmuel Ur (Galil)
Application Number: 15/815,823