EFFICIENTLY PROCESSING MULTIPLE NON-OVERLAPPING LAYER IMAGES IN DISPLAY PROCESSING UNITS
Efficiently processing multiple non-overlapping layer images in display processing units is disclosed herein. In this regard, in some exemplary aspects, a display processing unit comprising a plurality of memory access pipeline circuits and a layer mixer circuit is provided. For each non-overlapping layer image of a plurality of non-overlapping layer images, a memory access pipeline circuit obtains image configuration data for the non-overlapping layer image, and fetches the non-overlapping layer image from an image data storage device based on the image configuration data. The memory access pipeline circuit then outputs each pixel of the non-overlapping layer image as part of an intermediate preblend image data stream based on the image configuration data. The layer mixer circuit blends the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream, and outputs the display data stream to a display device.
The technology of the disclosure relates generally to display processing units in processor-based devices, and, in particular, to processing and blending layer images for display.
II. BackgroundModern processor-based devices may include a dedicated processing unit, known as a display processing unit, to handle operations for fetching, processing, and blending image data for display on a display device. A conventional display processing unit comprises multiple memory access pipelines, each of which includes, e.g., a fetch circuit that retrieves image data from memory or other image data storage device, as well as a pixel processing unit that performs image processing (e.g., color correction operations, zoom operation, scaling operations, and the like) on the image data. The image data is then fed by each memory access pipeline to a layer mixer circuit that performs blending of multiple layer images, and further to a pixel processing unit that performs display-specific image processing operations. Finally, the blended and processed image data is sent to a display device for display to a user.
One task commonly performed by a display processing unit is the fetching and blending of multiple non-overlapping layer images with a background layer image. In conventional display processing units, each memory access pipeline is configured to fetch a single non-overlapping layer image. Blending multiple non-overlapping layer images with a background layer image thus requires the use of multiple memory access pipelines, one for the background layer image and one for each non-overlapping layer image. Consequently, the number of non-overlapping images that can be blended by a display processing unit is limited by the number of memory access pipelines provided by the display processing unit.
SUMMARY OF THE DISCLOSUREAspects disclosed in the detailed description include efficiently processing multiple non-overlapping layer images in display processing units. Related apparatus, methods, and computer-readable media are also disclosed. In this regard, in some exemplary aspects disclosed herein, a display processing unit provides a plurality of memory access pipeline circuits and a layer mixer circuit. A first memory access pipeline circuit is configured to fetch and preblend a plurality of non-overlapping layer images (i.e., layer images whose position and dimensions do not overlap) into an intermediate preblend image, which is then blended by the layer mixer circuit with a background layer image fetched by a second memory access pipeline. To generate the intermediate preblend image, the first memory access pipeline performs a series of operations for each of the non-overlapping layer images. The first memory access pipeline first obtains image configuration data for the non-overlapping layer image, then fetches the non-overlapping layer image from an image data storage device based on the image configuration data. The first memory access pipeline next outputs each pixel of the non-overlapping layer image as part of the intermediate preblend image to an intermediate preblend image data stream based on the image configuration data. Upon receiving the intermediate preblend image data stream from the first memory access pipeline and a background image data stream from the second memory access pipeline, the layer mixer circuit blends the intermediate preblend image data stream and the background layer image data stream as a display data stream. The display data stream may then be transmitted to a display device for display to a user.
In some aspects, the first memory access pipeline circuit may comprise a plurality of image configuration registers, which may receive the image configuration data from, e.g., an executing software process, and which may be used to populate an image configuration queue. The first memory access pipeline circuit may then obtain the image configuration data for each non-overlapping layer image from a top entry in the image configuration queue (e.g., by “popping” the image configuration for each non-overlapping layer image from the image configuration queue). According to some aspects, the first memory access pipeline circuit may calculate a set of position coordinates and a size of the intermediate preblend image to be generated, based on the image configuration data received for the plurality of non-overlapping layer images. The first memory access pipeline circuit may then transmit the set of position coordinates and the size of the intermediate preblend image to the layer mixer circuit for use in blending the intermediate preblend image data stream and the background layer image data stream. In some aspects, the first memory access pipeline circuit may comprise two (2) image configuration queues, such that the first memory access pipeline circuit may obtain image configuration data from a first image configuration queue while receiving, in parallel, image configuration data for a next set of non-overlapping layer images into the second image configuration queue.
Some aspects may provide that, as part of outputting each pixel of the non-overlapping layer image to the intermediate preblend image data stream, the first memory access pipeline circuit may output a blend bypass pixel (e.g., a pixel having a pre-specified color value) for each pixel of the intermediate preblend image that does not correspond to a pixel of the non-overlapping layer image. When the layer mixer circuit subsequently blends the intermediate preblend image data stream and the background layer image data stream, the layer mixer circuit may blend each pixel of the intermediate preblend image data stream that corresponds to a non-overlapping layer with a corresponding pixel of the background layer image. The layer mixer circuit may also output a corresponding pixel of the background layer image for each blend bypass pixel of the intermediate preblend image data stream.
In some aspects, the layer mixer circuit may employ multiple blending stages to blend images from multiple memory access pipeline circuits. According to such aspects, each blending stage corresponds to a memory access pipeline circuit of the plurality of memory access pipeline circuits, and the layer mixer circuit outputs the display data stream of each blending stage preceding the final blending stage as the background layer image data stream to a successive blending stage. The layer mixer circuit then outputs the display data stream of the final blending stage to the display device. Some such aspects may enable the first memory access pipeline circuit to send image data to one or more blending stages by providing a layer mixer configuration queue, into which the layer mixer circuit receives a memory access pipeline identifier and a blending stage indication for the first memory access pipeline circuit. The layer mixer circuit assigns a tag corresponding to the layer mixer configuration queue to the first memory access pipeline circuit based on the memory access pipeline identifier, thereby associating the layer mixer configuration queue with the first memory access pipeline. Upon receiving the intermediate preblend image data stream from the first memory access pipeline circuit, the layer mixer circuit obtains the memory access pipeline identifier and the blending stage indication from the layer mixer configuration queue based on the tag, and blends the intermediate preblend image data stream and the background layer image data stream in the appropriate blending stage based on the blending stage indication.
In another aspect, a display processing unit is provided. The display processing unit comprises a plurality of memory access pipeline circuits and a layer mixer circuit. A first memory access pipeline circuit of the plurality of memory access pipeline circuits is configured to, for each non-overlapping layer image of a plurality of non-overlapping layer images, obtain image configuration data for the non-overlapping layer image. The first memory access pipeline circuit is further configured fetch the non-overlapping layer image from an image data storage device based on the image configuration data. The first memory access pipeline circuit is also configured to output each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data. The layer mixer circuit is configured to blend the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream. The layer mixer circuit is further configured to output the display data stream to a display device.
In another aspect, a display processing unit is provided. The display processing unit comprises means for, for each non-overlapping layer image of a plurality of non-overlapping layer images, obtaining image configuration data for the non-overlapping layer image, fetching the non-overlapping layer image from an image data storage device based on the image configuration data, and outputting each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data. The display processing unit further comprises means for blending the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream. The display processing unit also comprises means for outputting the display data stream to a display device.
In another aspect, a method for efficiently processing multiple non-overlapping layer images is provided. The method comprises, for each non-overlapping layer image of a plurality of non-overlapping layer images, obtaining, by a first memory access pipeline circuit of a display processing unit, image configuration data for the non-overlapping layer image. The method further comprises fetching, by the first memory access pipeline circuit, the non-overlapping layer image from an image data storage device based on the image configuration data. The method also comprises outputting, by the first memory access pipeline circuit, each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data. The method additionally comprises blending, by a layer mixer circuit of the display processing unit, the intermediate preblend image data stream and a background layer image data stream as a display data stream. The method further comprises outputting, by the layer mixer circuit, the display data stream to a display device.
With reference now to the drawing figures, several exemplary aspects of the present disclosure are described. The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.
Aspects disclosed in the detailed description include efficiently processing multiple non-overlapping layer images in display processing units. Related apparatus, methods, and computer-readable media are also disclosed. In this regard, in some exemplary aspects disclosed herein, a display processing unit provides a plurality of memory access pipeline circuits and a layer mixer circuit. A first memory access pipeline circuit is configured to fetch and preblend a plurality of non-overlapping layer images (i.e., layer images whose position and dimensions do not overlap) into an intermediate preblend image, which is then blended by the layer mixer circuit with a background layer image fetched by a second memory access pipeline. To generate the intermediate preblend image, the first memory access pipeline performs a series of operations for each of the non-overlapping layer images. The first memory access pipeline first obtains image configuration data for the non-overlapping layer image, then fetches the non-overlapping layer image from an image data storage device based on the image configuration data. The first memory access pipeline next outputs each pixel of the non-overlapping layer image as part of the intermediate preblend image to an intermediate preblend image data stream based on the image configuration data. Upon receiving the intermediate preblend image data stream from the first memory access pipeline and a background image data stream from the second memory access pipeline, the layer mixer circuit blends the intermediate preblend image data stream and the background layer image data stream as a display data stream. The display data stream may then be transmitted to a display device for display to a user.
In some aspects, the first memory access pipeline circuit may comprise a plurality of image configuration registers, which may receive the image configuration data from, e.g., an executing software process, and which may be used to populate an image configuration queue. The first memory access pipeline circuit may then obtain the image configuration data for each non-overlapping layer image from a top entry in the image configuration queue (e.g., by “popping” the image configuration for each non-overlapping layer image from the image configuration queue). According to some aspects, the first memory access pipeline circuit may calculate a set of position coordinates and a size of the intermediate preblend image to be generated, based on the image configuration data received for the plurality of non-overlapping layer images. The first memory access pipeline circuit may then transmit the set of position coordinates and the size of the intermediate preblend image to the layer mixer circuit for use in blending the intermediate preblend image data stream and the background layer image data stream. In some aspects, the first memory access pipeline circuit may comprise two (2) image configuration queues, such that the first memory access pipeline circuit may obtain image configuration data from a first image configuration queue while receiving, in parallel, image configuration data for a next set of non-overlapping layer images into the second image configuration queue.
Some aspects may provide that, as part of outputting each pixel of the non-overlapping layer image to the intermediate preblend image data stream, the first memory access pipeline circuit may output a blend bypass pixel (e.g., a pixel having a pre-specified color value) for each pixel of the intermediate preblend image that does not correspond to a pixel of the non-overlapping layer image. When the layer mixer circuit subsequently blends the intermediate preblend image data stream and the background layer image data stream, the layer mixer circuit may blend each pixel of the intermediate preblend image data stream that corresponds to a non-overlapping layer with a corresponding pixel of the background layer image. The layer mixer circuit may also output a corresponding pixel of the background layer image for each blend bypass pixel of the intermediate preblend image data stream.
In some aspects, the layer mixer circuit may employ multiple blending stages to blend images from multiple memory access pipeline circuits. According to such aspects, each blending stage corresponds to a memory access pipeline circuit of the plurality of memory access pipeline circuits, and the layer mixer circuit outputs the display data stream of each blending stage preceding the final blending stage as the background layer image data stream to a successive blending stage. The layer mixer circuit then outputs the display data stream of the final blending stage to the display device. Some such aspects may enable the first memory access pipeline circuit to send image data to one or more blending stages by providing a layer mixer configuration queue, into which the layer mixer circuit receives a memory access pipeline identifier and a blending stage indication for the first memory access pipeline circuit. The layer mixer circuit assigns a tag corresponding to the layer mixer configuration queue to the first memory access pipeline circuit based on the memory access pipeline identifier, thereby associating the layer mixer configuration queue with the first memory access pipeline. Upon receiving the intermediate preblend image data stream from the first memory access pipeline circuit, the layer mixer circuit obtains the memory access pipeline identifier and the blending stage indication from the layer mixer configuration queue based on the tag, and blends the intermediate preblend image data stream and the background layer image data stream in the appropriate blending stage based on the blending stage indication.
In this regard,
The display processing unit 104 of
The processor-based device 100 of
As noted above, one task commonly performed by conventional display processing units is the fetching and blending of multiple non-overlapping layer images with a background layer image. Because memory access pipelines in conventional display processing units are each configured to fetch a single non-overlapping layer image, the blending of multiple non-overlapping layer images with a background layer image requires the use of multiple memory access pipelines (i.e., one for the background layer image and one for each non-overlapping layer image). As a result, the number of non-overlapping images that can be blended is limited by the number of memory access pipelines that are provided by the display processing unit.
Accordingly, to efficiently process multiple non-overlapping layer images, the memory access pipeline circuits 108(0)-108(M) of the display processing unit 104 of
Before describing the memory access pipeline circuits 108(0)-108(M) and the layer mixer circuit 118 of
The operation of conventional display processing units when blending multiple non-overlapping layer images and a background layer image as described in
In
As seen in
Accordingly,
The memory access pipeline circuit 108(1) of
To generate the intermediate preblend image data stream 400, the memory access pipeline circuit 108(1) in some aspects may first obtain image configuration data (captioned as “IMG CONFIG” in
Some aspects may provide that, as the memory access pipeline circuit 108(1) receives the image configuration data 406 for each non-overlapping layer image 202(0)-202(3), the memory access pipeline circuit 108(1) may calculate a set of position coordinates (captioned as “POSITION” in
In some aspects, the memory access pipeline circuit 108(1) may provide two (2) image configuration queues 412(0) and 412(1). In such aspects, the memory access pipeline circuit 108(1) is configured to obtain image configuration data 406 from, e.g., the image configuration queue 412(0) while receiving, in parallel, image configuration data 406 for a next set of non-overlapping layer images (not shown) into a plurality of image configuration queue entries (captioned as “ENTRY” in
Exemplary image configuration registers 408(0)-408(C) according to some aspects may include the registers described below in Table 1:
After obtaining the image configuration data 406 for a non-overlapping layer image such as the non-overlapping layer image 202(0) from the image configuration queue 412(0), the memory access pipeline circuit 108(1) then uses the preblending circuit 122(1) to output each pixel of the non-overlapping layer image 202(0) as part of the intermediate preblend image to the intermediate preblend image data stream 400 based on the image configuration data 406 for the non-overlapping layer image 202(0). In some aspects, the operations for outputting each pixel of the non-overlapping layer image 202(0) to the intermediate preblend image data stream 400 involves outputting a blend bypass pixel (e.g., a pixel having a pre-specified color value) for each pixel of the intermediate preblend image that does not correspond to a pixel of the non-overlapping layer image 202(0), wherein the blend bypass pixel information contains coordinates (e.g., X and Y positions) of the intermediate preblend image.
The preblending circuit 122(1) uses the image configuration data to determine destination coordinates for the non-overlapping layer image 202(0) in the intermediate preblend image and tracks output pixel positions. Upon completing processing for the non-overlapping layer image 202(0), the memory access pipeline circuit 108(1) then obtains the image configuration data 406 for the next non-overlapping layer image (e.g., the non-overlapping layer image 202(1)) from the image configuration queue 412(0), and begins processing the image data for the non-overlapping layer image 202(1).
The layer mixer circuit 118 subsequently blends the intermediate preblend image data stream 400 and the background layer image data stream 402 into the display data stream 404 by blending each pixel of the intermediate preblend image data stream 400 that corresponds to one of the non-overlapping layer images 202(0)-202(3) with a corresponding pixel of the background layer image 200. In some aspects, each of the non-overlapping layer images 202(0)-202(3) may be associated with different alpha mode data (not shown) and/or different alpha value data (not shown) that is used by the layer mixer circuit 118 when blending each pixel of the intermediate preblend image data stream 400 that corresponds to one of the non-overlapping layer images 202(0)-202(3) with a corresponding pixel of the background layer image 200. The alpha mode data and/or alpha value data in some aspects may be stored in software-configurable read and write registers provided for each of a plurality of blend stages, which are discussed in greater detail below. The layer mixer circuit 118 also outputs a corresponding pixel of the background layer image 200 to the display data stream 404 for each blend bypass pixel of the intermediate preblend image data stream 400.
Some aspects of the layer mixer circuit 118 may provide that the non-overlapping layer images 202(0)-202(3) within the intermediate preblend image data stream 400 may each be processed within a separate blending stage of a plurality of blending stages, as discussed in greater detail below with respect to
To support functionality for providing blending stages, the layer mixer circuit 118 in some aspects provides a plurality of layer mixer configuration queues (captioned as “LM CONFIG QUEUE” in
The layer mixer configuration queues 420(0)-420(L) may be populated, e.g., by a software process such as a driver process, using layer mixer configuration registers (captioned as “LM CFG REG” in
When the layer mixer circuit 118 receives an intermediate preblend data stream from a memory access pipeline circuit such as the memory access pipeline circuit 108(1), or receives an end-of-line (EOL) indication upon detection of bypass_Y, the layer mixer circuit 118 obtains a memory access pipeline identifier (not shown) and a blending stage indication (not shown) from the layer mixer configuration queue 420(0)-420(L) identified by the tag associated with the memory access pipeline circuit 108(1) (e.g., by popping the top layer mixer configuration queue entry 422(0) from the layer mixer configuration queue 420(0)). The layer mixer circuit 118 then blends the intermediate preblend image data stream (e.g., the intermediate preblend image data stream 400) and the background layer image data stream (e.g., the background layer image data stream 402) in the appropriate blending stage based on the blending stage indication. An example of the use of blending stages in this manner is discussed in greater detail below with respect to
Exemplary results of the operations described above with respect to
As seen in
As discussed above with respect to
The memory access pipeline circuit 108(2) next provides a first intermediate preblend data stream 712 to the blending stage 700(2), which the layer mixer circuit 118 blends with the background layer image data stream 710 to generate a display data stream 714. The display data stream 714 is then provided to the final blending stage 700(3) as a background layer image data stream 714. The memory access pipeline circuit 108(2) provides a second intermediate preblend data stream 716 to the final blending stage 700(3), which is blended with the background layer image data stream 714 into a display data stream 718. The display data stream 718 is then output to the display device 106 for display.
To illustrate exemplary operations by the display processing unit 104 of
Referring now to
The memory access pipeline circuit 108(1) then outputs each pixel of the non-overlapping layer image 202(0) as part of the intermediate preblend image 500 to an intermediate preblend image data stream (e.g., the intermediate preblend image data stream 400 of
Turning now to
Subsequently, when the layer mixer circuit 118 receives the intermediate preblend image data stream 400 from the memory access pipeline circuit 108(1), the layer mixer circuit 118 obtains the memory access pipeline identifier and the blending stage indication from a top entry (e.g., the layer mixer configuration queue entry 422(0) of
Referring now to
Efficiently processing multiple non-overlapping layer images according to aspects disclosed herein may be provided in or integrated into any processor-based device. Examples, without limitation, include a set top box, an entertainment unit, a navigation device, a communications device, a fixed location data unit, a mobile location data unit, a global positioning system (GPS) device, a mobile phone, a cellular phone, a smart phone, a session initiation protocol (SIP) phone, a tablet, a phablet, a server, a computer, a portable computer, a mobile computing device, laptop computer, a wearable computing device (e.g., a smart watch, a health or fitness tracker, eyewear, etc.), a desktop computer, a personal digital assistant (PDA), a monitor, a computer monitor, a television, a tuner, a radio, a satellite radio, a music player, a digital music player, a portable music player, a digital video player, a video player, a digital video disc (DVD) player, a portable digital video player, an automobile, a vehicle component, an avionics system, a drone, and a multicopter.
In this regard,
Other master and slave devices can be connected to the system bus 1008. As illustrated in
The processor 1002 may also be configured to access the display controller(s) 1022 over the system bus 1008 to control information sent to one or more displays 1026. The display controller(s) 1022 sends information to the display(s) 1026 to be displayed via one or more video processors 1028, which process the information to be displayed into a format suitable for the display(s) 1026. The display(s) 1026 can include any type of display, including but not limited to a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, etc.
Those of skill in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithms described in connection with the aspects disclosed herein may be implemented as electronic hardware, instructions stored in memory or in another computer readable medium and executed by a processor or other processing device, or combinations of both. Memory disclosed herein may be any type and size of memory and may be configured to store any type of information desired. To clearly illustrate this interchangeability, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. How such functionality is implemented depends upon the particular application, design choices, and/or design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
The various illustrative logical blocks, modules, and circuits described in connection with the aspects disclosed herein may be implemented or performed with a processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration).
The aspects disclosed herein may be embodied in hardware and in instructions that are stored in hardware, and may reside, for example, in Random Access Memory (RAM), flash memory, Read Only Memory (ROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), registers, a hard disk, a removable disk, a CD-ROM, or any other form of computer readable medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a remote station. In the alternative, the processor and the storage medium may reside as discrete components in a remote station, base station, or server.
It is also noted that the operational steps described in any of the exemplary aspects herein are described to provide examples and discussion. The operations described may be performed in numerous different sequences other than the illustrated sequences. Furthermore, operations described in a single operational step may actually be performed in a number of different steps. Additionally, one or more operational steps discussed in the exemplary aspects may be combined. It is to be understood that the operational steps illustrated in the flowchart diagrams may be subject to numerous different modifications as will be readily apparent to one of skill in the art. Those of skill in the art will also understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Implementation examples are described in the following numbered clauses:
-
- 1. A display processing unit, comprising:
- a plurality of memory access pipeline circuits;
- a layer mixer circuit;
- a first memory access pipeline circuit of the plurality of memory access pipeline circuits configured to, for each non-overlapping layer image of a plurality of non-overlapping layer images:
- obtain image configuration data for the non-overlapping layer image;
- fetch the non-overlapping layer image from an image data storage device based on the image configuration data; and
- output each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data; and
- the layer mixer circuit configured to:
- blend the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream; and
- output the display data stream to a display device.
- 2. The display processing unit of clause 1, wherein the layer mixer circuit is further configured to:
- receive the intermediate preblend image data stream from the first memory access pipeline circuit; and
- receive the background layer image data stream from a second memory access pipeline circuit of the plurality of memory access pipeline circuits.
- 3. The display processing unit of any one of clauses 1-2, wherein the image configuration data for each non-overlapping layer image of the plurality of the non-overlapping layer images comprises one or more of a source image size, an output image size, a set of source image position coordinates, a set of output image position coordinates, a source address, a stride value, an image format, and an unpack pattern value.
- 4. The display processing unit of any one of clauses 1-3, wherein:
- the first memory access pipeline circuit further comprises an image configuration queue;
- the first memory access pipeline circuit is further configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, receive, from a plurality of image configuration registers, the image configuration data into the image configuration queue; and the first memory access pipeline circuit is configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, obtain the image configuration data from a top entry of the image configuration queue.
- 5. The display processing unit of clause 4, wherein:
- the first memory access pipeline circuit is further configured to:
- calculate a set of position coordinates and a size of the intermediate preblend image, based on the image configuration data for the plurality of non-overlapping layer images; and
- transmit the set of position coordinates and the size of the intermediate preblend image to the layer mixer circuit; and
- the layer mixer circuit configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream based on the set of position coordinates and the size of the intermediate preblend image.
- the first memory access pipeline circuit is further configured to:
- 6. The display processing unit of any one of clauses 4-5, wherein:
- the first memory access pipeline circuit comprises a first selectable image configuration queue and a second selectable image configuration queue;
- the first memory access pipeline circuit is further configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, receive, from a plurality of image configuration registers, the image configuration data into the first selectable image configuration queue;
- the first memory access pipeline circuit is configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, obtain the image configuration data from a top entry of the first selectable image configuration queue; and
- the first memory access pipeline circuit is further configured to, in parallel with obtaining the image configuration data from the top entry of the first selectable image configuration queue, for each non-overlapping layer image of a next plurality of non-overlapping layer images, receive, from the plurality of image configuration registers, next image configuration data into the second selectable image configuration queue.
- 7. The display processing unit of any one of clauses 1-6, wherein the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to, for each pixel of the intermediate preblend image data stream that corresponds to a non-overlapping layer image of the plurality of non-overlapping layer images, output, to the display data stream, a blend of the pixel with a corresponding pixel of the background layer image.
- 8. The display processing unit of clause 7, wherein:
- the first memory access pipeline circuit is further configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, output, to the intermediate preblend image data stream, a blend bypass pixel for each pixel of the intermediate preblend image that does not correspond to a pixel of the non-overlapping layer image; and
- the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to, for each blend bypass pixel of the intermediate preblend image data stream, output, to the display data stream, a corresponding pixel of the background layer image.
- 9. The display processing unit of clause 8, wherein the blend bypass pixel comprises a pixel having a pre-specified color value.
- 10. The display processing unit of any one of clauses 2-9, wherein:
- the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to:
- blend the intermediate preblend image data stream and the background layer image data stream as the display data stream within a blending stage of a plurality of blending stages, wherein each blending stage corresponds to a memory access pipeline circuit of the plurality of memory access pipeline circuits; and
- output the display data stream of each blending stage preceding a final blending stage of the plurality of blending stages as the background layer image data stream to a successive blending stage; and
- the layer mixer circuit is configured to output the display data stream to the display device by being configured to output the display data stream of the final blending stage of the plurality of blending stages to the display device.
- the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to:
- 11. The display processing unit of clause 10, wherein:
- the layer mixer circuit further comprises a plurality of layer mixer configuration queues;
- the layer mixer circuit is further configured to:
- receive a memory access pipeline identifier and a blending stage indication for the first memory access pipeline circuit into a layer mixer configuration queue of the plurality of layer mixer configuration queues;
- assign a tag corresponding to the layer mixer configuration queue to the first memory access pipeline circuit based on the memory access pipeline identifier; and
- responsive to receiving the intermediate preblend image data stream from the first memory access pipeline circuit, obtain the memory access pipeline identifier and the blending stage indication from a top entry of the layer mixer configuration queue based on the tag; and
- the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream within the blending stage of the plurality of blending stages based on the blending stage indication.
- 12. The display processing unit of any one of clauses 1-11, integrated into a device selected from the group consisting of: a set top box; an entertainment unit; a navigation device; a communications device; a fixed location data unit; a mobile location data unit; a global positioning system (GPS) device; a mobile phone; a cellular phone; a smart phone; a session initiation protocol (SIP) phone; a tablet; a phablet; a server; a computer; a portable computer; a mobile computing device; a wearable computing device; a desktop computer; a personal digital assistant (PDA); a monitor; a computer monitor; a television; a tuner; a radio; a satellite radio; a music player; a digital music player; a portable music player; a digital video player; a video player; a digital video disc (DVD) player; a portable digital video player; an automobile; a vehicle component; avionics systems; a drone; and a multicopter.
- 13. A display processing unit, comprising:
- means for, for each non-overlapping layer image of a plurality of non-overlapping layer images:
- obtaining image configuration data for the non-overlapping layer image;
- fetching the non-overlapping layer image from an image data storage device based on the image configuration data; and
- outputting each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data;
- means for blending the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream; and
- means for outputting the display data stream to a display device.
- means for, for each non-overlapping layer image of a plurality of non-overlapping layer images:
- 14. A method for efficiently processing multiple non-overlapping layer images, the method comprising:
- for each non-overlapping layer image of a plurality of non-overlapping layer images:
- obtaining, by a first memory access pipeline circuit of a plurality of memory access pipeline circuits of a display processing unit, image configuration data for the non-overlapping layer image;
- fetching, by the first memory access pipeline circuit, the non-overlapping layer image from an image data storage device based on the image configuration data; and
- outputting, by the first memory access pipeline circuit, each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data;
- blending, by a layer mixer circuit of the display processing unit, the intermediate preblend image data stream and a background layer image data stream as a display data stream; and
- outputting, by the layer mixer circuit, the display data stream to a display device.
- for each non-overlapping layer image of a plurality of non-overlapping layer images:
- 15. The method of clause 14, further comprising:
- receiving, by the layer mixer circuit, the intermediate preblend image data stream from the first memory access pipeline circuit; and
- receiving, by the layer mixer circuit, the background layer image data stream comprising a background layer image from a second memory access pipeline circuit of the plurality of memory access pipeline circuits.
- 16. The method of any one of clauses 14-15, wherein the image configuration data for each non-overlapping layer image of the plurality of the non-overlapping layer images comprises one or more of a source image size, an output image size, a set of source image position coordinates, a set of output image position coordinates, a source address, a stride value, an image format, and an unpack pattern value.
- 17. The method of any one of clauses 14-16, further comprising, for each non-overlapping layer image of the plurality of non-overlapping layer images, receiving, by the first memory access pipeline circuit from a plurality of image configuration registers, the image configuration data into an image configuration queue of the first memory access pipeline circuit;
- wherein, for each non-overlapping layer image of the plurality of non-overlapping layer images, obtaining the image configuration data for the non-overlapping layer image comprises obtaining the image configuration data from a top entry of the image configuration queue.
- 18. The method of clause 17, further comprising:
- calculating, by the first memory access pipeline circuit, a set of position coordinates and a size of the intermediate preblend image, based on the image configuration data for the plurality of non-overlapping layer images; and
- transmitting, by the first memory access pipeline circuit, the set of position coordinates and the size of the intermediate preblend image to the layer mixer circuit;
- wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream is based on the set of position coordinates and the size of the intermediate preblend image.
- 19. The method of any one of clauses 17-18, further comprising, for each non-overlapping layer image of the plurality of non-overlapping layer images, receiving, by the first memory access pipeline circuit from a plurality of image configuration registers, the image configuration data into a first selectable image configuration queue;
- wherein:
- for each non-overlapping layer image of the plurality of non-overlapping layer images, obtaining the image configuration data comprises obtaining the image configuration data from a top entry of the first selectable image configuration queue; and
- the method further comprises, in parallel with obtaining the image configuration data from the top entry of the first selectable image configuration queue, for each non-overlapping layer image of a next plurality of non-overlapping layer images, receiving, by the first memory access pipeline circuit from the plurality of image configuration registers, next image configuration data into a second selectable image configuration queue.
- wherein:
- 20. The method of any one of clauses 15-19, wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises, for each pixel of the intermediate preblend image data stream that corresponds to a non-overlapping layer image of the plurality of non-overlapping layer images, outputting, by the layer mixer circuit to the display data stream, a blend of the pixel with a corresponding pixel of the background layer image.
- 21. The method of any one of clauses 15-20, further comprising, for each non-overlapping layer image of the plurality of non-overlapping layer images, outputting, by the first memory access pipeline circuit to the intermediate preblend image data stream, a blend bypass pixel for each pixel of the intermediate preblend image that does not correspond to a pixel of the non-overlapping layer image;
- wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises, for each blend bypass pixel of the intermediate preblend image data stream, outputting, by the layer mixer circuit to the display data stream, a corresponding pixel of the background layer image.
- 22. The method of clause 21, wherein the blend bypass pixel comprises a pixel having a pre-specified color value.
- 23. The method of any one of clauses 15-22, wherein:
- blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises:
- blending, by the layer mixer circuit, the intermediate preblend image data stream and the background layer image data stream as the display data stream within a blending stage of a plurality of blending stages, wherein each blending stage corresponds to a memory access pipeline circuit of the plurality of memory access pipeline circuits; and
- outputting, by the layer mixer circuit, the display data stream of each blending stage preceding a final blending stage of the plurality of blending stages as the background layer image data stream to a successive blending stage; and
- outputting the display data stream to the display device comprises outputting, by the layer mixer circuit, the display data stream of the final blending stage of the plurality of blending stages to the display device.
- blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises:
- 24. The method of clause 23, further comprising:
- receiving, by the layer mixer circuit, a memory access pipeline identifier and a blending stage indication for the first memory access pipeline circuit into a layer mixer configuration queue of a plurality of layer mixer configuration queues;
- assigning, by the layer mixer circuit, a tag corresponding to the layer mixer configuration queue to the first memory access pipeline circuit based on the memory access pipeline identifier; and
- responsive to receiving the intermediate preblend image data stream from the first memory access pipeline circuit, obtaining, by the layer mixer circuit, the memory access pipeline identifier and the blending stage indication from a top entry of the layer mixer configuration queue based on the tag;
- wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream within the blending stage of the plurality of blending stages is based on the blending stage indication.
- 1. A display processing unit, comprising:
Claims
1. A display processing unit, comprising:
- a plurality of memory access pipeline circuits;
- a layer mixer circuit;
- a first memory access pipeline circuit of the plurality of memory access pipeline circuits configured to, for each non-overlapping layer image of a plurality of non-overlapping layer images: obtain image configuration data for the non-overlapping layer image; fetch the non-overlapping layer image from an image data storage device based on the image configuration data; and output each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data; and
- the layer mixer circuit configured to: blend the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream; and output the display data stream to a display device.
2. The display processing unit of claim 1, wherein the layer mixer circuit is further configured to:
- receive the intermediate preblend image data stream from the first memory access pipeline circuit; and
- receive the background layer image data stream from a second memory access pipeline circuit of the plurality of memory access pipeline circuits.
3. The display processing unit of claim 1, wherein the image configuration data for each non-overlapping layer image of the plurality of the non-overlapping layer images comprises one or more of a source image size, an output image size, a set of source image position coordinates, a set of output image position coordinates, a source address, a stride value, an image format, and an unpack pattern value.
4. The display processing unit of claim 1, wherein:
- the first memory access pipeline circuit further comprises an image configuration queue;
- the first memory access pipeline circuit is further configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, receive, from a plurality of image configuration registers, the image configuration data into the image configuration queue; and
- the first memory access pipeline circuit is configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, obtain the image configuration data from a top entry of the image configuration queue.
5. The display processing unit of claim 4, wherein:
- the first memory access pipeline circuit is further configured to: calculate a set of position coordinates and a size of the intermediate preblend image, based on the image configuration data for the plurality of non-overlapping layer images; and transmit the set of position coordinates and the size of the intermediate preblend image to the layer mixer circuit; and
- the layer mixer circuit configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream based on the set of position coordinates and the size of the intermediate preblend image.
6. The display processing unit of claim 4, wherein:
- the first memory access pipeline circuit comprises a first selectable image configuration queue and a second selectable image configuration queue;
- the first memory access pipeline circuit is further configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, receive, from a plurality of image configuration registers, the image configuration data into the first selectable image configuration queue;
- the first memory access pipeline circuit is configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, obtain the image configuration data from a top entry of the first selectable image configuration queue; and
- the first memory access pipeline circuit is further configured to, in parallel with obtaining the image configuration data from the top entry of the first selectable image configuration queue, for each non-overlapping layer image of a next plurality of non-overlapping layer images, receive, from the plurality of image configuration registers, next image configuration data into the second selectable image configuration queue.
7. The display processing unit of claim 1, wherein the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to, for each pixel of the intermediate preblend image data stream that corresponds to a non-overlapping layer image of the plurality of non-overlapping layer images, output, to the display data stream, a blend of the pixel with a corresponding pixel of the background layer image.
8. The display processing unit of claim 7, wherein:
- the first memory access pipeline circuit is further configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, output, to the intermediate preblend image data stream, a blend bypass pixel for each pixel of the intermediate preblend image that does not correspond to a pixel of the non-overlapping layer image; and
- the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to, for each blend bypass pixel of the intermediate preblend image data stream, output, to the display data stream, a corresponding pixel of the background layer image.
9. The display processing unit of claim 8, wherein the blend bypass pixel comprises a pixel having a pre-specified color value.
10. The display processing unit of claim 2, wherein:
- the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to: blend the intermediate preblend image data stream and the background layer image data stream as the display data stream within a blending stage of a plurality of blending stages, wherein each blending stage corresponds to a memory access pipeline circuit of the plurality of memory access pipeline circuits; and output the display data stream of each blending stage preceding a final blending stage of the plurality of blending stages as the background layer image data stream to a successive blending stage; and
- the layer mixer circuit is configured to output the display data stream to the display device by being configured to output the display data stream of the final blending stage of the plurality of blending stages to the display device.
11. The display processing unit of claim 10, wherein:
- the layer mixer circuit further comprises a plurality of layer mixer configuration queues;
- the layer mixer circuit is further configured to: receive a memory access pipeline identifier and a blending stage indication for the first memory access pipeline circuit into a layer mixer configuration queue of the plurality of layer mixer configuration queues; assign a tag corresponding to the layer mixer configuration queue to the first memory access pipeline circuit based on the memory access pipeline identifier; and responsive to receiving the intermediate preblend image data stream from the first memory access pipeline circuit, obtain the memory access pipeline identifier and the blending stage indication from a top entry of the layer mixer configuration queue based on the tag; and
- the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream within the blending stage of the plurality of blending stages based on the blending stage indication.
12. The display processing unit of claim 1, integrated into a device selected from the group consisting of: a set top box; an entertainment unit; a navigation device; a communications device; a fixed location data unit; a mobile location data unit; a global positioning system (GPS) device; a mobile phone; a cellular phone; a smart phone; a session initiation protocol (SIP) phone; a tablet; a phablet; a server; a computer; a portable computer; a mobile computing device; a wearable computing device; a desktop computer; a personal digital assistant (PDA); a monitor; a computer monitor; a television; a tuner; a radio; a satellite radio; a music player; a digital music player; a portable music player; a digital video player; a video player; a digital video disc (DVD) player; a portable digital video player; an automobile; a vehicle component; avionics systems; a drone; and a multicopter.
13. A display processing unit, comprising:
- means for, for each non-overlapping layer image of a plurality of non-overlapping layer images: obtaining image configuration data for the non-overlapping layer image; fetching the non-overlapping layer image from an image data storage device based on the image configuration data; and outputting each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data;
- means for blending the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream; and
- means for outputting the display data stream to a display device.
14. A method for efficiently processing multiple non-overlapping layer images, the method comprising:
- for each non-overlapping layer image of a plurality of non-overlapping layer images: obtaining, by a first memory access pipeline circuit of a plurality of memory access pipeline circuits of a display processing unit, image configuration data for the non-overlapping layer image; fetching, by the first memory access pipeline circuit, the non-overlapping layer image from an image data storage device based on the image configuration data; and outputting, by the first memory access pipeline circuit, each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data;
- blending, by a layer mixer circuit of the display processing unit, the intermediate preblend image data stream and a background layer image data stream as a display data stream; and
- outputting, by the layer mixer circuit, the display data stream to a display device.
15. The method of claim 14, further comprising:
- receiving, by the layer mixer circuit, the intermediate preblend image data stream from the first memory access pipeline circuit; and
- receiving, by the layer mixer circuit, the background layer image data stream comprising a background layer image from a second memory access pipeline circuit of the plurality of memory access pipeline circuits.
16. The method of claim 14, wherein the image configuration data for each non-overlapping layer image of the plurality of the non-overlapping layer images comprises one or more of a source image size, an output image size, a set of source image position coordinates, a set of output image position coordinates, a source address, a stride value, an image format, and an unpack pattern value.
17. The method of claim 14, further comprising, for each non-overlapping layer image of the plurality of non-overlapping layer images, receiving, by the first memory access pipeline circuit from a plurality of image configuration registers, the image configuration data into an image configuration queue of the first memory access pipeline circuit;
- wherein, for each non-overlapping layer image of the plurality of non-overlapping layer images, obtaining the image configuration data for the non-overlapping layer image comprises obtaining the image configuration data from a top entry of the image configuration queue.
18. The method of claim 17, further comprising:
- calculating, by the first memory access pipeline circuit, a set of position coordinates and a size of the intermediate preblend image, based on the image configuration data for the plurality of non-overlapping layer images; and
- transmitting, by the first memory access pipeline circuit, the set of position coordinates and the size of the intermediate preblend image to the layer mixer circuit;
- wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream is based on the set of position coordinates and the size of the intermediate preblend image.
19. The method of claim 17, further comprising, for each non-overlapping layer image of the plurality of non-overlapping layer images, receiving, by the first memory access pipeline circuit from a plurality of image configuration registers, the image configuration data into a first selectable image configuration queue;
- wherein: for each non-overlapping layer image of the plurality of non-overlapping layer images, obtaining the image configuration data comprises obtaining the image configuration data from a top entry of the first selectable image configuration queue; and the method further comprises, in parallel with obtaining the image configuration data from the top entry of the first selectable image configuration queue, for each non-overlapping layer image of a next plurality of non-overlapping layer images, receiving, by the first memory access pipeline circuit from the plurality of image configuration registers, next image configuration data into a second selectable image configuration queue.
20. The method of claim 15, wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises, for each pixel of the intermediate preblend image data stream that corresponds to a non-overlapping layer image of the plurality of non-overlapping layer images, outputting, by the layer mixer circuit to the display data stream, a blend of the pixel with a corresponding pixel of the background layer image.
21. The method of claim 15, further comprising, for each non-overlapping layer image of the plurality of non-overlapping layer images, outputting, by the first memory access pipeline circuit to the intermediate preblend image data stream, a blend bypass pixel for each pixel of the intermediate preblend image that does not correspond to a pixel of the non-overlapping layer image;
- wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises, for each blend bypass pixel of the intermediate preblend image data stream, outputting, by the layer mixer circuit to the display data stream, a corresponding pixel of the background layer image.
22. The method of claim 21, wherein the blend bypass pixel comprises a pixel having a pre-specified color value.
23. The method of claim 15, wherein:
- blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises: blending, by the layer mixer circuit, the intermediate preblend image data stream and the background layer image data stream as the display data stream within a blending stage of a plurality of blending stages, wherein each blending stage corresponds to a memory access pipeline circuit of the plurality of memory access pipeline circuits; and outputting, by the layer mixer circuit, the display data stream of each blending stage preceding a final blending stage of the plurality of blending stages as the background layer image data stream to a successive blending stage; and
- outputting the display data stream to the display device comprises outputting, by the layer mixer circuit, the display data stream of the final blending stage of the plurality of blending stages to the display device.
24. The method of claim 23, further comprising:
- receiving, by the layer mixer circuit, a memory access pipeline identifier and a blending stage indication for the first memory access pipeline circuit into a layer mixer configuration queue of a plurality of layer mixer configuration queues;
- assigning, by the layer mixer circuit, a tag corresponding to the layer mixer configuration queue to the first memory access pipeline circuit based on the memory access pipeline identifier; and
- responsive to receiving the intermediate preblend image data stream from the first memory access pipeline circuit, obtaining, by the layer mixer circuit, the memory access pipeline identifier and the blending stage indication from a top entry of the layer mixer configuration queue based on the tag;
- wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream within the blending stage of the plurality of blending stages is based on the blending stage indication.
Type: Application
Filed: Dec 14, 2022
Publication Date: Jun 20, 2024
Inventors: Sreekanth Modaikkal (Bangalore), Kumar Saurabh (Bangalore), Gary Arthur Ciambella (Newmarket), Chun Wang (Markham), Anitha Madugiri Siddaraju (Bangalore)
Application Number: 18/066,034