Abstract: The present disclosure relates to methods and devices for graphics processing including an apparatus, e.g., a GPU. The apparatus may modify at least one texture memory object to support a data structure for one or more tensor objects. The apparatus may also determine one or more supported memory layouts for the one or more tensor objects based on the modified at least one texture memory object. Additionally, the apparatus may access data associated with the one or more tensor objects based on the one or more supported memory layouts, the data for each of the one or more tensor objects corresponding to at least one data instruction. The apparatus may also execute the at least one data instruction based on the accessed data associated with the one or more tensor objects.

Abstract: The present disclosure relates to methods and devices for graphics processing including an apparatus, e.g., a GPU. The apparatus may modify at least one texture memory object to support a data structure for one or more tensor objects. The apparatus may also determine one or more supported memory layouts for the one or more tensor objects based on the modified at least one texture memory object. Additionally, the apparatus may access data associated with the one or more tensor objects based on the one or more supported memory layouts, the data for each of the one or more tensor objects corresponding to at least one data instruction. The apparatus may also execute the at least one data instruction based on the accessed data associated with the one or more tensor objects.

Abstract: The present disclosure relates to methods and apparatus for machine learning processing. For example, disclosed techniques facilitate improving execution of machine learning primitives. Aspects of the present disclosure may store a command stream generated by an application in a buffer, the command stream including a plurality of machine learning primitives for execution by a graphics processor. Further, aspects of the present disclosure identify, after receiving a request from the application to finalize the buffer, two or more machine learning primitives of the buffer that may be replaced with a fused shader kernel. Additionally, aspects of the present disclosure may store the fused shader kernel in the buffer to generate a fused command buffer.

Abstract: The present disclosure relates to methods and apparatus for selecting a sequence of shaders for performing a machine-learning operation on a graphics processing unit (GPU). The apparatus can receive a request to perform a machine-learning operation. The apparatus can determine a plurality of sequences of shaders that are capable of performing the machine-learning operation. The apparatus can determine a cost for each sequence of the plurality of sequences of shaders based on a cost function associated with each shader. The apparatus can execute a selected sequence of shaders of the plurality of sequences of shaders having a lowest cost.

Abstract: Methods, systems, and devices for processing are described. A device may parse a set of layers of a deep neural network. The set of layers may be associated with a set of machine learning operations of the deep neural network. The device may determine one or more layer parameters based on the determined set of layers. In some aspects, the device may determine an execution time associated with executing a shader dispatch based on the one or more layer parameters. The device may batch the shader dispatch to a command buffer based on the execution time and process the command buffer based on the batching. The device may determine a target execution time based on an assembly time associated with the command buffer, a processing time associated with the command buffer, a frequency level associated with processing the command buffer, the one or more layer parameters, or some combination thereof.

Abstract: The present disclosure relates to methods and apparatus for machine learning processing. For example, disclosed techniques facilitate tile-based GPU machine learning acceleration. Aspects of the present disclosure can determine a tile size based on a memory size of a first memory and a job input size associated with executing a computational job. In some examples, the computational job may be one of a quantity of computational jobs configured to execute a machine learning primitive. Aspects of the present disclosure can also load, based on the tile size, input data associated with a batch of computational jobs from a second memory to the first memory. Further, aspects of the present disclosure can generate batch output data by executing the batch of computational jobs using the input data loaded to the first memory. Additionally, aspects of the present disclosure can store the generated batch output data to the second memory.

Abstract: Methods, systems, and devices for processing are described. A device may parse a set of layers of a deep neural network. The set of layers may be associated with a set of machine learning operations of the deep neural network. The device may determine one or more layer parameters based on the determined set of layers. In some aspects, the device may determine an execution time associated with executing a shader dispatch based on the one or more layer parameters. The device may batch the shader dispatch to a command buffer based on the execution time and process the command buffer based on the batching. The device may determine a target execution time based on an assembly time associated with the command buffer, a processing time associated with the command buffer, a frequency level associated with processing the command buffer, the one or more layer parameters, or some combination thereof.

Abstract: The present disclosure relates to methods and apparatus for machine learning processing. For example, disclosed techniques facilitate improving execution of machine learning primitives. Aspects of the present disclosure may store a command stream generated by an application in a buffer, the command stream including a plurality of machine learning primitives for execution by a graphics processor. Further, aspects of the present disclosure identify, after receiving a request from the application to finalize the buffer, two or more machine learning primitives of the buffer that may be replaced with a fused shader kernel. Additionally, aspects of the present disclosure may store the fused shader kernel in the buffer to generate a fused command buffer.