Abstract: A method and system for performing general matrix-matrix multiplication (GEMM) operations on a graphics processor unit (GPU) using Smart kernels. During operation, the system may generate a set of kernels that includes at least one of a variable-dimension variable-K GEMM kernel, a variable-dimension constant-K GEMM kernel, or a combination thereof. A constant-K GEMM kernel performs computations for matrices with a specific value of K (e.g., the number of columns in a first matrix and the number of rows in a second matrix). Variable-dimension GEMM kernels allow for flexibility in the number of rows and columns used by a thread block to perform matrix multiplication for a sub-matrix. The system may generate rules to select the best (e.g., fastest) kernel for performing computations according to the particular parameter combination of the matrices being multiplied.