Abstract: The present invention describes a method of accelerating execution of one or more application tasks in a computing device using machine learning (ML) based model. According to one embodiment, a neural accelerating engine present in the computing device receives a ML input task for execution on the computing device from a user. The neural accelerating engine further retrieves a trained ML model and a corresponding optimal configuration file based on the received ML input task. Also, the current performance status of the computing device for executing the ML input task is obtained. Then, the neural accelerating engine dynamically schedules and dispatches parts of the ML input task to one or more processing units in the computing device for execution based on the retrieved optimal configuration file and the obtained current performance status of the computing device.

Abstract: A method of deep neural network (DNN) modularization for optimal loading includes receiving, by an electronic device, a DNN model for execution, obtaining, by the electronic device, a plurality of parameters associated with the electronic device and a plurality of parameters associated with the DNN model, determining, by the electronic device, a number of sub-models of the DNN model and a splitting index, based on the obtained plurality of parameters associated with the electronic device and the obtained plurality of parameters associated with the DNN model, and splitting, by the electronic device, the received DNN model into a plurality of sub-models, based on the determined number of sub-models of the DNN model and the determined splitting index.

Abstract: A method for generating an optimal neural network (NN) model may include determining intermediate outputs of the NN model by passing an input dataset through each intermediate exit gate of the plurality of intermediate exit gates, determining an accuracy score for each intermediate exit gate of the plurality of intermediate exit gates based on a comparison of the final output of the NN model with the intermediate output, identifying an earliest intermediate exit gate that produces the intermediate output closer to the final output based on the accuracy score, and generating the optimal NN model by removing remaining layers of the plurality of layers and remaining intermediate exit gates of the plurality of intermediate exit gates located after the determined earliest intermediate exit gate.

Abstract: Disclosed herein is a method and an optimization unit for optimizing and/or improving efficiency of resource utilization in an embedded computing system executing Artificial Intelligence (AI) applications. The method includes: detecting, by an optimization unit comprising processing circuitry and/or executable program instructions configured in the embedded computing system, a launch of an AI application on the embedded computing system; retrieving a runtime profile corresponding to the AI application, the runtime profile indicating resource requirements for executing the AI application; and configuring a runtime environment of the embedded computing system for the AI application based on the runtime profile corresponding to the AI application.

Abstract: Embodiments herein provide a NAS method of generating an optimized DNN model for executing a task in an electronic device. The method includes identifying the task to be executed in the electronic device. The method includes estimating a performance parameter to be achieved while executing the task. The method includes determining hardware parameters of the electronic device required to execute the task based on the performance parameter and the task, and determining optimal neural blocks from a plurality of neural blocks based on the performance parameter and the hardware parameter of the electronic device. The method includes generating the optimized DNN model for executing the task based on the optimal neural blocks, and executing the task using the optimized DNN model.

Abstract: A method, an apparatus, and a system for configuring a neural network across heterogeneous processors are provided. The method includes creating a unified neural network profile for the plurality of processors; receiving at least one request to perform at least one task using the neural network; determining a type of the requested at least one task as one of an asynchronous task and a synchronous task; and parallelizing processing of the neural network across the plurality of processors to perform the requested at least one task, based on the type of the requested at least one task and the created unified neural network profile.

Abstract: A method of processing a neural network model by using a plurality of processors includes allocating at least one slice to each layer from among a plurality of layers included in the neural network model, allocating each layer from among the plurality of layers to the plurality of processors based on respective processing times of the plurality of processors for processing each of the at least one slice, and processing the neural network model by using the plurality of processors based on a result of the allocation.

Abstract: The present invention describes a method of accelerating execution of one or more application tasks in a computing device using machine learning (ML) based model. According to one embodiment, a neural accelerating engine present in the computing device receives a ML input task for execution on the computing device from a user. The neural accelerating engine further retrieves a trained ML model and a corresponding optimal configuration file based on the received ML input task. Also, the current performance status of the computing device for executing the ML input task is obtained. Then, the neural accelerating engine dynamically schedules and dispatches parts of the ML input task to one or more processing units in the computing device for execution based on the retrieved optimal configuration file and the obtained current performance status of the computing device.