Abstract: A pixel feature vector extraction system for extracting multi-scale features contains a cellular neural networks (CNN) based integrated circuit (IC) for extracting pixel feature vector out of input imagery data by performing convolution operations using pre-trained filter coefficients of ordered convolutional layers in a deep learning model. The ordered convolutional layers are organized in a number of groups with each group followed by a pooling layer. Each group is configured for a different size of feature map. Pixel feature vector contains a combination of feature maps from at least two groups, for example, concatenation of the feature maps. The first group of the at least two groups contains the largest size of the feature maps amongst all of the at least two groups. Feature maps of the remaining of the at least two groups are modified to match the size of the feature map of the first group.

Abstract: A system for training an artificial intelligence (AI) model for an AI chip to implement an AI task may include an AI training unit to train weights of an AI model in floating point, a convolution quantization unit for quantizing the trained weights to a number of quantization levels, and an activation quantization unit for updating the weights of the AI model so that output of the AI model based at least on the updated weights are within a range of activation layers of the AI chip. The updated weights may be stored in fixed point and uploadable to the AI chip. The various units may be configured to account for the hardware constraints in the AI chip to minimize performance degradation when the trained weights are uploaded to the AI chip and expedite training convergence. Forward propagation and backward propagation may be combined in training the AI model.

Abstract: A system may include multiple client devices and a processing device communicatively coupled to the client devices. Each client device includes an artificial intelligence (AI) chip and is configured to generate an AI model. The processing device may be configured to (i) receive a respective AI model and an associated performance value of the respective AI model from each of the plurality of client devices; (ii) determine an optimal AI model based on the performance values associated with the respective AI models from the plurality of client devices; and (iii) determine a global AI model based on the optimal AI model. The system may load the global AI model into an AI chip of a client device to cause the client device to perform an AI task based on the global AI model in the AI chip. The AI model may include a convolutional neural network.

Abstract: A device for obtaining a local optimal AI model may include an artificial intelligence (AI) chip and a processing device configured to receive a first initial AI model from the host device. The device may load the initial AI model into the AI chip to determine a performance value of the AI model based on a dataset, and determine a probability that a current AI model should be replaced by the initial AI model. The device may determine, based on the probability, whether to replace the current AI model with the initial AI model. If it is determined that the current AI model be replaced, the device may replace the current AI model with the initial AI model. The device may repeat the above processes and obtain a final current AI model. The device may transmit the final current AI model to the host device.

Abstract: P feature encoding values are obtained for each of the Q frames in a video clip by image transformations of each frame along with performing computations of a specific succession of convolution and pooling layers of a CNN based deep learning model followed with operations of a nested invariance pooling layer. Each feature encoding value is then converted from real number to a corresponding integer value within a range designated for color display intensity according to a quantization scheme. A 2-D graphical symbol that contains N×N pixels is formed by placing respective color display intensities into the N×N pixels according to a data arrangement pattern for representing all frames of the video clip in form of P×Q feature encoding values, such that the 2-D graphical symbol possesses a semantic meaning of the video clip that can be recognized via image classification task using another trained CNN based deep learning model.

Abstract: A system may include a decentralized communication network and multiple processing devices on the network. Each processing device may have an artificial intelligence (AI) chip, the device may be configured to generate an AI model, determine the performance value of the AI model on the AI chip, receive a chain from the network where the chain contains a performance measure. If the performance value of the AI model is better than the performance measure, then the processing device may broadcast the AI model to the network for verification. If the AI model is verified by the network, the device may update the chain with the performance value so that the chain can be shared by the multiple processing devices on the network. Any processing device on the network may also verify an AI model broadcasted by any other device. Methods for generating the AI model are also provided.

Abstract: A system may include a decentralized communication network and multiple processing devices on the network. Each processing device may have an artificial intelligence (AI) chip, the device may be configured to generate an AI model, determine the performance value of the AI model on the AI chip, receive a chain from the network where the chain contains a performance measure. If the performance value of the AI model is better than the performance measure, then the processing device may broadcast the AI model to the network for verification. If the AI model is verified by the network, the device may update the chain with the performance value so that the chain can be shared by the multiple processing devices on the network. Any processing device on the network may also verify an AI model broadcasted by any other device. Methods for generating the AI model are also provided.

Abstract: An integrated circuit may include multiple cellular neural networks (CNN) processing engines coupled in a loop circuit and configured to perform an AI task. Each CNN processing engine includes multiple convolution layers, a first memory buffer to store imagery data and a second memory buffer to store filter coefficients. The CNN processing engines are configured to perform convolution operations over an input image simultaneously in one or more iterations. In each iteration, various sub-images of the input image are loaded to the first memory buffer circularly. A portion of the filter coefficients corresponding to the sub-image are loaded to the second memory buffer in a cyclic order. Data may be arranged in the second memory buffer to facilitate loading of duplicate filter coefficients among at least two convolution layers without requiring duplicate memory space. Methods of training a CNN model having duplicate weights are also provided.

Abstract: An integrated circuit may include multiple cellular neural networks (CNN) processing engines coupled to at least one input/output data bus and a clock-skew circuit in a loop circuit. Each CNN processing engine includes multiple convolution layers, a first memory buffer to store imagery data and a second memory buffer to store filter coefficients. Each of the CNN processing engines is configured to perform convolution operations over an input image simultaneously in a first clock cycle to generate output to be fed to an immediate neighbor CNN processing engine for performing convolution operations in a next clock cycle. The second memory buffer may store a first subset of filter coefficients for a first convolution layer of the CNN processing engine and store a reference location to the first subset of filter coefficients for a second convolution layer, where the filter coefficients for the first and second convolution layers are duplicate.

Abstract: A pixel feature vector extraction system for extracting multi-scale features contains a cellular neural networks (CNN) based integrated circuit (IC) for extracting pixel feature vector out of input imagery data by performing convolution operations using pre-trained filter coefficients of ordered convolutional layers in a deep learning model. The ordered convolutional layers are organized in a number of groups with each group followed by a pooling layer. Each group is configured for a different size of feature map. Pixel feature vector contains a combination of feature maps from at least two groups, for example, concatenation of the feature maps. The first group of the at least two groups contains the largest size of the feature maps amongst all of the at least two groups. Feature maps of the remaining of the at least two groups are modified to match the size of the feature map of the first group.

Abstract: P feature encoding values are obtained for each of the Q frames in a video clip by image transformations of each frame along with performing computations of a specific succession of convolution and pooling layers of a CNN based deep learning model followed with operations of a nested invariance pooling layer. Each feature encoding value is then converted from real number to a corresponding integer value within a range designated for color display intensity according to a quantization scheme. A 2-D graphical symbol that contains N×N pixels is formed by placing respective color display intensities into the N×N pixels according to a data arrangement pattern for representing all frames of the video clip in form of P×Q feature encoding values, such that the 2-D graphical symbol possesses a semantic meaning of the video clip that can be recognized via image classification task using another trained CNN based deep learning model.

Abstract: A system for training an artificial intelligence (AI) model for an AI chip may include a forward network and a backward propagation network. The AI model may be a convolution neural network (CNN). The forward network may infer the output of the AI chip based on the training data. The backward network may use the output of the AI chip and the ground truth data to train the weights of the AI model. In some examples, the system may train the AI model using a gradient descent method. The system may quantize the weights and update the weights during the training. In some examples, the system may perform a uniform quantization over the weights. The system may also determine the distribution of the weights. If the weight distribution is not symmetric, the system may group the weights and quantize the weights based on the grouping.

Abstract: A system for training an artificial intelligence (AI) model for an AI chip to implement an AI task may include an AI training unit to train weights of an AI model in floating point, a convolution quantization unit for quantizing the trained weights to a number of quantization levels, and an activation quantization unit for updating the weights of the AI model so that output of the AI model based at least on the updated weights are within a range of activation layers of the AI chip. The updated weights may be stored in fixed point and uploadable to the AI chip. The various units may be configured to account for the hardware constraints in the AI chip to minimize performance degradation when the trained weights are uploaded to the AI chip and expedite training convergence. Forward propagation and backward propagation may be combined in training the AI model.

Abstract: Methods and systems for using feature encoding for storing a video stream without redundant frames are disclosed. A video stream containing a plurality of frames is received in a computing system. Each frame is divided to one or more sub-frames with each sub-frame containing a resolution suitable as an input image to a deep learning model based on VGG-16 model, ResNet or MobilNet. Respective vectors of feature encoding values of all sub-frames of current and immediately prior frames are obtained by performing computations of the deep learning model. A difference metric between the current frame and the immediately prior frame is obtained by comparing the respective vectors using a difference measurement technique. The current frame is stored in a to-be-kept video file only when the difference metric indicates that the current frame and the immediately prior frame are different in accordance with a predefined criterion.

Abstract: In some examples, given an AI model in floating point, a system may use one or more artificial intelligence (AI) chips to train a global gain vector for use to convert the AI model in floating point to an AI model in fixed point for uploading to a physical AI chip. The system may determine initial gain vectors, and in each of multiple iterations, obtain the performance values of the AI chips based on the gain vectors and update the gam vectors for the next iteration. The gain vectors are updated based on a velocity of gain. The performance value may be based on feature maps of an AI model before and after the converting. The performance value may also be based on interference over a test dataset. Upon completion of the iterations, the system determines the global gain vector that resulted in the best performance value during the iterations.

Abstract: An image and the maximum number of tokens for a to-be-created image caption are received in a computing system. Font size of graphical image of the token is calculated from the maximum number of tokens and the dimension of desired input image for prediction-style image classification technique. Desired input image is divided into first and second portions. A 2-D symbol is formed by placing a resized image derived from the received image with substantially similar contents in the first portion and by initializing the second portion with blank images. Next token of the image caption is predicted by classifying the 2-D symbol using the prediction-style image classification technique. 2-D symbol is modified by appending the graphical image of just-predicted token to the existing image caption in the second portion, if termination condition for image caption creation is false. Next token is repeatedly predicted until termination condition becomes true.

Abstract: A list of keywords in a category of interest is defined and a list of to-be-excluded items is derived therefrom. A first set of general texts is obtained. A second set of texts is created by inserting or replacing a randomly selected item from the list of keywords into each of the first set at a randomly chosen location. A third set of texts is created by inserting or replacing a randomly selected item from the list of to-be-excluded into each of the first set at a randomly chosen location. First and second groups of 2-D symbols are formed to graphically represent the second set and the third set, respectively. The first group is associated with the category of interest while the second group is associated with the category of uninterested. Keyword detection training dataset is created by combining first and second groups of 2-D symbols.

Abstract: A system may be configured to obtain a global artificial intelligence (AI) model for uploading into an AI chip to perform AI tasks. The system may implement a training process including receiving updated AI models from one or more client devices, determining a global AI model based on the received AI models from the client devices, and updating initial AI models for the client devices. Each client device may receive an initial AI model and train an updated AI model by training the entire parameters of the AI model together, by training a subset of the parameters of the AI model in a layer by layer fashion, or by training a subset of the parameters by parameter types. Each client device may include one or more AI chips configured to run an AI task to measure performance of an AI model. The AI model may include a convolutional neural network.

Abstract: A system may include multiple client devices and a processing device communicatively coupled to the client devices. Each client device includes an artificial intelligence (AI) chip and is configured to generate an AI model. The processing device may be configured to (i) receive a respective AI model and an associated performance value of the respective AI model from each of the plurality of client devices; (ii) determine an optimal AI model based on the performance values associated with the respective AI models from the plurality of client devices; and (iii) determine a global AI model based on the optimal AI model. The system may load the global AI model into an AI chip of a client device to cause the client device to perform an AI task based on the global AI model in the AI chip. The AI model may include a convolutional neural network.

Abstract: A device for obtaining a local optimal AI model may include an artificial intelligence (AI) chip and a processing device configured to receive a first initial AI model from the host device. The device may load the initial AI model into the AI chip to determine a performance value of the AI model based on a dataset, and determine a probability that a current AI model should be replaced by the initial AI model. The device may determine, based on the probability, whether to replace the current AI model with the initial AI model. If it is determined that the current AI model be replaced, the device may replace the current AI model with the initial AI model. The device may repeat the above processes and obtain a final current AI model. The device may transmit the final current AI model to the host device.