Abstract: Vector-vector multiplication or matrix-matrix multiplication computation on computing systems can include computing a first portion of a vector-vector multiplication product based on a most-significant-bit set of a first vector and a most-significant-bit set of a second vector, and determining if the first portion of the vector-vector multiplication product is less than a threshold. If the first partial vector-vector multiplication product is not less than the threshold, a remaining portion of the vector-vector multiplication product can be computed, and a rectified linear vector-vector multiplication product can be determined for the sum of the first portion of the vector-vector multiplication product and the remaining portion of the vector-vector multiplication product.

Abstract: A system for using decoder information in video super resolution processing. A compressed video buffering module is used for receiving a compressed video stream and a decoder module is used for decoding the compressed video stream into an uncompressed stream and extracting motion vector information from the uncompressed stream. A video super resolution deep neural network processor module is used for processing the uncompressed stream in conjunction with the motion vector information to produce a video super resolution stream. An output buffer module is used for buffering the video super resolution stream for subsequent output.

Abstract: A method and apparatus for characteristic-based video processing include: in response to receiving a region of a picture of a video sequence, determining a characteristic in the region, the region being independent of other regions of the picture for video coding; determining a class associated with the region based on the characteristic, the class being selected from a plurality of classes; and encoding the region using a parameter set associated with the class, the parameter set being selected from a plurality of parameter sets for video coding at different quality levels.

Abstract: The present disclosure relates to a method and an apparatus for representation of a sparse matrix in a neural network. In some embodiments, an exemplary operation unit includes a buffer for storing a representation of a sparse matrix in a neural network, a sparse engine communicatively coupled with the buffer, and a processing array communicatively coupled with the sparse engine. The sparse engine includes circuitry to: read the representation of the sparse matrix from the buffer, the representation comprising a first level bitmap, a second level bitmap, and an element array; decompress the first level bitmap to determine whether a block of the sparse matrix comprises a non-zero element; and in response to the block comprising a non-zero element, decompress the second level bitmap using the element array to obtain the block of the sparse matrix. The processing array includes circuitry to execute the neural network with the sparse matrix.

Abstract: An innovative low-bit-width device may include a first digital-to-analog converter (DAC), a second DAC, a plurality of non-volatile memory (NVM) weight arrays, one or more analog-to-digital converters (ADCs), and a neural circuit. The first DAC is configured to convert a digital input signal into an analog input signal. The second DAC is configured to convert a digital previous hidden state (PHS) signal into an analog PHS signal. NVM weight arrays are configured to compute vector matrix multiplication (VMM) arrays based on the analog input signal and the analog PHS signal. The NVM weight arrays are coupled to the first DAC and the second DAC. The one or more ADCs are coupled to the plurality of NVM weight arrays and are configured to convert the VMM arrays into digital VMM values. The neural circuit is configured to process the digital VMM values into a new hidden state.

Abstract: Methods and systems are provided for implementing static channel filtering operations upon image datasets transformed to frequency domain representations, including decoding images of an image dataset to generate a frequency domain representation of the image dataset; discarding coefficient values of one or more particular frequency channels of each image of the image dataset in a frequency domain representation; and transporting the image dataset in a frequency domain representation to one or more special-purpose processor(s). Methods and systems of the present disclosure may enable a filtered image dataset to be input to a second layer of a learning model, bypassing a first layer, or may enable a learning model to be designed with a reduced-size first layer. This may achieve benefits such as reducing computational overhead and time of machine learning training and inference computations, reducing volume of image data input into the learning model, and reducing convergence time.

Abstract: The present disclosure provides methods, systems, and non-transitory computer readable media for performing inference with a neural network. The systems include one or more processing units configured to instantiate a neural network comprising a bypass switch that is associated with at least two bypass networks, wherein each of the at least two bypass networks have at least one hidden layer, the bypass switch is configured to select a bypass network of the at least two bypass networks to activate, and any non-selected bypass network of the at least two bypass networks is not activated.

Abstract: Methods and systems are provided for implementing preprocessing operations and augmentation operations upon image datasets transformed to frequency domain representations, including decoding images of an image dataset to generate a frequency domain representation of the image dataset; performing a resizing operation based on resizing factors on the image dataset in a frequency domain representation; performing a reshaping operation based on reshaping factors on the image dataset in a frequency domain representation; and performing a cropping operation on the image dataset in a frequency domain representation. The methods and systems may further include performing an augmentation operation on the image dataset in a frequency domain representation. Methods and systems of the present disclosure may free learning models from computational overhead caused by transforming image datasets into frequency domain representations.

Abstract: Discrete cosine transformation (DCT) information can be estimated from adjacent blocks of the same frame. DCT information can be estimated from different frames. Motion vectors can be used to track the position of objects in some frames of the video. For example, a stream of encoded frames is received; the encoded frames are entropy decoded and dequantized to produce DCT information for blocks of the frames; and DCT information for a block in a frame is determined using the DCT information produced from the entropy decoding and dequantizing for a different block.

Abstract: A system for determining the importance of encoded image components for artificial intelligence tasks includes an image capture or storage unit, a processor and a communication interface. The processor can receive components of transformed domain image data from the one or more image capture or storage units across the communication interface. The processor can be configured to determine the relative importance of the components of the transformed domain image data for an artificial intelligence task.

Abstract: Image data is accessed. The image data includes frequency domain components. A subset of the frequency domain components is selected based on the relative importance of the frequency domain components. Only the subset of the frequency domain components is provided to an accelerator that executes a neural network to perform an artificial intelligence task using the subset of frequency domain components.

Abstract: A system for processing encoded image components for artificial intelligence tasks. The system can include one or more compute units, one or more controllers and memory. The one or more controllers can include one or more micro-op schedulers and one or more channel switches. The one or more compute units can be configured to process components of the transformed domain image data according to one or more micro-operations for an artificial intelligence task. The one or more channel switches can be configured to selectively control the transfer of the components of transformed domain image data to the one or more compute units based on one or more gating flags. The one or more channel switches can also be configured to selectively control generation of the one or more micro-operations by the one or more micro-op schedulers based on the one or more gating flags.

Abstract: Methods and systems are provided for implementing training of learning models, including obtaining a pre-trained weight set for a learning model on a sample dataset and on a first loss function; selecting at least two tasks having heterogeneous features to be computed by a reference model; obtaining a reference dataset for the at least two tasks; designating a second loss function for feature embedding between the heterogeneous features of the at least two tasks; training the learning model on the first loss function and training the reference model on the second loss function, in turn; and updating the weight set based on a feature embedding learned by the learning model and a feature embedding learned by the reference model, in turn. Methods and systems of the present disclosure may alleviate computational overhead incurred by executing the learning model and loading different weight sets at a central network of the model.

Abstract: The systems and methods are configured to efficiently and effectively determine or find an estimated optimal encoding parameter set. In one embodiment, a video encoding parameter set estimation method comprises: performing an offline encoding parameter set characteristic prediction process that determines an estimate of a candidate encoding parameter set characteristic; and performing an encoding parameter set search process that identifies a predicted or estimated optimal video encoding parameter set. The encoding parameter set search process can include applying a constraint to the candidate encoding parameter set characteristic; and determining if candidate encoding parameter set meets an objective, wherein the determining is performed if the constraint is satisfied. The candidate encoding parameter set characteristic can be an estimated encoding time of the candidate encoding parameter set. The objective can be the best video quality out of a plurality of candidate encoding parameter sets.

Abstract: Systems and methods for reducing the impact of defects within a crossbar memory array when performing multiplication operations in which multiple control lines are concurrently selected are described. A group of memory cells within the crossbar memory array may be controlled by a local word line that is controlled by a local word line gating unit that may be configured to prevent the local word line from being biased to a selected word line voltage during an operation; the local word line may instead be set to a disabling voltage during the operation such that the memory cell currents through the group of memory cells are eliminated. If a defect has caused a short within one of the memory cells of the group of memory cells, then the local word line gating unit may be programmed to hold the local word line at the disabling voltage during multiplication operations.

Abstract: Systems and methods for reducing the impact of defects within a crossbar memory array when performing multiplication operations in which multiple control lines are concurrently selected are described. A group of memory cells within the crossbar memory array may be controlled by a local word line that is controlled by a local word line gating unit that may be configured to prevent the local word line from being biased to a selected word line voltage during an operation; the local word line may instead be set to a disabling voltage during the operation such that the memory cell currents through the group of memory cells are eliminated. If a defect has caused a short within one of the memory cells of the group of memory cells, then the local word line gating unit may be programmed to hold the local word line at the disabling voltage during multiplication operations.

Abstract: The present disclosure provides methods and systems for executing a neural network. The method includes: receiving a plurality of input vectors of input data; generating, among the plurality of input vectors, an estimation value associated with a subset of an input vector based on a weight vector of the activation function; determining whether the estimation value associated with the subset of the input vector satisfies a threshold condition; and determining an output of the activation function based on the estimation value.

Abstract: AI-assisted programmable hardware video codec is disclosed. According to certain embodiments, a video processing apparatus includes a programmable hardware encoder configured to execute an encoding process on a plurality of input video frames. The video processing apparatus further includes a controller coupled with the programmable hardware encoder. The controller is configured to execute a set of instructions to cause the video processing apparatus to: determine first information of the plurality of input video frames, and adjust the encoding process based on the first information.

Abstract: Systems and methods are disclosed for decoding data. A first block of data may be obtained from a storage medium or received from a computing device. The first block of data includes a first codeword generated based on an error correction code. A first set of likelihood values is obtained from a neural network. The first set of likelihood values indicates probabilities that the first codeword will be decoded into one of a plurality of decoded values. A second set of likelihood values is obtained from a decoder based on the first block of data. The second set of likelihood values indicates probabilities that the first codeword will be decoded into one of the plurality of decoded values. The first codeword is decoded to obtain a decoded value based on the first set of likelihood values and the second set of likelihood values.

Abstract: Systems and methods for dynamically encoding and decoding binary numbers using linear-time algorithms that encode and decode Hamming Distance-Based representations for the binary numbers are described. The binary numbers may correspond with integer values, such as 64-bit, 128-bit, or 256-bit integer values. In some cases, in response to detecting that a binary number is to be stored using a particular type of memory (e.g., a phase change memory), the binary number may first be encoded using a Hamming Distance-Based representation and then the encoded data may be written to the particular type of memory. The binary number may be encoded by generating a binary string or a binary array representing the binary number such that if one bit flips within the binary string or the binary array, the maximum distortion in the number is less than a threshold amount (e.g., less than 256).