Abstract: According to one embodiment, a method, computer system, and computer program product for routing acceleration in mixture of experts ensembles is provided. The present invention may include receiving input data at a router; generating a plurality of output vectors by applying a routing function to the input data, wherein each output vector is associated with one or more respective tiles or pairs of tiles in a plurality of MVM tiles; determining a plurality of sub-vectors in the output vectors, wherein each sub-vector in the plurality of sub-vectors is associated with a respective output vector in the plurality of output vectors, and merging the sub-vectors into an element vector; generating a probability distribution vector by applying a Softmax function to the element vector and determining the largest elements of the probability distribution; and configuring the router based on the one or more largest elements of the probability distribution.

Abstract: A method for balancing utilization of tiles in an analog in-memory computing system includes identifying, by a computer processor, a plurality of tiles in the analog in-memory computing system. The computer processor receives a plurality of layers in a neural network being processed by the analog in-memory computing system. The computer processor maps the plurality of layers in the neural network to the plurality of tiles. The computer processor determines a number of operations for each of the tiles in the plurality of tiles. The computer processor determines an equalized utilization rate for the tiles in the plurality of tiles. In addition, the computer processor assigns the layers to the plurality of tiles. The tiles are assigned so that a first utilization rate of a first tile is balanced relative to a second utilization rate of a second tile in the analog in-memory computing system.

Abstract: An embodiment establishes a neural network that comprises a plurality of layers. The embodiment receives a plurality of input data sequences into a layer of the neural network, the plurality of input data sequences comprises a first input data sequence and a second input data sequence. The embodiment superposes the first input data sequence and the second input data sequence, thereby creating a superposed embedding. The embodiment transforms the superposed embedding by applying a function to the superposed embedding, thereby creating a transformed superposed embedding. The embodiment infers a first output data element corresponding to the first input data sequence and a second output data element corresponding to the second input data sequence via application of an unbinding operation on the transformed superposed embedding.

Abstract: Method and apparatus for deep learning. A first input and a second input are accessed. A first embedding for the first input is generated using a binding network. A second embedding for the second input is generated using the binding network. The first and second embeddings are aggregated to generate a combined embedding. A transformation function is applied to the combined embedding to generate a transformed combined embedding. The transformed combined embedding is processed, using an unbinding network, to extract a first transformed embedding for the first input and a second transformed embedding for the second input. An inference function is applied to the first transformed embedding to generate a first output. The inference function is applied to the second transformed embedding to generate a second output.

Abstract: An approach for bundling a set of hypervectors may be provided herein. The approach may involve encoding a data structure into a plurality of hypervectors. The approach may further involve calculating the element-wise sum of a set of hypervectors to generate a sum hypervector. A plurality of blocks may be produced from the sum hypervectors. The block elements of the sum hypervector may be selected based on a selection criterion. A selection criterion may include a threshold value or simply be the largest element per block. Additionally, the approach may involve setting the non-selected elements of the sum hypervector to zero.

Abstract: An approach for factorizing hypervectors using a resonator network may be provided herein. The approach may involve providing alternative implementations of a step for each step of the iterative process of the resonator network. An input hypervector representing a data structure may be received, by a resonator network. The approach may further involve selecting a step from the provided implementation of each step of the iterative process. The iterative process may be executed based on the selected implementations, thereby factorizing the input hypervector.

Abstract: The present disclosure relates to a method for classifying a query information element using the similarity between the query information element and a set of support information elements. A resulting set of similarity scores is transformed using a sharpening function such that the transformed scores are decreasing as negative similarity scores increase and the transformed scores are increasing as positive similarity scores increase. A class of the query information element is determined based on the transformed similarity scores.

Abstract: A computer-implemented method to cluster data on an in-memory computing (IMC) system. The method includes determining, by an IMC system, centroid coordinate vectors as column vectors of dimension M, the column vectors representing normalized coordinates of initial centroids of clusters of the set of N points. The method includes storing N point coordinate vectors of dimension M across the memory systems, wherein the N point coordinate vectors represent normalized coordinates of the set of N points and can be represented as an M×N matrix. The method includes refining the centroid coordinate vectors by determining dot products of the column vectors with the matrix to obtain intermediate vectors of dimension N, determining row vectors in accordance with maxima of each column, performing dot products of the row vectors with a transposed matrix as second vector-matrix multiplications to obtain column vectors, and averaging each of the column vectors.

Abstract: The present disclosure relates to a method for representing an ordered group of symbols with a hypervector. The method comprises sequentially applying on at least part of the input hypervector associated with a current symbol a predefined number of circular shift operations associated with the current symbol, resulting in a shifted hypervector. A rotate operation may be applied on the shifted hypervector, resulting in an output hypervector. If the current symbol is not the last symbol of the ordered group of symbols the output hypervector may be provided as the input hypervector associated with a subsequent symbol of the current symbol; otherwise, the output hypervector of the last symbol of the ordered group of symbols may be provided as a hypervector that represents the ordered group of symbols.

Abstract: The present disclosure relates to training a classifier. The classifier includes a controller and an explicit memory. The training may include iteratively receiving one or more second training datasets, each comprising second data samples of a set of one or more associated novel classes, adding to the explicit memory one or more second output vectors indicative of the set of one or more associated novel classes, in response to providing the one or more second training datasets to the classifier, retraining the classifier using the one or more second training datasets and the first training dataset by minimizing a distance between the one or more second output vectors and the one or more prototype vectors, determining a set of updated prototype vectors indicative of first training dataset and the one or more second training datasets, and updating the explicit memory with the set of updated prototype vectors.

Abstract: A composite vector is received. A first candidate component vector is generated and evaluated. The first candidate component vector is selected, based on the evaluating, as an accurate component vector. The first candidate component vector is unbundled from the composite vector. The unbundling results in a first reduced vector.

Abstract: A probability distribution corresponding to the kernel function is determined and weights are sampled from the determined probability distribution corresponding to the given kernel function. Memristive devices of an analog crossbar are programmed based on the sampled weights, where each memristive device of the analog crossbar is configured to represent a corresponding weight. Two matrix-vector multiplication operations are performed on an analog input x and an analog input y using the programmed crossbar and a dot product is computed on results of the matrix-vector multiplication operations.

Abstract: A 3D compute-in-memory accelerator system and method for efficient inference of Mixture of Expert (MoE) neural network models. The system includes a plurality of compute-in-memory cores, each in-memory core including multiple tiers of in-memory compute cells. One or more tiers of in-memory compute cells correspond to an expert sub-model of the MoE model. One or more expert sub-models are selected for activation propagation based on a function-based routing, the tiers of the corresponding experts being activated based on this function. In one embodiment, this function is a hash-based tier selection function used for dynamic routing of inputs and output activations. In embodiments, the function is applied to select a single expert or multiple experts with input data-based or with layer-activation-based MoEs for single tier activation. Further, the system is configured as a multi-model system with single expert model selection or with a multi-model system with multi-expert selection.

Abstract: The disclosure includes a computer-implemented method of factorizing a vector by utilizing resonator network modules. Such modules include an unbinding module, as well as search-in-superposition modules. The method includes the following steps. A product vector is fed to the unbinding module to obtain unbound vectors. The latter represent estimates of codevectors of the product vector. A first operation is performed on the unbound vectors to obtain quasi-orthogonal vectors. The first operation is reversible. The quasi-orthogonal vectors are fed to the search-in-superposition modules, which rely on a single codebook. In this way, transformed vectors are obtained, utilizing a single codebook. A second operation is performed on the transformed vectors. The second operation is an inverse operation of the first operation, which makes it possible to obtain refined estimates of the codevectors.

Abstract: A computerized neuro-vector-symbolic architecture, that: receives image data associated with an artificial intelligence (AI) task; processes the image data using a frontend that comprises an artificial neural network (ANN) and a vector-symbolic architecture (VSA); and processes an output of the frontend using a backend that comprises a symbolic logical reasoning engine, to solve the AI task. The AI task, for example, may be an abstract visual reasoning task.

Abstract: Embodiments are disclosed for a method. The method includes determining a granularity of hypervectors. The method also includes receiving an input hypervector representing a data structure. Additionally, the method includes performing an iterative process to factorize the input hypervector into individual hypervectors representing the cognitive concepts. The iterative process includes, for each concept: determining an unbound version of a hypervector representing the concept by a blockwise unbinding operation between the input hypervector and estimate hypervectors of other concepts. The iterative process further includes determining a similarity vector indicating a similarity of the unbound version of the hypervector with each candidate code hypervector of the concept. Additionally, the iterative process includes generating an estimate of a hypervector representing the concept by a linear combination of the candidate code hypervectors, and weights of the similarity vector.

Abstract: Embodiments are disclosed for a method. The method includes bundling a set of M code hypervectors, each of dimension D, where M>1. The bundling includes receiving an M-dimensional vector comprising weights for weighting the set of code hypervectors. The bundling further includes mapping the M-dimensional vector to an S-dimensional vector, sk, such that each element of the S-dimensional vector, sk, indicates one of the set of code hypervectors, where S=D/L and L?1. Additionally, the bundling includes building a hypervector such that an ith element of the built hypervector is an ith element of the code hypervector indicated in an ith element of the S-dimensional vector, sk.

Abstract: The present disclosure relates to a resonator network system comprising a set of resonator networks, each resonator network being configured to execute a resonator network, the resonator network being configured to receive an input hypervector representing a data structure and to perform an iterative process in order to factorize the input hypervector into individual hypervectors representing a set of concepts respectively, the set of N resonator networks being associated with N permutations respectively. The resonator network system being configured for applying the N permutations to N first hypervectors respectively, the N first hypervectors representing a set of N data structures respectively; and combining the N permuted hypervectors into a bundled hypervector. The resonator networks being configured for processing the bundled hypervector respectively, thereby factorizing the first hypervectors.

Abstract: Predefined concepts are represented by codebooks. Each codebook includes candidate code hypervectors that represent items of a respective concept of the predefined concepts. A neuromorphic memory device with a crossbar array structure includes row lines and column lines stores a value of respective code hypervectors of an codebook. An input hypervector is stored in an input buffer. A plurality of estimate buffers are each associated with a different subset of row lines and a different codebook and initially store estimated hypervectors. An unbound hypervector is computed using the input hypervector and all the estimated hypervectors. An attention vector is computed that indicates a similarity of the unbound hypervector with one estimated hypervector. A linear combination of the one estimated hypervector, weighted by the attention vector, is computed and is stored in the estimate buffer that is associated with the one estimated hypervector.

Abstract: A computer-implemented method for performing a classification of an input signal by a neural network includes: computing, by a feature extraction unit of the neural network, a D-dimensional query vector, wherein D is an integer; generating, by a classification unit of the neural network, a set of C fixed D-dimensional quasi-orthogonal bipolar vectors as a fixed classification matrix, wherein C is an integer corresponding to a number of classes of the classification unit; and performing a classification of a query vector based, at least in part, on the fixed classification matrix.