Abstract: A device, system, and method is provided for storing a sparse neural network. A plurality of weights of the sparse neural network may be obtained. Each weight may represent a unique connection between a pair of a plurality of artificial neurons in different layers of a plurality of neuron layers. A minority of pairs of neurons in adjacent neuron layers are connected in the sparse neural network. Each of the plurality of weights of the sparse neural network may be stored with an association to a unique index. The unique index may uniquely identify a pair of artificial neurons that have a connection represented by the weight. Only non-zero weights may be stored that represent connections between pairs of neurons (and zero weights may not be stored that represent no connections between pairs of neurons).

Abstract: A device, system, and method is provided for storing a sparse neural network. A plurality of weights of the sparse neural network may be obtained. Each weight may represent a unique connection between a pair of a plurality of artificial neurons in different layers of a plurality of neuron layers. A minority of pairs of neurons in adjacent neuron layers are connected in the sparse neural network. Each of the plurality of weights of the sparse neural network may be stored with an association to a unique index. The unique index may uniquely identify a pair of artificial neurons that have a connection represented by the weight. Only non-zero weights may be stored that represent connections between pairs of neurons (and zero weights may not be stored that represent no connections between pairs of neurons).

Abstract: A device, system, and method is provided for training or prediction using a cluster-connected neural network. The cluster-connected neural network may be divided into a plurality of clusters of artificial neurons connected by weights or convolutional channels connected by convolutional filters. Within each cluster is a locally dense sub-network of intra-cluster weights or filters with a majority of pairs of neurons or channels connected by intra-cluster weights or filters that are co-activated together as an activation block during training or prediction. Outside each cluster is a globally sparse network of inter-cluster weights or filters with a minority of pairs of neurons or channels separated by a cluster border across different clusters connected by inter-cluster weights or filters. Training or predicting is performed using the cluster-connected neural network.

Abstract: A device, system, and method is provided for training a new neural network to mimic a target neural network without access to the target neural network or its original training dataset. The target neural network and the new neural network may be probed with input data to generate corresponding target and new output data. Input data may be detected that generate a maximum or above threshold difference between the corresponding target and new output data. A divergent probe training dataset may be generated comprising the input data that generate the maximum or above threshold difference and the corresponding target output data. The new neural network may be trained using the divergent probe training dataset to generate the target output data. The new neural network may be iteratively trained using an updated divergent probe training dataset dynamically adjusted as the new neural network changes during training.

Abstract: A device, system, and method for training or prediction of a neural network. A current value may be stored for each of a plurality of synapses or filters in the neural network. A historical metric of activity may be independently determined for each individual or group of the synapses or filters during one or more past iterations. A plurality of partial activations of the neural network may be iteratively executed. Each partial-activation iteration may activate a subset of the plurality of synapses or filters in the neural network. Each individual or group of synapses or filters may be activated in a portion of a total number of iterations proportional to the historical metric of activity independently determined for that individual or group of synapses or filters. Training or prediction of the neural network may be performed based on the plurality of partial activations of the neural network.

Abstract: A device, system, and method is provided for training or prediction using a cluster-connected neural network. The cluster-connected neural network may be divided into a plurality of clusters of artificial neurons connected by weights or convolutional channels connected by convolutional filters. Within each cluster is a locally dense sub-network of intra-cluster weights or filters with a majority of pairs of neurons or channels connected by intra-cluster weights or filters that are co-activated together as an activation block during training or prediction. Outside each cluster is a globally sparse network of inter-cluster weights or filters with a minority of pairs of neurons or channels separated by a cluster border across different clusters connected by inter-cluster weights or filters. Training or predicting is performed using the cluster-connected neural network.

Abstract: A device, system, and method for training or prediction of a neural network. A current value may be stored for each of a plurality of synapses or filters in the neural network. A historical metric of activity may be independently determined for each individual or group of the synapses or filters during one or more past iterations. A plurality of partial activations of the neural network may be iteratively executed. Each partial-activation iteration may activate a subset of the plurality of synapses or filters in the neural network. Each individual or group of synapses or filters may be activated in a portion of a total number of iterations proportional to the historical metric of activity independently determined for that individual or group of synapses or filters. Training or prediction of the neural network may be performed based on the plurality of partial activations of the neural network.

Abstract: A device, system, and method for approximating a neural network comprising N synapses or filters. The neural network may be partially-activated by iteratively executing a plurality of M partial pathways of the neural network to generate M partial outputs, wherein the M partial pathways respectively comprise M different continuous sequences of synapses or filters linking an input layer to an output layer. The M partial pathways may cumulatively span only a subset of the N synapses or filters such that a significant number of the remaining the N synapses or filters are not computed. The M partial outputs of the M partial pathways may be aggregated to generate an aggregated output approximating an output generated by fully-activating the neural network by executing a single instance of all N synapses or filters of the neural network. Training or prediction of the neural network may be performed based on the aggregated output.

Abstract: A method of determining a category of a malware file, using a malware determination system comprising a machine learning algorithm, the method comprising obtaining a file, which is assumed to constitute malware file, by the malware determination system, building a data structure representative of features present in said file, based on features present in at least one dictionary, wherein said dictionary stores at least, for each of one or more of categories Ci out of a plurality of N categories of malware files, with i from 1 to N and N>2, one or more features which are specific to said category Ci with respect to all other N?1 categories Cj, with j different from i, according to at least one first specificity criteria, feeding the data structure to the machine learning algorithm of the malware determination system, and providing prospects representative of one or more malware categories to which said file belongs, based on said data structure.

Abstract: Methods and systems are disclosed for training a malicious webpages detector for detecting malicious webpages, based on a training set comprising a plurality of samples representing malicious and non-malicious webpages. Text content can be extracted from the source code of each sample, and/or non-text content can be extracted from each sample, in order to train respectively at least a first deep learning neural network and a second deep learning neural network of the malicious webpages detector. A malicious webpages detector can detect whether or not a webpage is malicious, by extracting text content from the source code of the webpage, and/or non-text content from the webpage, thereafter providing prospects that the webpage is malicious based on the extracted data.

Abstract: A device, system, and method for approximating a neural network comprising N synapses or filters. The neural network may be partially-activated by iteratively executing a plurality of M partial pathways of the neural network to generate M partial outputs, wherein the M partial pathways respectively comprise M different continuous sequences of synapses or filters linking an input layer to an output layer. The M partial pathways may cumulatively span only a subset of the N synapses or filters such that a significant number of the remaining the N synapses or filters are not computed. The M partial outputs of the M partial pathways may be aggregated to generate an aggregated output approximating an output generated by fully-activating the neural network by executing a single instance of all N synapses or filters of the neural network. Training or prediction of the neural network may be performed based on the aggregated output.

Abstract: Methods and systems are disclosed for training a malicious webpages detector for detecting malicious webpages, based on a training set comprising a plurality of samples representing malicious and non-malicious webpages. Text content can be extracted from the source code of each sample, and/or non-text content can be extracted from each sample, in order to train respectively at least a first deep learning neural network and a second deep learning neural network of the malicious webpages detector. A malicious webpages detector can detect whether or not a webpage is malicious, by extracting text content from the source code of the webpage, and/or non-text content from the webpage, thereafter providing prospects that the webpage is malicious based on the extracted data.

Abstract: A device, system, and method is provided to mimic a pre-trained target model without access to the pre-trained target model or its original training dataset. A set of random or semi-random input data may be sent to randomly probe the pre-trained target model at a remote device. A set of corresponding output data may be received from the remote device that is generated by applying the pre-trained target model to the set of random or semi-random input data. A random probe training dataset may be generated comprising the set of random or semi-random input data and corresponding output data generated by randomly probing the pre-trained target model. A new model may be trained with the random probe training dataset so that the new model generates substantially the same corresponding output data in response to said input data to mimic the pre-trained target model.

Abstract: A device, system, and method for approximating a neural network comprising N synapses or filters. The neural network may be partially-activated by iteratively executing a plurality of M partial pathways of the neural network to generate M partial outputs, wherein the M partial pathways respectively comprise M different continuous sequences of synapses or filters linking an input layer to an output layer. The M partial pathways may cumulatively span only a subset of the N synapses or filters such that a significant number of the remaining the N synapses or filters are not computed. The M partial outputs of the M partial pathways may be aggregated to generate an aggregated output approximating an output generated by fully-activating the neural network by executing a single instance of all N synapses or filters of the neural network. Training or prediction of the neural network may be performed based on the aggregated output.

Abstract: A device, system, and method is provided to mimic a pre-trained target model without access to the pre-trained target model or its original training dataset. A set of random or semi-random input data may be sent to randomly probe the pre-trained target model at a remote device. A set of corresponding output data may be received from the remote device that is generated by applying the pre-trained target model to the set of random or semi-random input data. A random probe training dataset may be generated comprising the set of random or semi-random input data and corresponding output data generated by randomly probing the pre-trained target model. A new model may be trained with the random probe training dataset so that the new model generates substantially the same corresponding output data in response to said input data to mimic the pre-trained target model.

Abstract: According to some embodiments, a method for training a malware detector comprising a deep learning algorithm is described, which comprises converting a set of malware files and non malware files into vectors by using a feature based dictionary, and/or by using a conversion into an image, and providing prospects that the files constitute malware. Various features and combinations of features are described to build a feature based dictionary and adapt its size. According to some embodiments, a method for detecting a malware by using a malware detector comprising a deep learning algorithm is described, which comprises converting a file into a vector by using a feature based dictionary, and/or by using a conversion into an image, and providing prospects that the file constitutes malware. Methods for providing a plurality of prospects and aggregating these prospects are provided. Additional methods and systems in the field of malware detection are also described.

Abstract: A method of analyzing data exchange of at least one device includes feeding a plurality of data exchanged by the at least one device to a system for data exchange analysis that includes a deep learning algorithm. The deep learning algorithm includes at least an input layer, an output layer of the same size as the input layer, and hidden layers. Neurons of the hidden layers receive recurrently, at each time t, only a subset of the data exchanged by the at least one device up to time t, the subset of data comprising current data from time t and only a fraction of past data from time tpast to time t, with tpast<t. The method includes attempting to reconstruct, at the output layer, at each time t, data received at the input layer. The reconstructed data is compared with at least part of the plurality of data. In indication is provided on one or more anomalies in the data, based on at least the comparison.

Abstract: A device, system, and method for approximating a neural network comprising N synapses or filters. The neural network may be partially-activated by iteratively executing a plurality of M partial pathways of the neural network to generate M partial outputs, wherein the M partial pathways respectively comprise M different continuous sequences of synapses or filters linking an input layer to an output layer. The M partial pathways may cumulatively span only a subset of the N synapses or filters such that a significant number of the remaining the N synapses or filters are not computed. The M partial outputs of the M partial pathways may be aggregated to generate an aggregated output approximating an output generated by fully-activating the neural network by executing a single instance of all N synapses or filters of the neural network. Training or prediction of the neural network may be performed based on the aggregated output.

Abstract: A method of determining a category of a malware file, using a malware determination system comprising a machine learning algorithm, the method comprising obtaining a file, which is assumed to constitute malware file, by the malware determination system, building a data structure representative of features present in said file, based on features present in at least one dictionary, wherein said dictionary stores at least, for each of one or more of categories Ci out of a plurality of N categories of malware files, with i from 1 to N and N>2, one or more features which are specific to said category Ci with respect to all other N?1 categories Cj, with j different from i, according to at least one first specificity criteria, feeding the data structure to the machine learning algorithm of the malware determination system, and providing prospects representative of one or more malware categories to which said file belongs, based on said data structure.