Abstract: A low-order model (LOM) of biological neural networks and its mathematical equivalents including the clusterer interpreter probabilistic associative memory (CIPAM) are disclosed. They are artificial neural networks (ANNs) organized as networks of processing units (PUs), Each PU comprising artificial neuronal encoders, synapses, spiking/nonspiking neurons, and a scheme for maximal generalization. If the weights in the artificial synapses in a PU have been learned (and then fixed) or can be adjusted by the unsupervised accumulation rule and the unsupervised covariance rule (or supervised covariance rule), the PU is called unsupervised (or supervised) PU. The disclosed ANNs, with these Hebbian-type learning rules, can learn large numbers of large input vectors with temporally/spatially hierarchical causes with ease and recognize such causes with maximal generalization despite corruption, distortion and occlusion.

Abstract: A low-order model (LOM) of biological neural networks and its mathematical equivalents including the clusterer interpreter probabilistic associative memory (CIPAM) are disclosed. They are artificial neural networks (ANNs) organized as networks of processing units (PUs), each PU comprising artificial neuronal encoders, synapses, spiking/nonspiking neurons, and a scheme for maximal generalization. If the weights in the artificial synapses in a PU have been learned (and then fixed) or can be adjusted by the unsupervised accumulation rule and the unsupervised covariance rule (or supervised covariance rule), the PU is called unsupervised (or supervised) PU. The disclosed ANNs, with these Hebbian-type learning rules, can learn large numbers of large input vectors with temporally/spatially hierarchical causes with ease and recognize such causes with maximal generalization despite corruption, distortion and occlusion.

Abstract: A cortex-like learning machine, called a probabilistic associative memory (PAM), is disclosed for recognizing spatial and temporal patterns. A PAM is usually a multilayer or recurrent network of processing units (PUs). Each PU expands subvectors of a feature vector input to the PU into orthogonal vectors, and generates a probability distribution of the label of said feature vector, using expansion correlation matrices, which can be adjusted in supervised or unsupervised learning by a Hebbian-type rule. The PU also converts the probability distribution into a ternary vector to be included in feature subvectors that are input to PUs in the same or other layers. A masking matrix in each PU eliminates effect of corrupted components in query feature subvectors and enables maximal generalization by said PU and thereby that by the PAM. PAMs with proper learning can recognize rotated, translated and scaled patterns and are functional models of the cortex.

Abstract: A cortex-like learning machine, called a probabilistic associative memory (PAM), is disclosed for recognizing spatial and temporal patterns. A PAM is usually a multilayer or recurrent network of processing units (PUs). Each PU expands subvectors of a feature subvector input to the PU into orthogonal vectors, and generates a probability distribution of the label of said feature subvector, using expansion correlation matrices, which are adjusted in supervised or unsupervised learning by a Hebb rule. The PU also converts the probability distribution into a ternary vector to be included in feature subvectors that are input to PUs in the same or other layers. A masking matrix in each PU eliminates effect of corrupted components in query feature subvectors and enables maximal generalization by said PU and thereby that by the PAM. PAMs with proper learning can recognize rotated, translated and scaled patterns and are functional models of the cortex.

Abstract: A method of training neural systems and estimating regression coefficients of regression models with respect to an error criterion is disclosed. If the error criterion is a risk-averting error criterion, the invented method performs the training/estimation by starting with a small value of the risk-sensitivity index of the risk-averting error criterion and gradually increasing it to ensure numerical feasibility. If the error criterion is a risk-neutral error criterion such as a standard sum-of-squares error criterion, the invented method performs the training/estimation first with respect to a risk-averting error criterion associated with the risk-neutral error criterion. If the result is not satisfactory for the risk-neutral error criterion, further training/estimation is performed either by continuing risk-averting training/estimation with decreasing values of the associated risk-averting error criterion or by training/estimation with respect to the given risk-neutral error criterion or by both.

Abstract: A method of training neural systems and estimating regression coefficients of regression models with respect to an error criterion is disclosed. If the error criterion is a risk-averting error cri- terion, the invented method performs the training/estimation by starting with a small value of the risk-sensitivity index of the risk-averting error criterion and gradually increasing it to ensure numerical feasibility. If the error criterion is a risk-neutral error criterion such as a standard sum- of-squares error criterion, the invented method performs the training/estimation first with respect to a risk-averting error criterion associated with the risk-neutral error criterion. If the result is not satisfactory for the risk-neutral error criterion, further training/estimation is performed either by continuing risk-averting training/estimation with decreasing values of the associated risk-averting error criterion or by training/estimation with respect to the given risk-neutral error criterion or by both.

Abstract: A neural system is disclosed for processing an exogenous input process to produce a good outward output process with respect to a performance criterion, even if the range of one or both of these processes is necessarily large and/or keeps necessarily expanding during the operation of the neural system. The disclosed neural system comprises a recurrent neural network (RNN) and at least one range extender or reducer, each of which is a dynamic transformer. A range reducer transforms dynamically at least one component of the exogenous input process into inputs to at least one input neuron of said RNN. A range extender transforms dynamically outputs of at least one output neuron of said RNN into at least one component of the outward output process. There are many types of range extender and reducer, which have different degrees of effectiveness and computational costs.

Abstract: To enhance the safety and security of the operation of a railway network, a railway operation monitoring and diagnosing system is disclosed that monitors and diagnoses the entire railway network as an integrated system. The railway operation monitoring and diagnosing system comprises a railway operation predictor and a diagnosing means. The railway operation predictor generates anticipated values of selected railway operation state (ROS) variables. ROS variables may discrete or continuous. If there are continuous ROS variables selected, the railway operation predictor also determines the safety intervals of these continuous ROS variables. The diagnosing means examines the measured values of the selected ROS variables versus their anticipated values and/or safety intervals to detect and diagnose their discrepancies. A heuristics, statistics, fuzzy logic, artificial intelligence, neural network, or/and expert system is included in the diagnosing means for diagnosing the records of such discrepancies.

Abstract: A robust neural system for robust processing is disclosed for averting unacceptable or disastrous processing performances. This robust neural system either comprises a neural network or comprises a neural network and at least one range transformer. At least one adjustable weight of the robust neural system is a nonlinear weight of the neural work determined in a nonadaptive training of the robust neural system with respect to a nonadaptive risk-sensitive training criterion.If all the adjustable weights of the robust neural system are nonadaptively adjustable, all these weights are held fixed during the robust neural system's operation. If said neural network is recursive, and the nonadaptive training data used to construct said nonadaptive risk-sensitive training criterion contain data for each of a number of typical values of an environmental parameter, the robust neural system is not only robust but also adaptive to the environmental parameter.

Abstract: A recursive neurofilter comprising a recursive neural network (NN) is disclosed for processing an information process to estimate a signal process with respect to an estimation error criterion. The information process either consists of a measurement process, or if the signal and measurement processes are time-variant, consists of the measurement process as well as a time variance process, that describes the time-variant properties of the signal and measurement processes. The recursive neurofilter is synthesized from exemplary realizations of the signal and information processes. No assumptions such as Gaussian distribution, linear dynamics, additive noise, and Markov property are required. The synthesis is performed essentially through training recursive NNs. The training criterion is constructed to reflect the mentioned estimation error criterion with the exemplary realizations.

Abstract: An adaptive neural system (ANS) disclosed herein comprises a processor and an adaptor. The processor includes mainly a neural network whose adjustable weights are divided into nonadaptively and adaptively adjustable weights. The nonadaptively adjustable weights are determined through minimizing or reducing a nonadaptive training criterion in an off-line nonadaptive training. Being constructed with a priori training data, the nonadaptive training criterion is a function of the nonadaptively adjustable weights and the diversity variables associated with typical values of the environmental parameter. During an operation of the adaptive neural system, only the adaptively adjustable weights are adjusted on-line to adapt to the unknown environmental parameter. This adaptive training is achieved by minimizing or reducing an adaptive training criterion.

Abstract: A method and apparatus is provided for processing a measurement process to estimate a signal process, even if the signal and/or measurement processes have large and/or expanding ranges. The method synthesizes training data comprising realizations of the signal and measurement processes into a primary filter for estimating the signal process and, if required, an ancillary filter for providing the primary filter's estimation error statistics. The primary and ancillary filters each comprise an artificial recurrent neural network (RNN) and at least one range extender or reducer. Their implementation results in the filtering apparatus. Many types of range extender and reducer are disclosed, which have different degrees of effectiveness and computational cost.