Abstract: Provided is a multimedia data stream processing method, an electronic device and a storage medium, relating to the field of artificial intelligence, and specifically, to the technical fields of cloud computing, media cloud technology, and the like, which may be applied to scenes such as smart cloud. The multimedia data stream processing method includes: allocating a plurality of sub-streams of a multimedia data stream to a plurality of edge resource nodes, where the multimedia data stream is segmented into a plurality of slices and each of the plurality of sub-streams includes a part of the plurality of slices of the multimedia data stream; and scheduling the plurality of edge resource nodes to provide the plurality of sub-streams of the multimedia data stream for a terminal device.

Abstract: Provided is a multimedia data stream processing method, an electronic device and a storage medium, relating to the field of artificial intelligence, and specifically, to the technical fields of cloud computing, media cloud technology, and the like, which may be applied to scenes such as smart cloud. The multimedia data stream processing method includes: allocating a plurality of sub-streams of a multimedia data stream to a plurality of edge resource nodes, where the multimedia data stream is segmented into a plurality of slices and each of the plurality of sub-streams includes a part of the plurality of slices of the multimedia data stream; and scheduling the plurality of edge resource nodes to provide the plurality of sub-streams of the multimedia data stream for a terminal device.

Abstract: Techniques and mechanisms to facilitate a data classification functionality by communicating feedback signals with a spiked neural network. In an embodiment, input signaling, provided to the spiking neural network, results in one or more output spike trains which are indicative of that the input signaling corresponds to a particular data type. Based on the one or more output spike trains, feedback signals are variously communicated each to a respective node of the spiking neural network. The feedback signals variously control signal response characteristics of the nodes. Subsequent output signaling by the spiking neural network, in further response the input signaling, is improved based on the feedback control of nodes' signal responses. In another embodiment, the feedback signals are used to adjust synaptic weight values during training of the spiking neural network.

Abstract: Techniques and mechanisms to update a synaptic weight of a spiking neural network which is trained to provide a decision of a decision-making sequence. In an embodiment, a synapse of the spiking neural network is associated with a weight which is to be given to communications via that given synapse. The spiking neural network generates output signaling, indicating a decision to the decision-making process, which is evaluated to determine whether, according to predefined test criteria, the decision-making process is successful or unsuccessful. One or more nodes of the spiking neural network receive a reward/penalty signal which is based on the evaluation. In response to the reward/penalty signal indicating a reward event or a penalty event, a synaptic weight value is updated. In another embodiment, input signaling provided to the spiking neural network represents a sub-sequence of two or more most recent states in a sequence of states.

Abstract: Techniques and mechanisms to facilitate a data classification functionality by communicating feedback signals with a spiked neural network. In an embodiment, input signaling, provided to the spiking neural network, results in one or more output spike trains which are indicative of that the input signaling corresponds to a particular data type. Based on the one or more output spike trains, feedback signals are variously communicated each to a respective node of the spiking neural network. The feedback signals variously control signal response characteristics of the nodes. Subsequent output signaling by the spiking neural network, in further response the input signaling, is improved based on the feedback control of nodes' signal responses. In another embodiment, the feedback signals are used to adjust synaptic weight values during training of the spiking neural network.

Abstract: Described is a system for pattern recognition designed for neuromorphic hardware. The system generates a spike train of neuron spikes for training patterns with each excitatory neuron in an excitatory layer, where each training pattern belongs to a pattern class. A spiking rate distribution of excitatory neurons is generated for each pattern class. Each spiking rate distribution of excitatory neurons is normalized, and a class template is generated for each pattern class from the normalized spiking rate distributions. An unlabeled input pattern is classified using the class templates. A mechanical component of an autonomous device can be controlled based on classification of the unlabeled input pattern.

Abstract: Described is a system for converting convolutional neural networks to spiking neural networks. A convolutional neural network (CNN) is adapted to fit a set of requirements of a spiking neural network (SNN), resulting in an adapted CNN. The adapted CNN is trained to obtain a set of learned weights, and the set of learned weights is then applied to a converted SNN having an architecture similar to the adapted CNN. The converted SNN is then implemented on neuromorphic hardware, resulting in reduced power consumption.

Abstract: Described is a system for pattern recognition designed for neuromorphic hardware. The system generates a spike train of neuron spikes for training patterns with each excitatory neuron in an excitatory layer, where each training pattern belongs to a pattern class. A spiking rate distribution of excitatory neurons is generated for each pattern class. Each spiking rate distribution of excitatory neurons is normalized, and a class template is generated for each pattern class from the normalized spiking rate distributions. An unlabeled input pattern is classified using the class templates. A mechanical component of an autonomous device can be controlled based on classification of the unlabeled input pattern.

Abstract: Described is a system and method for ultra-low power consumption state deep online learning. The system operates by filtering an input image to generate one or more feature maps. The one or more feature maps are divided into non-overlapping small regions with feature values in each small region pooled to generate decreased size feature maps. The decreased size feature maps are divided into overlapping patches which are joined together to form a collection of cell maps having connections to the decreased sized feature maps. The collection of cell maps are then divided into non-overlapping small regions, with feature values in each small region pooled to generate a decreased sized collection of cell maps. The decreased sized collection of cell maps are then mapped to a single cell, which results in a class label being generated as related to the input image based on the single cell.

Abstract: Thermal management of a computing device is achieved using reward-based updating of synaptic weights of a spiking neural network. The thermal management is achieved using machine readable mediums having instructions that cause a processor to, during a first time window, generate weights to be applied to input trains of spikes from input neurons of a spiking neural network. The instructions further cause the processor to, based on a number of spikes included in an output train of spikes output by an output neuron of the spiking neural network during the first time window, adjust the workload of the processor, and, based on whether a surface temperature of an enclosure housing the processor meets a first threshold or a workload of the processor meets a second threshold, generate a penalty. The instructions also cause the processor to train the spiking neural network by updating the weights.

Abstract: Described is a system for object detection in images or videos using spiking neural networks. An intensity saliency map is generated from an intensity of an input image having color components using a spiking neural network. Additionally, a color saliency map is generated from a plurality of colors in the input image using a spiking neural network. An object detection model is generated by combining the intensity saliency map and multiple color saliency maps. The object detection model is used to detect multiple objects of interest in the input image.

Abstract: Described is a system for compensating for ego-motion during video processing. The system generates an initial estimate of camera ego-motion of a moving camera for consecutive image frame pairs of a video of a scene using a projected correlation method, the camera configured to capture the video from a moving platform. An optimal estimation of camera ego-motion is generated using the initial estimate as an input to a valley search method or an alternate line search method. All independent moving objects are detected in the scene using the described hybrid method at superior performance compared to existing methods while saving computational cost.

Abstract: Described is a system for ghost removal in video footage. During operation, the system generates a background subtraction map and an original bounding box that surrounds a detected foreground object through background subtraction. A detected foreground map is then generated. The detected foreground map includes at least two detected foreground (DF) bounding boxes of detected foregrounds obtained by a difference of two consecutive frames in video footage. Further, the original bounding box is then trimmed into a trimmed box, the trimmed box being a smallest box that contains the at least two DF bounding boxes. The trimmed box is designated as containing a real-world object, which can then be used for object tracking.

Abstract: Described is a system for estimating ego-motion of a moving camera for detection of independent moving objects in a scene. For consecutive frames in a video captured by a moving camera, a first ego-translation estimate is determined between the consecutive frames from a first local minimum. From a second local minimum, a second ego-translation estimate is determined. If the first ego-translation estimate is equivalent to the second ego-translation estimate, the second ego-translation estimate is output as the optimal solution. Otherwise, a cost function is minimized to determine an optimal translation until the first ego-translation estimate is equivalent to the second ego-translation estimate, and an optimal solution is output. Ego-motion of the camera is estimated using the optimal solution, and independent moving objects are detected in the scene.

Abstract: Described is a system for compensating ego-translations in video captured with a moving camera. Translative ego-motion is estimated on a sequence of image frames captured by a moving camera by minimizing a cost function that is based on at least one image frame difference between consecutive image frames. An alternating one directional search is performed to minimize the cost function to find an optimal translation. The optimal translation is applied to the sequence of image frames, resulting in a sequence of image frames with ego-translations.

Abstract: Described is a system for compensating for ego-motion during video processing. The system generates an initial estimate of camera ego-motion of a moving camera for consecutive image frame pairs of a video of a scene using a projected correlation method, the camera configured to capture the video from a moving platform. An optimal estimation of camera ego-motion is generated using the initial estimate as an input to a valley search method or an alternate line search method. All independent moving objects are detected in the scene using the described hybrid method at superior performance compared to existing methods while saving computational cost.

Abstract: Systems and methods for supervised learning and cascaded training of a neural network are described. In an example, a supervised process is used for strengthening connections to classifier neurons, with a supervised learning process of receiving a first spike at a classifier neuron from a processing neuron in response to training data, and receiving an out-of-band communication of a second desired (artificial) spike at the classifier neuron that corresponds to the classification of the training data. As a result of spike timing dependent plasticity, connections to the classifier neuron are strengthened. In another example, a cascaded technique is disclosed to generate a plurality of trained neural networks that are separately initialized and trained based on different types or forms of training data, which may be used with cascaded or parallel operation of the plurality of trained neural networks.

Abstract: Described is system for simulating spiking neural networks for image and video processing. The system processes an image with a spiking neural network simulator having a plurality of inter-connected modules. Each module comprises a plurality of neuron elements. Processing the image further comprises performing a neuron state update for each module, that includes aggregating input spikes and updating neuron membrane potentials, and performing spike propagation for each module, which includes transferring spikes generated in a current time step. Finally, an analysis result is output.

Abstract: Described is a system for collision detection. The system divides an image in a sequence of images into multiple sub-fields comprising complementary visual sub-fields. For each visual sub-field, motion is detected in a direction corresponding to the visual sub-field using a spiking Reichardt detector with a spiking neural network. Motion in a direction complementary to the visual sub-field is also detected using the spiking Reichardt detector. Outputs of the spiking Reichardt detector, comprising data corresponding to one direction of movement from two complementary visual sub-fields, are processed using a movement detector. Based on the output of the movement detector, an impending collision is signaled.

Abstract: Described is a system for ghost removal in video footage. During operation, the system generates a background subtraction map and an original bounding box that surrounds a detected foreground object through background subtraction. A detected foreground map is then generated. The detected foreground map includes at least two detected foreground (DF) bounding boxes of detected foregrounds obtained by a difference of two consecutive frames in video footage. Further, the original bounding box is then trimmed into a trimmed box, the trimmed box being a smallest box that contains the at least two DF bounding boxes. The trimmed box is designated as containing a real-world object, which can then be used for object tracking.