Abstract: In accordance with some embodiments, systems, methods, and media for generating and using neural networks having improved efficiency for analyzing video are provided. In some embodiments, the method comprises: providing image data to a trained neural network; receiving, at a neuron, a delta-based input ?in from a previous layer; generating an output g(?in) of a linear transform g; generating an updated state variable a based on g(?in) and a current a; generating an output ƒ(a) of an activation function ƒ based on updated a; generating an updated state variable d based on a current d, a state variable b, and ƒ(a); generating an updated b based on output ƒ(a); transmitting d to a next layer based on a transmission policy and subtracting the value from d; and receiving an output from the trained neural network that represents a prediction based on the image data.

Abstract: Methods and systems for processing frame data are disclosed. The methods and systems include: obtaining a sequence of data, the sequence comprising at least a first frame; applying the sequence of data to a machine learning model; and providing an inference result based on the machine learning output score from the machine learning model. The machine learning model is configured to: determine at least one token-level error based on a difference between at least one token of a first representation corresponding to the first frame and a second representation; perform a first transformer operation based on the token subset to produce an output representation corresponding to the token subset; scatter the output representation to a buffer representation corresponding to the first representation; and produce a machine learning output score based on the buffer representation. Other aspects, embodiments, and features are also claimed and described.

Abstract: Methods and systems for detecting changes via an event camera are disclosed. The methods and systems include: monitoring a plurality of pixel measurements from an image sensor, determining an estimate of intensity for a scene using a current frame; detecting changes of the plurality of pixel measurements using the estimate of intensity for the scene; maintaining a stored flux value for each of a first plurality of pixels, wherein the first plurality pixels have not changed intensities; transmitting change information for each of a second plurality of pixels, wherein the second plurality of pixels have changed intensities; determining if a change in the scene has occurred based on the change information for each of the second plurality of pixels; triggering an event in response to the change in the scene; and rendering an event camera image. Other aspects, embodiments, and features are also claimed and described.

Abstract: In accordance with some embodiments, systems, methods, and media for generating digital images using low bit depth image sensor data are provided. In some embodiments, the system comprises: an image sensor; a processor programmed to: receive, from the image sensor, a series of low bit depth frames; provide low bit depth image information to a trained machine learning model comprising: a 3D convolutional layer; a 2D convolutional LSTM layer; a concatenation layer configured to generate a tensor that includes an output of the 2D convolutional LSTM layer and the low bit depth image information; and a 2D convolutional layer configured to generate an output based on the tensor; and generate a high bit depth image of a scene based on an output of the two-dimensional convolutional layer.

Abstract: Methods and systems for processing frame data are disclosed. The methods and systems include: obtaining a sequence of data, the sequence comprising at least a first frame; applying the sequence of data to a machine learning model; and providing an inference result based on the machine learning output score from the machine learning model. The machine learning model is configured to: determine at least one token-level error based on a difference between at least one token of a first representation corresponding to the first frame and a second representation; perform a first transformer operation based on the token subset to produce an output representation corresponding to the token subset; scatter the output representation to a buffer representation corresponding to the first representation; and produce a machine learning output score based on the buffer representation. Other aspects, embodiments, and features are also claimed and described.

Abstract: In accordance with some embodiments, systems, methods, and media for generating digital images using low bit depth image sensor data are provided. In some embodiments, the system comprises: an image sensor; a processor programmed to: receive, from the image sensor, a series of low bit depth frames; provide low bit depth image information to a trained machine learning model comprising: a 3D convolutional layer; a 2D convolutional LSTM layer; a concatenation layer configured to generate a tensor that includes an output of the 2D convolutional LSTM layer and the low bit depth image information; and a 2D convolutional layer configured to generate an output based on the tensor; and generate a high bit depth image of a scene based on an output of the two-dimensional convolutional layer.

Abstract: In accordance with some embodiments, systems, methods, and media for generating and using neural networks having improved efficiency for analyzing video are provided. In some embodiments, the method comprises: providing image data to a trained neural network; receiving, at a neuron, a delta-based input ?in from a previous layer; generating an output g(?in) of a linear transform g; generating an updated state variable a based on g(?in) and a current a; generating an output f(a) of an activation function f based on updated a; generating an updated state variable d based on a current d, a state variable b, and f(a); generating an updated b based on output f(a); transmitting d to a next layer based on a transmission policy and subtracting the value from d; and receiving an output from the trained neural network that represents a prediction based on the image data.

Abstract: In accordance with some embodiments, systems, methods, and media for generating digital images using low bit depth image sensor data are provided. In some embodiments, the system comprises: an image sensor; a processor programmed to: receive, from the image sensor, a series of low bit depth frames; provide low bit depth image information to a trained machine learning model comprising: a 3D convolutional layer; a 2D convolutional LSTM layer; a concatenation layer configured to generate a tensor that includes an output of the 2D convolutional LSTM layer and the low bit depth image information; and a 2D convolutional layer configured to generate an output based on the tensor; and generate a high bit depth image of a scene based on an output of the two-dimensional convolutional layer.

Abstract: In accordance with some embodiments, systems, methods, and media for generating and using spiking neural networks with improved efficiency are provided. In some embodiments, a method comprises: receiving image data; providing the image data to a trained spiking neural network (SNN), the SNN comprising a plurality of neurons, each of the plurality of neurons associated with a respective initialization value V0 of a plurality of initialization values, wherein a first layer of the trained SNN comprises a first subset of the plurality of neurons, and a second layer of the trained SNN comprises a second subset of the plurality of neurons, and wherein a mean of the plurality of initialization values is about 0.5, and a standard deviation of the initialization values is at least 0.