Abstract: An apparatus and method for gesture detection and recognition. The apparatus includes a processing element, a radar sensor, a depth sensor, and an optical sensor. The radar sensor, the depth sensor, and the optical sensor are coupled to the processing element, and the radar sensor, the depth sensor, and the optical sensor are configured for short range gesture detection and recognition. The processing element is further configured to detect and recognize a hand gesture based on data acquired with the radar sensor, the depth sensor, and the optical sensor.

Abstract: An apparatus and method for radar based gesture detection. The apparatus includes a processing element and a transmitter configured to transmit radar signals. The transmitter is coupled to the processing element. The apparatus further includes a plurality of receivers configured to receive radar signal reflections, where the plurality of receivers is coupled to the processing element. The transmitter and plurality of receivers are configured for short range radar and the processing element is configured to detect a hand gesture based on the radar signal reflections received by the plurality of receivers.

Abstract: Estimating a three-dimensional (3D) pose of an object, such as a hand or body (human, animal, robot, etc.), from a 2D image is necessary for human-computer interaction. A hand pose can be represented by a set of points in 3D space, called keypoints. Two coordinates (x,y) represent spatial displacement and a third coordinate represents a depth of every point with respect to the camera. A monocular camera is used to capture an image of the 3D pose, but does not capture depth information. A neural network architecture is configured to generate a depth value for each keypoint in the captured image, even when portions of the pose are occluded, or the orientation of the object is ambiguous. Generation of the depth values enables estimation of the 3D pose of the object.

Abstract: A method, computer readable medium, and system are disclosed for classifying video image data. The method includes the steps of processing training video image data by at least a first layer of a convolutional neural network (CNN) to extract a first set of feature maps and generate classification output data for the training video image data. Spatial classification accuracy data is computed based on the classification output data and target classification output data and spatial discrimination factors for the first layer are computed based on the spatial classification accuracies and the first set of feature maps.

Abstract: Detection of activity in video content, and more particularly detecting in video start and end frames inclusive of an activity and a classification for the activity, is fundamental for video analytics including categorizing, searching, indexing, segmentation, and retrieval of videos. Existing activity detection processes rely on a large set of features and classifiers that exhaustively run over every time step of a video at multiple temporal scales, or as a small improvement computationally propose segments of the video on which to perform classification. These existing activity detection processes, however, are computationally expensive, particularly when trying to achieve activity detection accuracy, and moreover are not configurable for any particular time or computation budget. The present disclosure provides a time and/or computation budget-aware method for detecting activity in video that relies on a recurrent neural network implementing a learned policy.

Abstract: An apparatus and method for gesture detection and recognition. The apparatus includes a processing element, a radar sensor, a depth sensor, and an optical sensor. The radar sensor, the depth sensor, and the optical sensor are coupled to the processing element, and the radar sensor, the depth sensor, and the optical sensor are configured for short range gesture detection and recognition. The processing element is further configured to detect and recognize a hand gesture based on data acquired with the radar sensor, the depth sensor, and the optical sensor.

Abstract: A method, computer readable medium, and system are disclosed for visual sequence learning using neural networks. The method includes the steps of replacing a non-recurrent layer within a trained convolutional neural network model with a recurrent layer to produce a visual sequence learning neural network model and transforming feedforward weights for the non-recurrent layer into input-to-hidden weights of the recurrent layer to produce a transformed recurrent layer. The method also includes the steps of setting hidden-to-hidden weights of the recurrent layer to initial values and processing video image data by the visual sequence learning neural network model to generate classification or regression output data.

Abstract: A method, computer readable medium, and system are disclosed to generate coordinates of landmarks within images. The landmark locations may be identified on an image of a human face and used for emotion recognition, face identity verification, eye gaze tracking, pose estimation, etc. A transform is applied to input image data to produce transformed input image data. The transform is also applied to predicted coordinates for landmarks of the input image data to produce transformed predicted coordinates. A neural network model processes the transformed input image data to generate additional landmarks of the transformed input image data and additional predicted coordinates for each one of the additional landmarks. Parameters of the neural network model are updated to reduce differences between the transformed predicted coordinates and the additional predicted coordinates.

Abstract: A method, computer readable medium, and system are disclosed for sequential multi-tasking to generate coordinates of landmarks within images. The landmark locations may be identified on an image of a human face and used for emotion recognition, face identity verification, eye gaze tracking, pose estimation, etc. A neural network model processes input image data to generate pixel-level likelihood estimates for landmarks in the input image data and a soft-argmax function computes predicted coordinates of each landmark based on the pixel-level likelihood estimates.

Abstract: A method, computer readable medium, and system are disclosed for detecting and classifying hand gestures. The method includes the steps of receiving an unsegmented stream of data associated with a hand gesture, extracting spatio-temporal features from the unsegmented stream by a three-dimensional convolutional neural network (3DCNN), and producing a class label for the hand gesture based on the spatio-temporal features.

Abstract: An apparatus and method for gesture detection and recognition. The apparatus includes a processing element, a radar sensor, a depth sensor, and an optical sensor. The radar sensor, the depth sensor, and the optical sensor are coupled to the processing element, and the radar sensor, the depth sensor, and the optical sensor are configured for short range gesture detection and recognition. The processing element is further configured to detect and recognize a hand gesture based on data acquired with the radar sensor, the depth sensor, and the optical sensor.

Abstract: A method, computer readable medium, and system are disclosed for neural network pruning. The method includes the steps of receiving first-order gradients of a cost function relative to layer parameters for a trained neural network and computing a pruning criterion for each layer parameter based on the first-order gradient corresponding to the layer parameter, where the pruning criterion indicates an importance of each neuron that is included in the trained neural network and is associated with the layer parameter. The method includes the additional steps of identifying at least one neuron having a lowest importance and removing the at least one neuron from the trained neural network to produce a pruned neural network.

Abstract: A method, computer readable medium, and system are disclosed for classifying video image data. The method includes the steps of processing training video image data by at least a first layer of a convolutional neural network (CNN) to extract a first set of feature maps and generate classification output data for the training video image data. Spatial classification accuracy data is computed based on the classification output data and target classification output data and spatial discrimination factors for the first layer are computed based on the spatial classification accuracies and the first set of feature maps.

Abstract: Various techniques are disclosed to suppress distortion in images (e.g., video or still images), such as distortion caused by atmospheric turbulence. For example, similar image blocks from a sequence of images may be identified and tracked along motion trajectories to construct spatiotemporal volumes. The motion trajectories are smoothed to estimate the true positions of the image blocks without random displacements/shifts due to the distortion, and the smoothed trajectories are used to aggregate the image blocks in their new estimated positions to reconstruct the sequence of images with the random displacements/shifts suppressed. Blurring that may remain within each image block of the spatiotemporal volumes may be suppressed by modifying the spatiotemporal volumes in a collaborative fashion.

Abstract: A method, computer readable medium, and system are disclosed for detecting and classifying hand gestures. The method includes the steps of receiving an unsegmented stream of data associated with a hand gesture, extracting spatio-temporal features from the unsegmented stream by a three-dimensional convolutional neural network (3DCNN), and producing a class label for the hand gesture based on the spatio-temporal features.

Abstract: An apparatus and method for gesture detection and recognition. The apparatus includes a processing element, a radar sensor, a depth sensor, and an optical sensor. The radar sensor, the depth sensor, and the optical sensor are coupled to the processing element, and the radar sensor, the depth sensor, and the optical sensor are configured for short range gesture detection and recognition. The processing element is further configured to detect and recognize a hand gesture based on data acquired with the radar sensor, the depth sensor, and the optical sensor.

Abstract: An apparatus and method for radar based gesture detection. The apparatus includes a processing element and a transmitter configured to transmit radar signals. The transmitter is coupled to the processing element. The apparatus further includes a plurality of receivers configured to receive radar signal reflections, where the plurality of receivers is coupled to the processing element. The transmitter and plurality of receivers are configured for short range radar and the processing element is configured to detect a hand gesture based on the radar signal reflections received by the plurality of receivers.

Abstract: Various techniques are disclosed to suppress distortion in images (e.g., video or still images), such as distortion caused by atmospheric turbulence. For example, similar image blocks from a sequence of images may be identified and tracked along motion trajectories to construct spatiotemporal volumes. The motion trajectories are smoothed to estimate the true positions of the image blocks without random displacements/shifts due to the distortion, and the smoothed trajectories are used to aggregate the image blocks in their new estimated positions to reconstruct the sequence of images with the random displacements/shifts suppressed. Blurring that may remain within each image block of the spatiotemporal volumes may be suppressed by modifying the spatiotemporal volumes in a collaborative fashion.

Abstract: Multi-beam antenna array for anti jam and anti-spoof protection of GPS satellite data using multiple directional antennas disposed in various orientations jamming or spoofing signals from any direction cannot damage all said directional antennas simultaneously. Each said directional antenna connected to filtering amplifier and to multiple GPS processors for calculating direction of signal arrival. An anti-jam/anti-spoof processor comparing directions of signals arrival with real satellites positions for arrival time from data storage filters signals from jamming or spoofing signals, which are not corresponding to the correct positions stored for each satellite at the transmit time.

Abstract: A gating large area hybrid photomultiplier tube that includes an envelope, a photocathode for emitting electrons in correspondence with incident light entering the envelope, a collecting anode having a semiconductor device which has an electron incident surface for receiving photoelectrons emitted from the photocathode, a gating grid for gating the photoelectrons emitted from the photocathode, an electron optical system for focusing and directing the photoelectrons generated by the photocathode toward the electron incident surface, and an ion target for collecting positive ions from the photoelectrons. The envelope has a first opening and a second opening; the photocathode is disposed at the first opening, while the collecting anode is disposed at the second opening of the envelope.