Abstract: Methods, systems, and apparatuses, among other things, may detect and store activity in videos based on a spatiotemporal graph representation. Spatiotemporal proximity graphs may be built based on one or more received tracks and may include one or more nodes and each node may include one or more attributes associated with a corresponding entity. One or more spatiotemporal relationships may be identified between the entities based on each spatiotemporal proximity graph one or more activities of the entities may be identified based on the spatiotemporal relationships.

Abstract: A method of machine learning model development includes building an autoencoder including an encoder trained to map an input into a latent representation, and a decoder trained to map the latent representation to a reconstruction of the input. The method includes building an artificial neural network classifier including the encoder, and a classification layer partially trained to perform a classification in which a class to which the input belongs is predicted based on the latent representation. Neural network inversion is applied to the classification layer to find inverted latent representations within a decision boundary between classes in which a result of the classification is ambiguous, and inverted inputs are obtained from the inverted latent representations. Each inverted input is labeled with a class that is its ground truth, and thereby producing added training data for the classification, and the classification layer is further trained using the added training data.

Abstract: Methods, systems, and apparatuses, among other things, may detect and store activity in videos based on a spatiotemporal graph representation. Spatiotemporal proximity graphs may be built based on one or more received tracks and may include one or more nodes and each node may include one or more attributes associated with a corresponding entity. One or more spatiotemporal relationships may be identified between the entities based on each spatiotemporal proximity graph one or more activities of the entities may be identified based on the spatiotemporal relationships.

Abstract: Methods, systems, and apparatuses, among other things, may perform persistent object tracking and reidentification through detection and continuous feature comparison. For example, video frames may be received (e.g., from a camera, an application, or a data storage device) and an object of interest may be detected at a first position in a video frame and the object of interest may be detected at a second position in another video frame. A track associated with the object of interest may be generated based on the detected first and second positions of the object of interest.

Abstract: An apparatus is provided for automated object tracking in a video feed. The apparatus receives and sequentially processes a plurality of frames of the video feed to track objects. In particular, a plurality of objects in a frame are detected and assigned to a respective track fragment. A kinematic, visual, temporal or machine learning-based feature of an object is then identified and stored in metadata associated with the track fragment. A track fragment for the object is identified in earlier frames based on a comparison of the feature and a corresponding feature in metadata associated with the earlier frames. The track fragments for the object in the frame and the object in the earlier frames are linked to form a track of the object. The apparatus then outputs the video feed with the track of the object as an overlay thereon.

Abstract: Systems, methods, and computer-readable media storing instructions for determining cross-track error of an aircraft on a taxiway are disclosed herein. The disclosed techniques capture electronic images of a portion of the taxiway using cameras or other electronic imaging devices mounted on the aircraft, pre-process the electronic images to generate regularized image data, apply a trained multichannel neural network model to the regularized image data to generate a preliminary estimate of cross-track error relative to the centerline of the taxiway, and post-process the preliminary estimate to generate an estimate of cross-track error of the aircraft. Further embodiments adjust a GPS-based location estimate of the aircraft using the estimate of cross-track error or adjust the heading of the aircraft based upon the estimate of cross-track error.

Abstract: A method of machine learning model development includes building an autoencoder including an encoder trained to map an input into a latent representation, and a decoder trained to map the latent representation to a reconstruction of the input. The method includes building an artificial neural network classifier including the encoder, and a classification layer partially trained to perform a classification in which a class to which the input belongs is predicted based on the latent representation. Neural network inversion is applied to the classification layer to find inverted latent representations within a decision boundary between classes in which a result of the classification is ambiguous, and inverted inputs are obtained from the inverted latent representations. Each inverted input is labeled with a class that is its ground truth, and thereby producing added training data for the classification, and the classification layer is further trained using the added training data.

Abstract: Systems, methods, and computer-readable media storing instructions for determining cross-track error of an aircraft on a taxiway are disclosed herein. The disclosed techniques capture electronic images of a portion of the taxiway using cameras or other electronic imaging devices mounted on the aircraft, pre-process the electronic images to generate regularized image data, apply a trained multichannel neural network model to the regularized image data to generate a preliminary estimate of cross-track error relative to the centerline of the taxiway, and post-process the preliminary estimate to generate an estimate of cross-track error of the aircraft. Further embodiments adjust a GPS-based location estimate of the aircraft using the estimate of cross-track error or adjust the heading of the aircraft based upon the estimate of cross-track error.

Abstract: An apparatus is provided for automated object and activity tracking in a live video feed. The apparatus receives and processes a live video feed to identify a plurality of objects and activities therein. The apparatus also generates natural language text that describes a storyline of the live video feed using the plurality of objects and activities so identified. The live video feed is processed using computer vision, natural language processing and machine learning, and a catalog of identifiable objects and activities. The apparatus then outputs the natural language text audibly or visually with a display of the live video feed.

Abstract: An apparatus is provided for automated object tracking in a video feed. The apparatus receives and sequentially processes a plurality of frames of the video feed to track objects. In particular, a plurality of objects in a frame are detected and assigned to a respective track fragment. A kinematic, visual, temporal or machine learning-based feature of an object is then identified and stored in metadata associated with the track fragment. A track fragment for the object is identified in earlier frames based on a comparison of the feature and a corresponding feature in metadata associated with the earlier frames. The track fragments for the object in the frame and the object in the earlier frames are linked to form a track of the object. The apparatus then outputs the video feed with the track of the object as an overlay thereon.

Abstract: An apparatus is provided for automated object and activity tracking in a live video feed. The apparatus receives and processes a live video feed to identify a plurality of objects and activities therein. The apparatus also generates natural language text that describes a storyline of the live video feed using the plurality of objects and activities so identified. The live video feed is processed using computer vision, natural language processing and machine learning, and a catalog of identifiable objects and activities. The apparatus then outputs the natural language text audibly or visually with a display of the live video feed.