Abstract: Methods and systems for detecting and correcting anomalies include detecting an anomaly in a cyber-physical system, based on a classification of time series information from sensors that monitor the cyber-physical system as being anomalous. A similarity graph is determined for each of the sensors, based on the time series information. A subset of the sensors that are related to the classification is selected, based on a spectral embedding of the similarity graphs. A corrective action is performed responsive to the detected anomaly, prioritized according to the selected subset.

Abstract: A method for using a multi-hop reasoning framework to perform multi-step compositional long-term reasoning is presented. The method includes extracting feature maps and frame-level representations from a video stream by using a convolutional neural network (CNN), performing object representation learning and detection, linking objects through time via tracking to generate object tracks and image feature tracks, feeding the object tracks and the image feature tracks to a multi-hop transformer that hops over frames in the video stream while concurrently attending to one or more of the objects in the video stream until the multi-hop transformer arrives at a correct answer, and employing video representation learning and recognition from the objects and image context to locate a target object within the video stream.

Abstract: Systems and methods for solving queries on image data are provided. The system includes a processor device coupled to a memory device. The system includes a detector manager with a detector application programming interface (API) to allow external detectors to be inserted into the system by exposing capabilities of the external detectors and providing a predetermined way to execute the external detectors. An ontology manager exposes knowledge bases regarding ontologies to a reasoning engine. A query parser transforms a natural query into query directed acyclic graph (DAG). The system includes a reasoning engine that uses the query DAG, the ontology manager and the detector API to plan an execution list of detectors. The reasoning engine uses the query DAG, a scene representation DAG produced by the external detectors and the ontology manager to answer the natural query.

Abstract: Systems and methods for labelling data is provided. The method includes receiving data at a detector, and identifying a set of objects and features in the data using a neural network. The method further includes annotating the data based on the identified set of objects and features, and receiving a query from a user. The method further includes transforming the query into a representation that can be processed by a symbolic engine, and receiving the annotated data and a transformed query at the symbolic engine. The method further includes matching the transformed query with the annotated data, and presenting the annotated data that matches the transformed query to the user in a labelling interface. The method further includes applying new labels received from the user for the annotated data that matches the transformed query, recursively utilizing the newly annotated data to refine the detector.

Abstract: A method for using a multi-hop reasoning framework to perform multi-step compositional long-term reasoning is presented. The method includes extracting feature maps and frame-level representations from a video stream by using a convolutional neural network (CNN), performing object representation learning and detection, linking objects through time via tracking to generate object tracks and image feature tracks, feeding the object tracks and the image feature tracks to a multi-hop transformer that hops over frames in the video stream while concurrently attending to one or more of the objects in the video stream until the multi-hop transformer arrives at a correct answer, and employing video representation learning and recognition from the objects and image context to locate a target object within the video stream.

Abstract: Methods and systems for detecting and correcting anomalies include predicting normal behavior of a monitored system based on training data that includes only sensor data collected during normal behavior of the monitored system. The predicted normal behavior is compared to recent sensor data to determine that the monitored system is behaving abnormally. A corrective action is performed responsive to the abnormal behavior to correct the abnormal behavior.

Abstract: A false alarm reduction system and method are provided for reducing false alarms in an automatic defect detection system. The false alarm reduction system includes a defect detection system, generating a list of image boxes marking detected potential defects in an input image. The false alarm reduction system further includes a feature extractor, transforming each of the image boxes in the list into a respective set of numerical features. The false alarm reduction system also includes a classifier, computing as a classification outcome for the each of the image boxes whether the detected potential defect is a true defect or a false alarm responsive to the respective set of numerical features for each of the image boxes.

Abstract: Methods and systems for detecting and correcting anomalies include detecting an anomaly in a cyber-physical system, based on a classification of time series information from sensors that monitor the cyber-physical system as being anomalous. A similarity graph is determined for each of the sensors, based on the time series information. A subset of the sensors that are related to the classification is selected, based on a spectral embedding of the similarity graphs. A corrective action is performed responsive to the detected anomaly, prioritized according to the selected subset.

Abstract: A method is provided for model training to detect defective products. The method includes sampling training images of a product to (i) extract image portions therefrom made of a center patch and its context and (ii) black-out the center patch. The method further includes performing unsupervised back-propagation training of a Contextual Auto-Encoder (CAE) model using (i) the image portions with the blacked-out center patch as an input and, (ii) the center patch as a target output and, (iii) an image-based loss function, to obtain a trained CAE model. The method also includes sampling positive and negative center-patch-sized portions from the training images. The method additionally includes normalizing, using the trained CAE model, the positive and negative center-patch-sized portions.

Abstract: A false alarm reduction system is provided that includes a processor cropping each input image at randomly chosen positions to form cropped images of a same size at different scales in different contexts. The system further includes a CONDA-GMM, having a first and a second conditional deep autoencoder for respectively (i) taking each cropped image without a respective center block as input for measuring a discrepancy between a reconstructed and a target center block, and (ii) taking an entirety of cropped images with the target center block. The CONDA-GMM constructs density estimates based on reconstruction error features and low-dimensional embedding representations derived from image encodings. The processor determines an anomaly existence based on a prediction of a likelihood of the anomaly existing in a framework of a CGMM, given the context being a representation of the cropped image with the center block removed and having a discrepancy above a threshold.

Abstract: Systems and methods for solving queries on image data are provided. The system includes a processor device coupled to a memory device. The system includes a detector manager with a detector application programming interface (API) to allow external detectors to be inserted into the system by exposing capabilities of the external detectors and providing a predetermined way to execute the external detectors. An ontology manager exposes knowledge bases regarding ontologies to a reasoning engine. A query parser transforms a natural query into query directed acyclic graph (DAG). The system includes a reasoning engine that uses the query DAG, the ontology manager and the detector API to plan an execution list of detectors. The reasoning engine uses the query DAG, a scene representation DAG produced by the external detectors and the ontology manager to answer the natural query.

Abstract: Systems and methods for detecting and correcting defective products include capturing at least one image of a product with at least one image sensor to generate an original image of the product. An encoder encodes portions of an image extracted from the original image to generate feature space vectors. A decoder decodes the feature space vectors to reconstruct the portions of the image into reconstructed portions by predicting defect-free structural features in each of the portions according to hidden layers trained to predict defect-free products. Each of the reconstructed portions are merged into a reconstructed image of a defect-free representation of the product. The reconstructed image is communicated to a contrastor to detect anomalies indicating defects in the product.

Abstract: A method is provided for model training to detect defective products. The method includes sampling training images of a product to (i) extract image portions therefrom made of a center patch and its context and (ii) black-out the center patch. The method further includes performing unsupervised back-propagation training of a Contextual Auto-Encoder (CAE) model using (i) the image portions with the blacked-out center patch as an input and, (ii) the center patch as a target output and, (iii) an image-based loss function, to obtain a trained CAE model. The method also includes sampling positive and negative center-patch-sized portions from the training images. The method additionally includes normalizing, using the trained CAE model, the positive and negative center-patch-sized portions.

Abstract: Systems and methods for diagnosing a patient condition include a medical imaging device for generating an anatomical image. A reconstructor reconstructs the anatomical image by reconstructing portions of the anatomical image to be a healthy representation of the portions and merging the portions into the anatomical image to generate a reconstructed image. A contrastor contrasts the anatomical image with the reconstructed image to generate an anomaly map indicating locations of difference between the anatomical image and the reconstructed image. An anomaly tagging device tags the locations of difference as anomalies corresponding to anatomical abnormalities in the anatomical image, and a display displays the anatomical image with tags corresponding to the anatomical abnormalities.

Abstract: A false alarm reduction system and method are provided for reducing false alarms in an automatic defect detection system. The false alarm reduction system includes a defect detection system, generating a list of image boxes marking detected potential defects in an input image. The false alarm reduction system further includes a feature extractor, transforming each of the image boxes in the list into a respective set of numerical features. The false alarm reduction system also includes a classifier, computing as a classification outcome for the each of the image boxes whether the detected potential defect is a true defect or a false alarm responsive to the respective set of numerical features for each of the image boxes.

Abstract: A false alarm reduction system is provided that includes a processor cropping each input image at randomly chosen positions to form cropped images of a same size at different scales in different contexts. The system further includes a CONDA-GMM, having a first and a second conditional deep autoencoder for respectively (i) taking each cropped image without a respective center block as input for measuring a discrepancy between a reconstructed and a target center block, and (ii) taking an entirety of cropped images with the target center block. The CONDA-GMM constructs density estimates based on reconstruction error features and low-dimensional embedding representations derived from image encodings. The processor determines an anomaly existence based on a prediction of a likelihood of the anomaly existing in a framework of a CGMM, given the context being a representation of the cropped image with the center block removed and having a discrepancy above a threshold.

Abstract: Methods and systems for detecting and correcting anomalies include predicting normal behavior of a monitored system based on training data that includes only sensor data collected during normal behavior of the monitored system. The predicted normal behavior is compared to recent sensor data to determine that the monitored system is behaving abnormally. A corrective action is performed responsive to the abnormal behavior to correct the abnormal behavior.

Abstract: Systems and methods for predicting new relationships in the knowledge graph, including embedding a partial triplet including a head entity description and a relationship or a tail entity description to produce a separate vector for each of the head, relationship, and tail. The vectors for the head entity, relationship, and tail entity can be combined into a first matrix, and adaptive kernels generated from the entity descriptions can be applied to the matrix through convolutions to produce a second matrix having a different dimension from the first matrix. An activation function can be applied to the second matrix to obtain non-negative feature maps, and max-pooling can be used over the feature maps to get subsamples. A fixed length vector, Z, flattens the subsampling feature maps into a feature vector, and a linear mapping method is used to map the feature vectors into a prediction score.

Abstract: Systems and methods for detecting and correcting defective products include capturing at least one image of a product with at least one image sensor to generate an original image of the product. An encoder encodes portions of an image extracted from the original image to generate feature space vectors. A decoder decodes the feature space vectors to reconstruct the portions of the image into reconstructed portions by predicting defect-free structural features in each of the portions according to hidden layers trained to predict defect-free products. Each of the reconstructed portions are merged into a reconstructed image of a defect-free representation of the product. The reconstructed image is communicated to a contrastor to detect anomalies indicating defects in the product.

Abstract: Systems and methods for diagnosing a patient condition include a medical imaging device for generating an anatomical image. A reconstructor reconstructs the anatomical image by reconstructing portions of the anatomical image to be a healthy representation of the portions and merging the portions into the anatomical image to generate a reconstructed image. A contrastor contrasts the anatomical image with the reconstructed image to generate an anomaly map indicating locations of difference between the anatomical image and the reconstructed image. An anomaly tagging device tags the locations of difference as anomalies corresponding to anatomical abnormalities in the anatomical image, and a display displays the anatomical image with tags corresponding to the anatomical abnormalities.