Abstract: Described is a system and method for robotic perception and action. The system includes a mobile autonomous platform programmed to receive a visual input related to an environment and perform a movement sequence that affects the environment based on the visual input. A neural network is in communication with the mobile autonomous platform. The neural network includes a vision autoencoder trained, based on movements of the mobile autonomous platform, to encode visual features of an object in the visual input onto a visual latent space. In addition, the neural network includes an action autoencoder trained, based on movements of the mobile autonomous platform, to encode action features of the movement sequence onto an action latent space. The neural network also includes a mapping layer between the vision autoencoder and the action autoencoder. The mapping layer decodes the visual inputs into an action output of the mobile autonomous platform.

Abstract: Described is a system for detecting and classifying new patterns of objects and images for applications where labeled data is scarce. In operation, the system trains a neural network with unlabeled images and extracts features with the neural network from both the unlabeled images and a set of labeled images to generate a feature space. Labels are propagated in the feature space using nearest neighbors, allowing for modeling of a per-class simplified distribution. An object in a new test image can then be classified using reconstruction error based on the per-class simplified distributions.

Abstract: Described is a system for pattern recognition and graph extraction. The system operates by obtaining data from a quantum dot device having gate defined semiconductor quantum dots. An image with pixel-level coordinates is then generated from the data. Ground truth graph annotations are received of the image. A deep network processes the image and pixel-level coordinates to generate a predicted graph. The predicted graph is compared to the graph annotations to generate a loss, which allows the deep network to be optimized by updating parameters in the deep network based on the loss. The image with pixel-level coordinates is then used to generate an optimized predicted graph, which allows for identifying operational voltages to apply to the quantum dot device.

Abstract: A method, system and computer program product are provided for determining wire contact insertion based on image analysis. Methods include: acquiring at least one image of a connector having a plurality of wire contact insertion holes; identifying a wire contact within a wire contact insertion hole of the plurality of wire contact insertion holes; determining that the wire contact insertion hole is a correct wire contact insertion hole; determining that the wire contact has been fully inserted into the correct wire contact insertion hole; and providing feedback indicating that the wire contact is fully inserted into the correct wire contact insertion hole. The feedback includes in some cases a first indicator indicating that the wire contact is in the correct wire contact insertion hole and a second indicator indicating that the wire contact is fully inserted into the correct wire contact insertion hole.

Abstract: Described is a system for optimizing controllers for micro-air vehicles. The system identifies binary rules that lead to a desired behavior of a micro-air vehicle. The binary rules are identified by sampling rule-sets from a probability distribution over a search space. A fitness value of each rule-set for the desired behavior is determined, and top-performing rule-sets are selected. The top-performing rule-sets are used to update the probability distribution over the search space until a convergence criterion is met, resulting in a fittest rule-set. A control signal based on the identified binary rules is transmitted to one or more actuators of the micro-air vehicle.

Abstract: A method, system and computer program product are provided for automated insertion of a wire contact into an insertion hole of a connector. Methods include: controlling a robot having an end-effector to position a wire contact proximate to a connector using a wire gripper and a separator device of the end-effector; controlling the robot to advance the separator device between two or more wires previously connected to the connector; controlling the robot to align the wire contact with a insertion hole of the connector; controlling the robot to advance the wire contact toward the insertion hole of the connector and at least partially insert the wire contact into the insertion hole; controlling the robot to release the wire contact from the wire gripper; and controlling the robot to withdraw the wire gripper and the separator device from between the two or more wires previously connected to the connector.

Abstract: Examples for detection of defects in an additively manufactured object are provided. In one aspect, a method is provided. The method comprises receiving in-situ spectral data measured from the additively manufactured object during an additive manufacturing process, constructing a graph data structure using the in-situ spectral data, and outputting a predicted defect region using the graph data structure and a trained graph-learning neural network.

Abstract: A method, apparatus, system, and computer program product for positions a wire contact. A sequence of images of a wire contact is generated while the wire contact moves from a first position to a second position. The sequence of images is generated by the camera system connected to the end effector and the wire contact is held by the end effector. Edges are detected in the sequence of images to form edge images. Background edges are removed from the edges in the edge images leaving contact edges in the edges for the wire contact to form a contact edge image. The wire contact is identified using the contact edges in the contact edge image. A pose of the wire contact is determined from the wire contact identified in the contact edge image.

Abstract: Described is a system for object detection that is robust to adversarial attacks. An initial hypothesis of an identity of an object in an input image is generated using a sparse convolutional neural network (CNN) and a distribution aware classifier. A foveated hypothesis verification process is performed for identifying a region of the input image that supports the initial hypothesis. Using a part-based classifier, an identity of a part of the object in the region of the input image is predicted. An attack probability for the predicted identity of the part, and the initial hypothesis is updated based on the predicted identity of the part and the attack probability. The foveated hypothesis verification process and updating of hypotheses is performed until a hypothesis reaches a certainty threshold. The object is labeled based on the hypothesis that reached the certainty threshold.

Abstract: A method, system and computer program product are provided for determining wire contact insertion based on image analysis. Methods include: acquiring at least one image of a connector having a plurality of wire contact insertion holes; identifying a wire contact within a wire contact insertion hole of the plurality of wire contact insertion holes; determining that the wire contact insertion hole is a correct wire contact insertion hole; determining that the wire contact has been fully inserted into the correct wire contact insertion hole; and providing feedback indicating that the wire contact is fully inserted into the correct wire contact insertion hole. The feedback includes in some cases a first indicator indicating that the wire contact is in the correct wire contact insertion hole and a second indicator indicating that the wire contact is fully inserted into the correct wire contact insertion hole.

Abstract: A method, system and computer program product are provided for automated insertion of a wire contact into an insertion hole of a connector. Methods include: controlling a robot having an end-effector to position a wire contact proximate to a connector using a wire gripper and a separator device of the end-effector; controlling the robot to advance the separator device between two or more wires previously connected to the connector; controlling the robot to align the wire contact with a insertion hole of the connector; controlling the robot to advance the wire contact toward the insertion hole of the connector and at least partially insert the wire contact into the insertion hole; controlling the robot to release the wire contact from the wire gripper; and controlling the robot to withdraw the wire gripper and the separator device from between the two or more wires previously connected to the connector.

Abstract: Described is a system for automatically identifying chemical properties of a molecule. A chemical representation of a molecular structure is converted into atomic features and an adjacency matrix. The atomic features and the adjacency matrix are processed with a neural network, resulting in neural activations corresponding to each atom in the molecular structure. The system determines a probability for each atom quantifying its relevance for a given chemical characteristic. The probabilities are displayed as a graphical representation on the molecular structure, and groups of atoms are identified for the given chemical characteristic from the graphical representation. The identified groups of atoms for the given chemical characteristic are stored in a database, and a new molecule having the given chemical characteristic is designed based on the stored identified groups of atoms.

Abstract: A method, system and computer program product are provided for correction of automated insertion of a wire contact into a target hole of a connector. Methods include controlling a robot having an end-effector to: align the wire contact with the target hole of the connector; advance the wire contact toward and into the target hole of the connector; cease insertion in response to a force between the wire contact and the connector exceeding a predefined value; determining a depth of insertion; in response to the depth of insertion being above a predefined depth, perform a pull test on the inserted wire contact; in response to the depth of insertion being below a predetermined depth, identify an error condition using visual feedback; determine a number of corrective operations performed and perform error correction if the number is below a predefined number, while withdrawing the wire contact otherwise.

Abstract: Described is a system for learning object labels for control of an autonomous platform. Pseudo-task optimization is performed to identify an optimal pseudo-task for each source model of one or more source models. An initial target network is trained using the optimal pseudo-task. Source image components are extracted from source models, and an attribute dictionary of attributes is generated from the source image components. Using zero-shot attribution distillation, the unlabeled target data is aligned with the source models similar to the unlabeled target data. The unlabeled target data are mapped onto attributes in the attribute dictionary. A new target network is generated from the mapping, and the new target network is used to assign an object label to an object in the unlabeled target data. The autonomous platform is controlled based on the object label.

Abstract: Described is a system and method for the classification of agents based on agent movement patterns. In operation, the system receives position data of a moving agent from a camera or sensor. Motion data of the moving agent is then extracted and used to generate a predicted future motion of the moving agent using a set of pre-calculated Echo State Networks (ESN). Each ESN represents an agent classification and generates a predicted future motion. A prediction error is generated for each ESN by comparing the predicted future motion for each ESN with actual motion data. Finally, the agent is classified based on the ESN having the smallest prediction error.

Abstract: Described is a system for detecting backdoor attacks in deep convolutional neural networks (CNNs). The system compiles specifications of a pretrained CNN into an executable model, resulting in a compiled model. A set of Universal Litmus Patterns (ULPs) are fed through the compiled model, resulting in a set of model outputs. The set of model outputs are classified and used to determine presence of a backdoor attack in the pretrained CNN. The system performs a response based on the presence of the backdoor attack.

Abstract: A method, system and computer program product are provided for aligning and inserting a wire contact within a target hole of a connector to facilitate the automated insertion of the wire ends of a wire bundle assembly into the wire contact insertion holes of a connector. Methods may include: obtaining captured images, from at least two image capture devices attached to an end-effector of a robot, of a wire gripper of the end-effector; causing the robot to advance the end-effector to move the wire contact within a predetermined distance of the connector; causing the robot to advance the end-effector to move the wire contact toward the connector a predetermined additional amount more; and identifying, based on movement of the wire contact the predetermined additional amount more, if alignment is correct from force feedback at the wire gripper.

Abstract: Described is a system for health assessment. The system is implemented on a mobile device having at least one of an accelerometer, a geographic location sensor, and a camera. In operation, the system obtains sensor data related to an operator of the mobile device from one of the sensors. A network of networks (NoN) is generated based on the sensor data, the NoN having a plurality of layers with linked nodes. Tuples are thereafter generated. Each tuple contains a node from each layer that optimizes importance, diversity, and coherence. Storylines are created based on the tuples that solves a longest path problem for each tuple. The storylines track multiple symptom progressions of the operator. Finally, a disease prediction of the operator is provided based on the storylines.

Abstract: Described is a system for learning actions for image-based action recognition in an autonomous vehicle. The system separates a set of labeled action image data from a source domain into components. The components are mapped onto a set of action patterns, thereby creating a dictionary of action patterns. For each action in the set of labeled action data, a mapping is learned from the action pattern representing the action onto a class label for the action. The system then maps a set of new unlabeled target action image data onto a shared embedding feature space in which action patterns can be discriminated. For each target action in the set of new unlabeled target action image data, a class label for the target action is identified. Based on the identified class label, the autonomous vehicle is caused to perform a vehicle maneuver corresponding to the identified class label.

Abstract: Described is a system for action recognition through application of deep embedded clustering. For each image frame of an input video, the system computes skeletal joint-based pose features representing an action of a human in the image frame. Non-linear mapping of the pose features into an embedded action space is performed. Temporal classification of the action is performed and a set of categorical gesture-based labels is obtained. The set of categorical gesture-based labels is used to control movement of a machine.