Abstract: Described is system for identifying functional elements of objects for robotic manipulation. The system causes a robot to manipulate an object, the robot having an audio sensor and touch sensors. A three-dimensional (3D) location of each audio event that produces a response during manipulation of the object is recorded. Additionally, a 3D location of each tactile event that produces a response during manipulation of the object is recorded. A 3D location of each audio and tactile event in 3D space is then determined. A 3D audio point cloud and a 3D tactile point cloud are generated. Then, the 3D audio point cloud and the 3D tactile point cloud are registered with a 3D model point cloud of the object. Finally, an annotated 3D model point cloud of the object is generated, which encodes the location of a functional element of the object.

Abstract: A method of manufacturing an article includes the steps of: establishing a defined location for a tool activity to occur; determining an actual location of the tool; comparing the actual location of the tool with the defined location; ascertaining an occurrence of the tool activity at the actual location; and verifying whether the tool activity occurred at the defined location.