Abstract: Various embodiments of the technology described herein generally relate to robotic systems for interacting with objects in a warehouse environment. More specifically, certain embodiments relate to systems and methods for collecting data related to robotic picking of objects through test interactions. In some embodiments, a robotic device may work in collaboration with a computer vision system for collecting data related to new objects in a warehouse, commercial, industrial, or similar environment. A robotic picking system may operate in a data collection mode during which objects are sent to a robotic picking device for data collection during one or more test interactions or test stimuli. The test interactions and stimuli may be used to produce a whitelist of objects that the robotic picking device may attempt to pick up during regular operation.