Abstract: An autonomous mobile device (AMD) may move around a physical space while performing tasks. Sensor data is used to determine an occupancy map of the physical space. Some objects within the physical space may be difficult to detect because of characteristics that result in lower confidence in sensor data, such as transparent or reflective objects. To include difficult-to-detect objects in the occupancy map, image data is processed to identify portions of the image that includes features associated with difficult-to-detect objects. Given the portion that possibly includes difficult-to-detect objects, the AMD attempts to determine where in the physical space that portion corresponds to. For example, the AMD may use stereovision to determine the physical area associated with the features depicted in the portion. Objects in that area are included in an occupancy map annotated as objects that should persist unless confirmed to not be within the physical space.