Abstract: Performing a “cold start” of a robotic stacking operation is disclosed. In various embodiments, estimated state information representing an estimated state of one or both of a receptacle and one or more objects stacked on or in the receptacle is stored. An indication is received that the estimated state information is not suitable to make a next placement decision with respect to a next object to be stacked on or in the receptacle. Constructed estimated state information is generated at least in part by processing sensor information generated by one or more sensors positioned and configured to generate sensor information providing an at least partial view of one or both of the receptacle and the one or more objects stacked on or in the receptacle.