Abstract: An all-optical hologram reconstruction system and method is disclosed that can instantly retrieve the image of an unknown object from its in-line hologram and eliminate twin-image artifacts without using a digital processor or a computer. Multiple transmissive diffractive layers are trained using deep learning so that the diffracted light from an arbitrary input hologram is processed all-optically to reconstruct the image of an unknown object at the speed of light propagation and without the need for any external power. This passive diffractive optical network, which successfully generalizes to reconstruct in-line holograms of unknown, new objects and exhibits improved diffraction efficiency as well as extended depth-of-field at the hologram recording distance. The system and method can find numerous applications in coherent imaging and holographic display-related applications owing to its major advantages in terms of image reconstruction speed and computer-free operation.