Abstract: Techniques are described for using machine learning (ML) models to create information technology (IT) infrastructures at a service provider network based on image of IT system architecture diagrams. To create IT system architecture diagrams, system architects often use tools ranging from pen and paper and whiteboards to various types of software-based drawing programs. Based on a user-provided image of an IT system architecture diagram (for example, a digital scan of a hand drawn system diagram, an image file created by a software-based drawing program, or the like), a service provider network uses one or more ML models to analyze the image to identify the constituent elements of the depicted IT system architecture and to create an infrastructure template that can be used to automatically provision corresponding computing resources at the service provider network.