Abstract: Techniques are disclosed for automating GUI development from a GUI screen image that includes text information and one or more graphic user interface components. The GUI screen image is analyzed to extract text information and to identify the UI components included in the GUI screen. One or more text regions in the GUI screen image are detected and are replaced with placeholders. Images of one or more graphic user interface components in the GUI screen are extracted from the GUI screen image and are classified using a machine learning-based classifier. A GUI model is generated for the GUI based upon the classification results, locations of the one or more text regions, and locations of the one or more graphic user interface components. The generated model can then be used to generate one or more implementations (e.g., executable code) of the GUI, possibly for various platforms in different programming languages.

Abstract: Techniques are disclosed for automating GUI development from a GUI screen image that includes text information and one or more graphic user interface components. The GUI screen image is analyzed to extract text information and to identify the UI components included in the GUI screen. One or more text regions in the GUI screen image are detected and are replaced with placeholders. Images of one or more graphic user interface components in the GUI screen are extracted from the GUI screen image and are classified using a machine learning-based classifier. A GUI model is generated for the GUI based upon the classification results, locations of the one or more text regions, and locations of the one or more graphic user interface components. The generated model can then be used to generate one or more implementations (e.g., executable code) of the GUI, possibly for various platforms in different programming languages.