Abstract: The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. According to one embodiment, a method includes: determining one or more logical relationships between textual elements and non-textual elements of one or more images of a user interface; building a hierarchy comprising some or all of the non-textual elements and some or all of the textual elements in order to form a data structure representing functionality of the user interface; and outputting the data structure to a memory.

Abstract: The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. According to one embodiment, a method includes: determining one or more logical relationships between textual elements and non-textual elements of one or more images of a user interface; building a hierarchy comprising some or all of the non-textual elements and some or all of the textual elements in order to form a data structure representing functionality of the user interface; and outputting the data structure to a memory.

Abstract: The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. According to one embodiment, a method includes: determining one or more logical relationships between textual elements and non-textual elements of one or more images of a user interface; building a hierarchy comprising some or all of the non-textual elements and some or all of the textual elements in order to form a data structure representing functionality of the user interface; and outputting the data structure to a memory.

Abstract: The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. The inventive computer program products include program instructions configured to cause a computer, upon execution thereof, to perform a method including: identifying, from among a plurality of elements within one or more images of a user interface, a first set of elements and a second set of elements, wherein the first set of elements and the second set of elements each independently comprise either or both of: textual elements, and non-textual elements; determining one or more logical relationships between the textual elements and the non-textual elements; building a hierarchy comprising some or all of the first set of elements and some or all of the second set of elements, wherein building the hierarchy forms a data structure representing functionality of the user interface; and outputting the data structure to a memory.

Abstract: The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. The inventive computer program products include program instructions configured to cause a computer, upon execution thereof, to perform a method including: identifying, from among a plurality of elements within one or more images of a user interface, a first set of elements and a second set of elements, wherein the first set of elements and the second set of elements each independently comprise either or both of: textual elements, and non-textual elements; determining one or more logical relationships between the textual elements and the non-textual elements; building a hierarchy comprising some or all of the first set of elements and some or all of the second set of elements, wherein building the hierarchy forms a data structure representing functionality of the user interface; and outputting the data structure to a memory.

Abstract: The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. The inventive techniques include: identifying first and second sets of elements within one or more images of a user interface, where each of the first set of elements is independently selected from: vertical lines, horizontal lines, and rectangular elements, and each of the second set of elements is independently selected from: radio buttons, icons, and textual elements. The methods also include determining one or more logical relationships between the textual elements and non-textual elements of the image; building a hierarchy comprising some or all of the first set of elements and some or all of the second set of elements in order to form a tree-based data structure representing functionality of the user interface; and outputting the tree-based data structure to a memory.

Abstract: The presently disclosed inventive concepts are directed to systems, computer program products, and methods for intelligent screen automation. The inventive techniques include: identifying first and second sets of elements within one or more images of a user interface, where each of the first set of elements is independently selected from: vertical lines, horizontal lines, and rectangular elements, and each of the second set of elements is independently selected from: radio buttons, icons, and textual elements. The methods also include determining one or more logical relationships between the textual elements and non-textual elements of the image; building a hierarchy comprising some or all of the first set of elements and some or all of the second set of elements in order to form a tree-based data structure representing functionality of the user interface; and outputting the tree-based data structure to a memory.