Abstract: A computerized mechanism to formulate a data flow representation from a syntax tree. The syntax tree may be, for instance, an abstract syntax tree (AST) that is formulated by compiling query script, such as big data query script. Each node in the syntax tree is composed of one or more tokens (script portions) that are taken from the query script, which relationships between the tokens being syntactically represented by links between those tokens. Accordingly, the data flow representation may also be a data flow representation of the original query script itself. In order to formulate the data flow representation from the syntax tree, the data types of the various inputs and outputs of the syntax tree nodes are identified and bound to a corresponding data flow. This may be performed whilst honoring dependencies between the nodes. The data flow may be visualized to an author or reviewer of the script.

Abstract: A computerized mechanism to formulate a data flow representation from a syntax tree. The syntax tree may be, for instance, an abstract syntax tree (AST) that is formulated by compiling query script, such as big data query script. Each node in the syntax tree is composed of one or more tokens (script portions) that are taken from the query script, which relationships between the tokens being syntactically represented by links between those tokens. Accordingly, the data flow representation may also be a data flow representation of the original query script itself. In order to formulate the data flow representation from the syntax tree, the data types of the various inputs and outputs of the syntax tree nodes are identified and bound to a corresponding data flow. This may be performed whilst honoring dependencies between the nodes. The data flow may be visualized to an author or reviewer of the script.

Abstract: A computerized mechanism to automatically correlate positions of query script to portions of a data flow representation of the query script. When parsing the query script to generate the tokens, at least some of the tokens have an associated script location marker that identifies a location in the query script where the token originated from. The syntax tree of multiple nodes is then formulated, each node comprising one or more of the tokens parsed from the query script. Accordingly, the syntax tree retains the script location markers. A data flow representation of the query script is then formulated into a data flow representation. That data flow representation might, for instance, be based on the syntax tree, but augmented with data types of the various data flows. Nevertheless, the location marker is retained within the data flow representation.