Abstract: A system includes an execution engine and a processor. The execution engine receives application code that, when executed, is configured to generate a second data element from a first data element that is stored in a first database and store the second element in a second database. The execution engine converts the code into an execution plan and executes it. The execution plan includes a first operation to obtain the first element from the first database, a second operation to apply a transformation to the first element, and a third operation to store the first element as the second element in the second database. The processor accesses the execution plan and determines that the third operation includes storing the second element. In response, the processor generates a data lineage for the second element by extracting, from each operation from the third to the first, a portion of the data lineage.

Abstract: A system includes an execution engine and a processor. The execution engine receives application code that, when executed, is configured to generate a second data element from a first data element that is stored in a first database and store the second element in a second database. The execution engine converts the code into an execution plan and executes it. The execution plan includes a first operation to obtain the first element from the first database, a second operation to apply a transformation to the first element, and a third operation to store the first element as the second element in the second database. The processor accesses the execution plan and determines that the third operation includes storing the second element. In response, the processor generates a data lineage for the second element by extracting, from each operation from the third to the first, a portion of the data lineage.

Abstract: Systems for dynamically transforming data are provided. Database data may be received and ingested into a system. Ingesting the data may include executing one or more first data governance functions, such as data quality evaluation functions, data controls, and the like. The ingested data may then be output for further processing as first processed data and first data governance information may be captured and stored. The first processed data may be processed to execute one or more data transformations. Data transformations may include calculations, formatting, derivations, and the like. In some arrangements, second data governance functions may be executed on the transformed data. The transformed data may then be output as second processed data. The system may capture second data governance information as the data is transformed. The second processed data may then be published to one or more downstream databases for use in one or more applications executed by an entity.

Abstract: Systems for dynamically transforming data are provided. Database data may be received and ingested into a system. Ingesting the data may include executing one or more first data governance functions, such as data quality evaluation functions, data controls, and the like. The ingested data may then be output for further processing as first processed data and first data governance information may be captured and stored. The first processed data may be processed to execute one or more data transformations. Data transformations may include calculations, formatting, derivations, and the like. In some arrangements, second data governance functions may be executed on the transformed data. The transformed data may then be output as second processed data. The system may capture second data governance information as the data is transformed. The second processed data may then be published to one or more downstream databases for use in one or more applications executed by an entity.