Abstract: A computer-based system may be configured to collect metadata for each source and target defined for a data pipeline and formatting information (e.g., schemas, transformations, etc.) associated with each entity and field. During the definition of the pipeline, how the data will end up in the target may be defined, for example, by a user of the computer-based system via a GUI/interface and/or the like. Information (e.g., modification information, etc.) describing how the data will end up in the target may be defined, stored, and accessed to determine and/or track over which fields and entities are affected by the user-defined mutations and over which schemas. Lineage information (e.g., a genealogical tree, data lineage tracing, etc.) describing a data, version, and transformation may be generated and used to determine a source for a data record, how changes to the data record are related, how lineage evolved, and/or the like.

Abstract: A computer-based system may be configured to collect metadata and/or the like indicative of all the metrics exposed from a data pipeline (e.g., an ETL pipeline, etc.) and transform the metrics into a single group of metrics user-facing, user-specific, user-configured, and/or the like metrics that allow the maturity and quality of data and/or data records to be analyzed and/or displayed. Collected metrics can be agnostic of a source data flow component of a data pipeline and/or resource technology (e.g., API, etc.). Collected metrics may indicate a measure of data freshness, data duplication, new data records, updated date data records, data errors, and/or the like.

Abstract: Methods and apparatus for marshalling markup language objects into programming language objects in a scripting language runtime environment. A markup language integration method may be implemented in scripting runtime environments that allows the use of simple and complex markup language structures in scripts. At runtime of a script, only classes for the schema structures that are referenced by the script are generated. Document Object Model (DOM) objects may be incrementally constructed according to the schema structures referenced by the script, and bytecode may be generated corresponding to the schema structures according to class metadata for the schema structures. The bytecode may be dynamically linked in the runtime environment. The classes defined by the schema act as wrappers for the DOM objects; when an object is updated by the script, a corresponding DOM object is also updated. Type checking may be applied to catch script errors during development.

Abstract: Methods and apparatus for marshalling markup language objects into programming language objects in a scripting language runtime environment. A markup language integration method may be implemented in scripting runtime environments that allows the use of simple and complex markup language structures in scripts. At runtime of a script, only classes for the schema structures that are referenced by the script are generated. Document Object Model (DOM) objects may be incrementally constructed according to the schema structures referenced by the script, and bytecode may be generated corresponding to the schema structures according to class metadata for the schema structures. The bytecode may be dynamically linked in the runtime environment. The classes defined by the schema act as wrappers for the DOM objects; when an object is updated by the script, a corresponding DOM object is also updated. Type checking may be applied to catch script errors during development.

Abstract: A computer-implemented method can include generating Business Process Modeling Notation (BPMN) scripts for one or more BPMN processes, and storing each BPMN script in a class oriented format in a file that is separate from the corresponding BPMN process.