Abstract: A comprehensive/unified platform for data integration/processing including building/generating and managing data applications is disclosed. The data integration platform takes a modular approach such that new functionality can easily be added to the platform as future needs dictate. In this regard, the data integration platform includes modules for data ingestion, data transformation, data loading/storage, data analytics and the like that provide end-to-end functionality. The data integration platform provides for unified data operations capabilities, including data governance, data quality control, data control and data lineage. The data integration platform may leverage the use AI including ML, such as generative AI to facilitate the capturing of metadata, enhancing/improving captured data, and assisting in data analytics.