Abstract: The present disclosure provides a method for interconnecting a data lake and a relational database, including the following steps: S1: adding a data source class of a relational database to a data lake; S2: matching and using, by the data lake, a data source class of the relational database; and S3: determining and loading a corresponding driver according to the data source class, so as to connect the corresponding relational database. By cascading a data source registering configuration file, a relational database configuration file and a driver package catalog in a parameter passing method, when the data lake is started, a specific database to be used is designated unnecessarily, but a corresponding database is used directly. The configuration file is also traversed unnecessarily, but the user acquires configuration information as required in the parameter passing method.

Abstract: The present disclosure provides a high-performance data lake system and a data storage method. The data storage method includes the following steps: S1: converting a file into a file stream; S2: converting the file stream into an array in which multiple subarrays are nested; and S3: converting the array into a resilient distributed dataset (RDD), and storing the RDD to a storage layer of a data lake. The present disclosure provides a nested field structure, which lays the foundation for parallel processing in reading, and effectively improves read performance. Furthermore, the present disclosure flexibly generates a number of nested subarrays according to hardware cores, such that the data lake achieves better extension performance, and can keep optimal writing efficiency for different users.

Abstract: The present disclosure provides a method for interconnecting a data lake and a relational database, including the following steps: S1: adding a data source class of a relational database to a data lake; S2: matching and using, by the data lake, a data source class of the relational database; and S3: determining and loading a corresponding driver according to the data source class, so as to connect the corresponding relational database. By cascading a data source registering configuration file, a relational database configuration file and a driver package catalog in a parameter passing method, when the data lake is started, a specific database to be used is designated unnecessarily, but a corresponding database is used directly. The configuration file is also traversed unnecessarily, but the user acquires configuration information as required in the parameter passing method.

Abstract: The present disclosure provides a high-performance data lake system and a data storage method. The data storage method includes the following steps: S1: converting a file into a file stream; S2: converting the file stream into an array in which multiple subarrays are nested; and S3: converting the array into a resilient distributed dataset (RDD), and storing the RDD to a storage layer of a data lake. The present disclosure provides a nested field structure, which lays the foundation for parallel processing in reading, and effectively improves read performance. Furthermore, the present disclosure flexibly generates a number of nested subarrays according to hardware cores, such that the data lake achieves better extension performance, and can keep optimal writing efficiency for different users.