Abstract: Techniques are described for generating a map of operations to ingest external data. In an embodiment, DBMS receives a query schema that specifies an external data source and client-defined columns to arrange queried data from data fields of the external data source. DBMS generates an ingest schema for the queried data of the external data source such that the ingest schema specifies ingest data fields from the data fields of the external data source to be mapped to the one or more client-defined columns. Additionally, the DBMS identifies surplus data in the external data source, which has not been specified by the ingest schema and is different from the queried data defined by the client-requested query schema. The process not only ingests the ingest data fields from the external data source but also the surplus data from the external data source. In response to receiving a query according to a different query schema, DBMS generates a result set that based on the surplus data of the external data source.

Abstract: Techniques are described for generating a map of operations to ingest external data. In an embodiment, DBMS receives a query schema that specifies an external data source and client-defined columns to arrange queried data from data fields of the external data source. DBMS generates an ingest schema for the queried data of the external data source such that the ingest schema specifies ingest data fields from the data fields of the external data source to be mapped to the one or more client-defined columns. Additionally, the DBMS identifies surplus data in the external data source, which has not been specified by the ingest schema and is different from the queried data defined by the client-requested query schema. The process not only ingests the ingest data fields from the external data source but also the surplus data from the external data source. In response to receiving a query according to a different query schema, DBMS generates a result set that based on the surplus data of the external data source.

Abstract: According to one or more embodiments, multiple related data streams are compressed jointly, such that substantive similarities between the multiple related streams are leveraged to reduce the overall size of the resulting compressed data. Specifically, given the compressed version of a primary data stream, a secondary data stream may be highly compressed by utilizing portions of the compressed version of the primary data stream to represent similar portions of the secondary data stream. A compression application is configured to receive (either concurrently or separately) information identifying multiple related input streams. For each input stream provided to the compression application, the application outputs one output stream. The computing system writes the output streams of the compression application to storage. Embodiments allow the size of the compressed version of a secondary data stream to be greatly reduced compared to the size of the stream compressed without reference to a primary data stream.

Abstract: According to one or more embodiments, multiple related data streams are compressed jointly, such that substantive similarities between the multiple related streams are leveraged to reduce the overall size of the resulting compressed data. Specifically, given the compressed version of a primary data stream, a secondary data stream may be highly compressed by utilizing portions of the compressed version of the primary data stream to represent similar portions of the secondary data stream. A compression application is configured to receive (either concurrently or separately) information identifying multiple related input streams. For each input stream provided to the compression application, the application outputs one output stream. The computing system writes the output streams of the compression application to storage. Embodiments allow the size of the compressed version of a secondary data stream to be greatly reduced compared to the size of the stream compressed without reference to a primary data stream.