Abstract: A storage system communicatively coupled to a database management system (DBMS performs storage-side scanning of data sources that are not stored in native database storage format of the DBMS. Data sources for external tables are accessible in a storage system referred to as a distributed data access system (DDAS), e.g. a Hadoop Distributed File System. To execute a query that references an external table, a DBMS first generates an execution plan. The DDAS supplies the DBMS with information that specifies each portion of the data source, and specifies which data node to use to access the portion. The DBMS sends a request for each portion to the respective data node, requesting that the data node generate rows from data in the portion. The request may specify scanning criteria, specifying one or more columns to project and/or filter on, and code modules for the data node to execute to generate records.

Abstract: An approach, referred to herein as parallelized-external-table access, generates rows from a single external table in parallel for a given query. Under parallelized-external-table access, an execution plan generated for the query includes multiple work granules that generate rows for a single external table from a data source. Such work granules are referred to herein as external work granules. Each external work granule of the execution plan may be assigned to a slave process, which executes the external work granule in parallel with another slave process executing another external work granule. External tables are accessible on a cluster of data nodes in a distributed data access system (e.g. Hadoop Distributed File System) connected to a DBMS.

Abstract: A storage system communicatively coupled to a DBMS performs storage-side scanning of data sources that are not stored in the native database storage format of the DBMS. Data sources for external tables are accessible in a storage system referred to herein as a distributed data access system, e.g. a Hadoop Distributed File System. To execute a query that references an external table, a DBMS first generates an execution plan. The distributed data access system supplies the DBMS with information that specifies each portion of the data source, and specifies which data node to use to access the portion. The DBMS sends a request for each portion to the respective data node, the request requesting that the data node generate rows from data in the portion. The request may specify scanning criteria, specifying one or more columns to project and/or filter on. The request may also specify code modules for the data node to execute to generate rows or records and columns.

Abstract: An approach, referred to herein as parallelized-external-table access, generates rows from a single external table in parallel for a given query. Under parallelized-external-table access, an execution plan generated for the query includes multiple work granules that generate rows for a single external table from a data source. Such work granules are referred to herein as external work granules. Each external work granule of the execution plan may be assigned to a slave process, which executes the external work granule in parallel with another slave process executing another external work granule. External tables are accessible on a cluster of data nodes in a distributed data access system (e.g. Hadoop Distributed File System) connected to a DBMS.

Abstract: Techniques for transferring objects between database systems. A transfer is controlled by a master table in the RDBMS that is performing the transfer operation. The master table specifies the kind of transfer operation to be performed, a set of objects to be transferred, operations to be performed on the objects as they are being transferred, and filters for selecting a subset of the objects. During execution of the transfer, the transfer mechanism maintains and updates state in the master table and thereby makes it possible for the entity that is doing the transfer to determine the current status of the transfer and to restart the transfer after it has been stopped. The entity that is performing the transfer may also detach from the transfer without stopping the transfer and later again attach to the transfer.

Abstract: Techniques and systems are disclosed for storing instances of an opaque type in a database according to a direct path loading approach. According to one aspect, an opaque type implementor registers, with a loader application, routines that the opaque type implementor implements. In response, the loader application associates the opaque type with the routines. The loader application reads data that comprises instances of the opaque type. The loader application determines which routines are associated with the opaque type. The loader application invokes the routines, which create an array for storing instances of the opaque type and populate the array with values specified in the data. The loader application converts the array into a data stream that conforms to the format of the database's data blocks. The loader application then streams the data to a database server, which writes the data directly into data blocks in the database.

Abstract: Techniques for transferring objects between database systems. As implemented in a relational database management system, the techniques employ a data transfer mechanism that operates under control of a master table in the RDBMS that is performing the transfer operation. The master table specifies the kind of transfer operation to be performed, a set of objects to be transferred, operations to be performed on the objects as they are being transferred, and filters for selecting a subset of the objects. During execution of the transfer, the transfer mechanism maintains and updates state in the master table such that queries may be made on the master table to determine the current status of the operation and such that the transfer mechanism may restart the operation after it has been stopped either at the request of a client that is performing the operation or because of an error in the transfer.

Abstract: Techniques and systems are disclosed for storing instances of an opaque type in a database according to a direct path loading approach. According to one aspect, an opaque type implementor registers, with a loader application, routines that the opaque type implementor implements. In response, the loader application associates the opaque type with the routines. The loader application reads data that comprises instances of the opaque type. The loader application determines which routines are associated with the opaque type. The loader application invokes the routines, which create an array for storing instances of the opaque type and populate the array with values specified in the data. The loader application converts the array into a data stream that conforms to the format of the database's data blocks. The loader application then streams the data to a database server, which writes the data directly into data blocks in the database.