Abstract: Systems, methods, and devices for incrementally refreshing a materialized view are disclosed. A method includes generating a materialized view based on a source table. The method includes merging the source table and the materialized view to generate a merged table to identify whether an update has been executed on the source table that is not reflected in the materialized view. The method includes, in response to detecting an update made to the source table that is not reflected in the materialized view, applying the update to the materialized view.

Abstract: The subject technology serializes, by at least one hardware processor, non-primary key data of column-organized data into compressed serialized value data that is in a row-organized sequence, the compressed serialized value data compressed using at least one bitmap, the non-primary key data comprising a schema identifier, the column-organized data being stored in a columnar database system, the column-organized data comprising primary key data and the non-primary key data. The subject technology stores the compressed serialized value data in a key-value data store of a key-value database system, the key-value database system processing key-value data in a key-value format. The subject technology receives a query by the columnar database system. The subject technology deserializes a portion of the compressed serialized value data that corresponds to the query. The subject technology processes the query using the columnar database system.

Abstract: Systems, methods, and devices for retrying a query. A method includes receiving a query directed to database data and assigning execution of the query to one or more execution nodes of an execution platform, the one or more execution nodes configured to execute the query on a first version of a database platform. The method includes determining that execution of the query was unsuccessful. The method includes assigning a first retry execution of the query to the one or more execution nodes of the execution platform and determining whether a regression or an intermittent fault caused the execution of the query to be unsuccessful based at least in part on whether the first retry execution of the query was successful or unsuccessful.

Abstract: A method includes partitioning a database table into a first plurality of partitions. A plurality of changes are executed on the database table in response to a transaction. The changes occur at a corresponding plurality of timestamps and result in a second plurality of partitions. The database table is updated to include a log with the plurality of changes. For each change of the plurality of changes, the log includes and an identification of a portion of the database table the change is applied to. A selection of a first timestamp and a second timestamp of the plurality of timestamps is detected. A delta is generated in response to the selection. The delta indicates a total change occurring to the database table between a first change of the plurality of changes corresponding to the first timestamp and a second change of the plurality of changes corresponding to the second timestamp.

Abstract: Example resource management systems and methods are described. In one implementation, a resource manager is configured to manage data processing tasks associated with multiple data elements. An execution platform is coupled to the resource manager and includes multiple execution nodes configured to store data retrieved from multiple remote storage devices. Each execution node includes a cache and a processor, where the cache and processor are independent of the remote storage devices. A metadata manager is configured to access metadata associated with at least a portion of the multiple data elements.

Abstract: Example resource management systems and methods are described. In one implementation, a resource manager is configured to manage data processing tasks associated with multiple data elements. An execution platform is coupled to the resource manager and includes multiple execution nodes configured to store data retrieved from multiple remote storage devices. Each execution node includes a cache and a processor, where the cache and processor are independent of the remote storage devices. A metadata manager is configured to access metadata associated with at least a portion of the multiple data elements.

Abstract: Embodiments of the present disclosure describe techniques for monitoring usage of data listings and pricing data listings based thereon. Metadata comprising a pricing plan may be added to a data listing to generate a monetized data listing. The monetized data listing may be imported to the consumer account. The usage data of the monetized data listing and listing data of the monetized data listing may be exported to a data warehousing and analysis module. A usage level for a set of jobs processed using the monetized data listing during a current incremental interval of a current billing interval may be calculated and combined with a usage level of all previous incremental intervals of the current billing interval to generate a cumulative usage record. An invoice for the billing interval may be generated based on the cumulative usage record.

Abstract: Systems, methods, and devices for sharing materialized views in multiple tenant database systems. A method includes defining a materialized view over a source table that is associated with a first account of a multiple tenant database. The method includes defining cross-account access rights to the materialized view to a second account such that that second account can read the materialized view without copying the materialized view. The method includes modifying the source table for the materialized view. The method includes identifying whether the materialized view is stale with respect to the source table by merging the materialized view and the source table.

Abstract: A method, apparatus, and system for join operations of a plurality of relations that are distributed over a plurality of storage locations over a network of computing components.

Abstract: A data platform for managing an application as a first-class database object. The data platform includes at least one processor and a memory storing instructions that cause the at least one processor to perform operations including detecting a data request from a browser for a data object located on the data platform, executing a stored procedure, the stored procedure containing instructions that cause the at least one processor to perform additional operations including instantiating a User Defined Function (UDF) server, an application engine, and the application within a security context of the data platform based on a security policy determined by an owner of the data object. The data platform then communicates with the browser using the application engine as a proxy server.

Abstract: Techniques for schema mismatch detection and evolution are described. When data is being uploaded into a source table, schema of the data to be uploaded can be compared with the schema for the source table. If a schema mismatch is detected, the schema of the source table can be modified, and the upload can be continued without data loss.

Abstract: The subject technology receives a command to commit a table in a different table format on an external volume. The subject technology generates a first snapshot of the table on internal storage. The subject technology generates a first list of metadata files on the internal storage. The subject technology generates, based on the first list of metadata files, a first set of metadata files on the internal storage. The subject technology generates a second snapshot of the table on the external volume. The subject technology generates a second list of metadata files on the external volume. The subject technology generates, based on the second list of metadata files, a second set of metadata files on the external volume. The subject technology generates a first set of data files in a different file format on the external volume.

Abstract: Systems and methods for managing membership in a private data exchange are provided herein. In one embodiment, the method includes generating a plurality of listings in a data exchange, each of the plurality of listings comprising version metadata and referencing a data set within a database. The method further includes receiving a request for access to a first listing of the plurality of listings and, in response to the request, accessing a first version of the data set that is compatible with first version metadata of the first listing.

Abstract: In an embodiment, a database platform receives a query that is directed at least in part to external data in an external table stored on a data storage platform that is external to the database platform. The external table includes a plurality of partitions. The database platform identifies, from external-table metadata, a subset of the plurality of partitions of the external table as including data that potentially satisfies the query. The external-table metadata is stored by the database platform. The database platform identifies data that satisfies the query by scanning the identified subset of the partitions, and responds to the query at least in part with the identified data that satisfies the query.

Abstract: A network device communication system can configure network devices, such as a first database in a multi-tenant deployment and a second database in a private deployment, to send and receive sequence messages, such as replication data, over a channel comprising a plurality of private network nodes. The first database can create a link specifying the data share and the second database. The second database selects the link and a secure area in the private deployment is created into which data is replicated and shared with further accounts in a computationally secure and efficient manner.

Abstract: A system and method for pruning data based on metadata. The method may include receiving a query that includes a plurality of predicates and identifying one or more applicable files including database data satisfying at least one of the plurality of predicates. The identifying the one or more applicable files including reading metadata stored in a metadata store that is separate from the database data. The method further includes pruning inapplicable files including database data that does not satisfy at least one of the plurality of predicates to create a reduced set of files and reading the reduced set of files to execute the query.

Abstract: A method, system, and apparatus for managing a persistent query result set are disclosed. A resource manager is disclosed that receives a query from a client over a computer network. A computer database server is disclosed that executes the query and generates a result set based on and in response to the received query. An execution platform is disclosed that divides the result set into a plurality of chunks, wherein each chunk comprises a portion of the generated result set. The resource manager adds metadata to each of the plurality of chunks. A first chunk of the plurality of chunks comprises metadata pointing to the plurality of chunks. The resource manager delivers at least the first chunk of the plurality of chunks to the client in response to the query. The execution platform offloads the remaining plurality of chunks to cloud storage in parallel.

Abstract: Providing access to consumer information on a private data exchange is described. In an example, privileges are assigned to a share object, granting access to consumer information obtained from a query against data referenced by a data listing of a data exchange. The data exchange comprises one or more data listings provided by one or more data providers. In response to the share object being added to a list of consumed share objects of a data provider that provided the data listing, a database is generated that references the consumer information based on the share object. Access to the database is granted to the data provider that provided the data listing.

Abstract: A system and method of using stateless nodes to process data of catalog objects. The method includes accessing a catalog object including metadata associated with a dataset. The method includes determining, based on the metadata, whether a stateless node is to process the dataset. The method includes detecting, by one or more processors, a failure associated with the stateless node in response to determining that the stateless node is to process the dataset. The method includes replacing the stateless node with a different node without recreating a particular state.

Abstract: Embodiments of the present disclosure relate to sharing database roles using hidden roles. A database role may be generated within a database container having a plurality of data objects, wherein the database role exists exclusively within the database container. A set of grants to a particular subset of the plurality of data objects of the database container may be assigned to the database role and the database role may be granted to the share object. The share object is mounted within a consumer account to generate an imported database container within the consumer account, the imported database container including an imported copy of the database role. The imported copy of the database role may be granted to each of one or more account level roles of the consumer account to share the particular subset of the plurality of data objects without creating proxy objects in the consumer account that represent the particular subset of the plurality of data objects.