Abstract: Embodiments of the present disclosure provide techniques for mountless querying of listing data. A processing device obtains a query that includes a universal listing identifier of a database, wherein the universal listing identifier is different from an identifier for the database. The processing device activates, at runtime, at least one role for accessing the database and shared objects based on the universal listing identifier. The processing device generates, based on the universal listing identifier and the at least one activated role, an in-memory placeholder object associated with the database. The processing device provides access to data of the database based on the in-memory placeholder object and the query.

Abstract: Aspects of the present disclosure relate to configuring managed policy for internal stage network policy. More specifically, a method of the present disclosure includes generating and activating a set of network policies on a control plane of a service provider. The method includes determining to convert the set of network policies activated on the control plane of the service provider into an IAM policy at the service provider. The method includes utilizing at least one of the converted IAM policies of an active network policy in response to a creation of a presigned URL. The method includes determining whether an STS token is approved based on the presigned URL and at least one of the set of network policies or the IAM policy. The method includes utilizing an approved STS token to access the service provider.

Abstract: Various example embodiments described herein provide for systems, methods, devices, instructions, and the like that performs inference operations on an individual AI model using dynamically selected parallelism, where the parallelism selection can be based on one or more computational workload conditions associated with the individual AI model. For example, the AI model system of some example embodiments switches between two or more different parallelism modes of operation with respect to an individual AI model based on one or more inference request patterns or based on one or more batch characteristics of a set of inference requests received by the AI model system.

Abstract: Various embodiments described herein provide for systems, methods, devices, instructions, and like for generating communication content using one or more large language models (LLMs). In particular, some embodiments provide a communication content generation system that generates content for a communication to a target organization using one or more LLMs and information regarding the target organization provided by an organization database, which can comprise curated organization-intelligence data.

Abstract: A replication engine is provided that receives a set of interface implementations from features in a distributed data platform, where each interface implementation provides replication constraints. The replication engine constructs a dependency graph based on relationships between the features using the replication constraints and determines a synchronization order for replication operations using the dependency graph. The replication engine executes the replication operations according to the determined synchronization order by coordinating between multiple handlers including a snapshot handler for primary-side operations, an association handler for managing cross-cutting relationships, and a synchronization handler for secondary-side operations.

Abstract: Techniques for continuous ingestion of files using custom file formats are described. A custom file format may include formats not natively supported by a data system. Unstructured files (e.g., images) may also be considered custom file formats. A custom file format may be set using a user defined table function and scanner options.

Abstract: Interleaved infrastructure can interleave fragment execution during query compilation. For example, compilation can be paused, and a plan fragment is executed. The results of the plan fragment can be fed back into the compiler, which can then restart the compilation and modify the compilation based on the results of the plan fragment execution. The interleaved workflow provides a flexible, robust infrastructure, which can enable and improve a variety of database features and query optimizations.

Abstract: A cluster is provided to execute at least one containerized service in a cloud data platform. One or more monitor agents deployed on a plurality of worker nodes in the cluster collect a set of metrics associated with the at least one containerized service. A controller in the cluster aggregates the set of metrics according to at least one aggregation type defined in one or more autoscaling policies associated with the at least one containerized service and evaluates the aggregated set of metrics against criteria based on the one or more autoscaling policies to generate one or more autoscaling decisions.

Abstract: The subject technology receives a query directed to a set of source tables, each source table organized into a set of micro-partitions. The subject technology determines a set of metadata, the set of metadata comprising table metadata, query metadata, and historical data related to the query. The subject technology predicts, using a machine learning model, an indicator of an amount of computing resources for executing the query based at least in part on the set of metadata. The subject technology generates a query plan for executing the query based at least in part on the predicted indicator of the amount of computing resources. The subject technology executes the query based at least in part on the query plan.

Abstract: A method to process UDFs includes performing, by at least one hardware processor of a database system, a resolution of application components to determine a plurality of resolved components of an application and a plurality of data types corresponding to the plurality of resolved components. The method includes instantiating a plurality of UDF clients. The plurality of UDF clients are associated with the plurality of data types. The method includes detecting a received database query specifies a data type of the plurality of data types. The method includes executing the received database query at a UDF client of the plurality of UDF clients.

Abstract: Row-level security (RLS) may provide fine-grained access control based on flexible, user-defined access policies to databases, tables, objects, and other data structures. A RLS policy may be an entity or object that defines rules for row access. A RLS policy may be decoupled or independent from any specific table. This allows more robust and flexible control. A RLS policy may then be attached to one or more tables. The RLS policy may include a Boolean-valued expression.

Abstract: Systems and methods for managing column hiding are provided. The systems and methods receive, from a client device, a query associated with a table. The systems and methods determine an access restriction associated with the client device. The systems and methods identify a column of the table that is restricted by the access restriction associated with the client device. In response to identifying the column of the table that is restricted by the access restriction associated with the client device, the systems and methods provide a result of the query that excludes data corresponding to the column.

Abstract: Embodiments of the present disclosure provide systems and methods for using secure schemas to address inconsistencies between standard RBAC rules and the use of inherited grants. A secure schema may be defined that transfers ownership of an object created in the secure schema to a role that owns the secure schema. An inherited grant may be attached to the secure schema, where the inherited grant specifies a permission on a first type of object in the secure schema and a grant of the permission to the role that owns the secure schema. When objects are created in the secure schema, ownership of each of the set of objects is transferred to the role that owns the secure schema to authorize the role that owns the secure schema to manage grants to the set of objects on the secure schema.

Abstract: In an embodiment, a data platform creates an application in a data-provider account. The application includes one or more APIs corresponding to one or more underlying code blocks. The data platform shares provider data with the application in the data-provider account, and also installs, in a data-consumer account, an application instance of the application. The application instance includes one or more APIs corresponding to the one or more APIs in the application in the data-provider account. The data platform shares consumer data with the application instance in the data-consumer account, and invokes one or more of the APIs of the application instance to execute respective associated underlying code blocks, which are not visible to the data-consumer account. The data platform also saves output of the one or more respective associated underlying code blocks locally within the data-consumer account.

Abstract: Various example embodiments described herein provide for systems, methods, devices, instructions, and the like for optimizing a query plan for executing a query by pushing down top-k information in the query plan with respect to a group-by grouping sets operator or a window function operator, which can be used within a data platform (e.g., database) environment. In particular, various example embodiments enable information from a top-k operator of a query plan to be pushed down the query plan and enable pushing the information to a select operator (e.g., aggregation operator) positioned below the top-k operator that implements the group-by grouping sets operator or the window function operator.

Abstract: A data platform is provided that builds a container image filesystem from a container image. The data platform executes an application that accesses the container image filesystem and the data platform records directory and file access activity of the container image filesystem by the application during execution. The data platform dynamically rebuilds the container image filesystem using the directory and file access activity, and mounts the container image filesystem as rebuilt during a subsequent execution of the application.

Abstract: The subject technology stores an original file master key identifier (FMKID) in metadata associated with an encrypted file, the encrypted file including data. The subject technology determines that the encrypted file is to be re-encrypted based at least on a period of time since the original FMKID was stored. The subject technology re-encrypts the data using a new encryption key associated with a new FMKID. The subject technology generates a mapping from the original FMKID to the new FMKID. The subject technology stores the mapping in a metadata database, the mapping being associated with a re-encrypted file, the re-encrypted file including the re-encrypted data.

Abstract: Techniques for configuring a managed event table (MET) include detecting, by at least one hardware processor, a query for the MET. The query received at a first computing node of a network-based database system. The method includes retrieving via an ingestion function configured at the first computing node, reformatted data from a dedicated storage location of a first process into the MET. The reformatted data is based on log data associated with a second process. The first process and the second process are executing at a second computing node of the network-based database system. The method includes processing the query using the reformatted data in the MET.

Abstract: Monitoring health metrics of computing devices in a data system can be implemented at different levels. At a first level, local background services can be run locally on the computing devices to monitor a set of health metrics on the respective computing devices. At a second level, a central health marker service can monitor a set of health metrics for the computing devices.

Abstract: Different database deployments, or other data system deployments, may want to communicate with each other without sacrificing security or control. To this end, embodiments of the present disclosure may provide secure message exchange techniques for a source and/or target deployment. Configurable rule sets may be stored in the deployments; the rule sets may define what messages may be communicated between deployments. The deployments may implement a selective filtering scheme in one or more stages based on the rule sets to filter outgoing and/or incoming messages.