Abstract: A network egress request is received from a container service within a cloud data platform. A cryptographically signed egress policy associated with the network egress request is received by a trusted service controller of the cloud data platform. The network egress request is validated against the cryptographically signed egress policy. Based on the validation, a determination of whether the network egress request complies with the cryptographically signed egress policy is established. Upon validation, the network egress request is granted or denied based on the determination.

Abstract: Various example embodiments described herein provide for systems, methods, devices, instructions, and the like for structured language parsing to execute a differentially private query on a database system. According to some example embodiments, a user (e.g., an analyst) submits to a data system (e.g., data platform) a differentially private query using a structured language interface (e.g., SQL interface), which causes the calling of one or more stored procedures on the data system, where the one or more stored procedures encapsulate or facilitate use of a differential privacy engine, which can execute the differentially private query and generate a differentially private query result.

Abstract: Various embodiments described herein provide for systems, methods, devices, instructions, and like for generating a structured language data query based on a natural language question and semantic data associated with a schema of a data store (e.g., database or the like). In particular, some embodiments use a set of large language models to generate a structured language data query for a data store based on semantic data and the natural language question, determines whether the structured language data query is valid, causes the structured language data query to be performed on a data store in response to determining that the structured language data query is valid, and generating a response that comprises a query result from the data store.

Abstract: The subject technology receives a first set of statements to create an image repository, the first set of statements comprising a set of Structured Query Language (SQL) statements. The subject technology creates the image repository, the creating including generating a uniform resource locator (URL), the URL associated with a location for storing data. The subject technology performs a domain name service (DNS) mapping of the URL to an image registry service. The subject technology receives a first command to tag a container image to an alias corresponding to the URL. The subject technology assigns the alias to the container image. The subject technology receives a second command to push the tagged container image to the image repository. The subject technology sends the tagged container image to the location to store the tagged container image.

Abstract: A top K query directed at a table is received. Run-time pruning is performed during execution of the top K query on the table. The run-time pruning comprises determining, by a top K node, a current boundary based on a set of values identified by a table scan node in scanning the table and applying, by the table scan node, the current boundary to prune data during the scanning of the table. The applying of the current boundary comprises reducing scanning ranges of the table scan node based on the top K column being a key column of the table and filtering values scanned by the table scan node based on the top K column being a non-key column of the table. The result set is returned responsive to the top K query based on the run-time pruning performed during execution of the top K query on the table.

Abstract: Embodiments of the present disclosure provide techniques for mountless querying of listing data. A processing device obtains a query that includes a universal listing identifier of a database, wherein the universal listing identifier is different from an identifier for the database. The processing device activates, at runtime, at least one role for accessing the database and shared objects based on the universal listing identifier. The processing device generates, based on the universal listing identifier and the at least one activated role, an in-memory placeholder object associated with the database. The processing device provides access to data of the database based on the in-memory placeholder object and the query.

Abstract: Aspects of the present disclosure relate to configuring managed policy for internal stage network policy. More specifically, a method of the present disclosure includes generating and activating a set of network policies on a control plane of a service provider. The method includes determining to convert the set of network policies activated on the control plane of the service provider into an IAM policy at the service provider. The method includes utilizing at least one of the converted IAM policies of an active network policy in response to a creation of a presigned URL. The method includes determining whether an STS token is approved based on the presigned URL and at least one of the set of network policies or the IAM policy. The method includes utilizing an approved STS token to access the service provider.

Abstract: Various example embodiments described herein provide for systems, methods, devices, instructions, and the like that performs inference operations on an individual AI model using dynamically selected parallelism, where the parallelism selection can be based on one or more computational workload conditions associated with the individual AI model. For example, the AI model system of some example embodiments switches between two or more different parallelism modes of operation with respect to an individual AI model based on one or more inference request patterns or based on one or more batch characteristics of a set of inference requests received by the AI model system.

Abstract: Various embodiments described herein provide for systems, methods, devices, instructions, and like for generating communication content using one or more large language models (LLMs). In particular, some embodiments provide a communication content generation system that generates content for a communication to a target organization using one or more LLMs and information regarding the target organization provided by an organization database, which can comprise curated organization-intelligence data.

Abstract: A replication engine is provided that receives a set of interface implementations from features in a distributed data platform, where each interface implementation provides replication constraints. The replication engine constructs a dependency graph based on relationships between the features using the replication constraints and determines a synchronization order for replication operations using the dependency graph. The replication engine executes the replication operations according to the determined synchronization order by coordinating between multiple handlers including a snapshot handler for primary-side operations, an association handler for managing cross-cutting relationships, and a synchronization handler for secondary-side operations.

Abstract: Techniques for continuous ingestion of files using custom file formats are described. A custom file format may include formats not natively supported by a data system. Unstructured files (e.g., images) may also be considered custom file formats. A custom file format may be set using a user defined table function and scanner options.

Abstract: Interleaved infrastructure can interleave fragment execution during query compilation. For example, compilation can be paused, and a plan fragment is executed. The results of the plan fragment can be fed back into the compiler, which can then restart the compilation and modify the compilation based on the results of the plan fragment execution. The interleaved workflow provides a flexible, robust infrastructure, which can enable and improve a variety of database features and query optimizations.

Abstract: A cluster is provided to execute at least one containerized service in a cloud data platform. One or more monitor agents deployed on a plurality of worker nodes in the cluster collect a set of metrics associated with the at least one containerized service. A controller in the cluster aggregates the set of metrics according to at least one aggregation type defined in one or more autoscaling policies associated with the at least one containerized service and evaluates the aggregated set of metrics against criteria based on the one or more autoscaling policies to generate one or more autoscaling decisions.

Abstract: The subject technology receives a query directed to a set of source tables, each source table organized into a set of micro-partitions. The subject technology determines a set of metadata, the set of metadata comprising table metadata, query metadata, and historical data related to the query. The subject technology predicts, using a machine learning model, an indicator of an amount of computing resources for executing the query based at least in part on the set of metadata. The subject technology generates a query plan for executing the query based at least in part on the predicted indicator of the amount of computing resources. The subject technology executes the query based at least in part on the query plan.

Abstract: A method to process UDFs includes performing, by at least one hardware processor of a database system, a resolution of application components to determine a plurality of resolved components of an application and a plurality of data types corresponding to the plurality of resolved components. The method includes instantiating a plurality of UDF clients. The plurality of UDF clients are associated with the plurality of data types. The method includes detecting a received database query specifies a data type of the plurality of data types. The method includes executing the received database query at a UDF client of the plurality of UDF clients.

Abstract: Row-level security (RLS) may provide fine-grained access control based on flexible, user-defined access policies to databases, tables, objects, and other data structures. A RLS policy may be an entity or object that defines rules for row access. A RLS policy may be decoupled or independent from any specific table. This allows more robust and flexible control. A RLS policy may then be attached to one or more tables. The RLS policy may include a Boolean-valued expression.

Abstract: Systems and methods for managing column hiding are provided. The systems and methods receive, from a client device, a query associated with a table. The systems and methods determine an access restriction associated with the client device. The systems and methods identify a column of the table that is restricted by the access restriction associated with the client device. In response to identifying the column of the table that is restricted by the access restriction associated with the client device, the systems and methods provide a result of the query that excludes data corresponding to the column.

Abstract: Embodiments of the present disclosure provide systems and methods for using secure schemas to address inconsistencies between standard RBAC rules and the use of inherited grants. A secure schema may be defined that transfers ownership of an object created in the secure schema to a role that owns the secure schema. An inherited grant may be attached to the secure schema, where the inherited grant specifies a permission on a first type of object in the secure schema and a grant of the permission to the role that owns the secure schema. When objects are created in the secure schema, ownership of each of the set of objects is transferred to the role that owns the secure schema to authorize the role that owns the secure schema to manage grants to the set of objects on the secure schema.

Abstract: In an embodiment, a data platform creates an application in a data-provider account. The application includes one or more APIs corresponding to one or more underlying code blocks. The data platform shares provider data with the application in the data-provider account, and also installs, in a data-consumer account, an application instance of the application. The application instance includes one or more APIs corresponding to the one or more APIs in the application in the data-provider account. The data platform shares consumer data with the application instance in the data-consumer account, and invokes one or more of the APIs of the application instance to execute respective associated underlying code blocks, which are not visible to the data-consumer account. The data platform also saves output of the one or more respective associated underlying code blocks locally within the data-consumer account.

Abstract: Various example embodiments described herein provide for systems, methods, devices, instructions, and the like for optimizing a query plan for executing a query by pushing down top-k information in the query plan with respect to a group-by grouping sets operator or a window function operator, which can be used within a data platform (e.g., database) environment. In particular, various example embodiments enable information from a top-k operator of a query plan to be pushed down the query plan and enable pushing the information to a select operator (e.g., aggregation operator) positioned below the top-k operator that implements the group-by grouping sets operator or the window function operator.