Abstract: A method is provided for optimizing query performance by transforming an artificial intelligence (AI) join task into an AI classification task. The method comprises receiving a request to join a first table and a second table using an AI filter applicable to values of a first column of the first table and a selected column of the second table. A modified query is generated to perform the join with the AI classification task by performing, for each row in the first table, the AI classification function with the value from the first column and the values from the selected column. The AI classification function is configured to determine, in a single inference call, which candidate values from the selected column that are relevant to the value from the first column in the row. The method also includes performing the modified query and outputting a result from performing the modified query.

Abstract: Methods and systems for generating a global database profile are described. The methods and systems receive, from each of a plurality of layers of a database system, profiling data, the profiling data comprising a first set of profiling data received from a first layer of the plurality of layers and a second set of profiling data received from a second layer of the plurality of layers, the profiling data representing database logic operations and physical states at each of the plurality of layers. The methods and systems aggregate the profiling data received from the plurality of layers into global database profiling data associated with the plurality of layers of the database system and present visual output based on the profiling data, the visual output representing the database logic operations and the physical states at each of the plurality of layers over time.

Abstract: The subject technology receives a first query plan corresponding to a query, the first query plan comprising a set of predicates. The subject technology receives, during execution of a first portion of the first query plan, a set of rowsets. The subject technology determines a set of metrics for a first number of rows from a plurality of rows, the first number of rows corresponding to a first predicate order. The subject technology determines, using a heuristic, a second predicate order based at least in part on the set of metrics. The subject technology processes, during execution of the first portion of the first query plan using the second predicate order, a second set of rowsets, the second set of rowsets comprising a second plurality of rows that correspond to the first portion of the first query plan that has been executed based on the second predicate order.

Abstract: A network egress request is received from a container service within a cloud data platform. A cryptographically signed egress policy associated with the network egress request is received by a trusted service controller of the cloud data platform. The network egress request is validated against the cryptographically signed egress policy. Based on the validation, a determination of whether the network egress request complies with the cryptographically signed egress policy is established. Upon validation, the network egress request is granted or denied based on the determination.

Abstract: Various example embodiments described herein provide for systems, methods, devices, instructions, and the like for structured language parsing to execute a differentially private query on a database system. According to some example embodiments, a user (e.g., an analyst) submits to a data system (e.g., data platform) a differentially private query using a structured language interface (e.g., SQL interface), which causes the calling of one or more stored procedures on the data system, where the one or more stored procedures encapsulate or facilitate use of a differential privacy engine, which can execute the differentially private query and generate a differentially private query result.

Abstract: Various embodiments described herein provide for systems, methods, devices, instructions, and like for generating a structured language data query based on a natural language question and semantic data associated with a schema of a data store (e.g., database or the like). In particular, some embodiments use a set of large language models to generate a structured language data query for a data store based on semantic data and the natural language question, determines whether the structured language data query is valid, causes the structured language data query to be performed on a data store in response to determining that the structured language data query is valid, and generating a response that comprises a query result from the data store.

Abstract: The subject technology receives a first set of statements to create an image repository, the first set of statements comprising a set of Structured Query Language (SQL) statements. The subject technology creates the image repository, the creating including generating a uniform resource locator (URL), the URL associated with a location for storing data. The subject technology performs a domain name service (DNS) mapping of the URL to an image registry service. The subject technology receives a first command to tag a container image to an alias corresponding to the URL. The subject technology assigns the alias to the container image. The subject technology receives a second command to push the tagged container image to the image repository. The subject technology sends the tagged container image to the location to store the tagged container image.

Abstract: A top K query directed at a table is received. Run-time pruning is performed during execution of the top K query on the table. The run-time pruning comprises determining, by a top K node, a current boundary based on a set of values identified by a table scan node in scanning the table and applying, by the table scan node, the current boundary to prune data during the scanning of the table. The applying of the current boundary comprises reducing scanning ranges of the table scan node based on the top K column being a key column of the table and filtering values scanned by the table scan node based on the top K column being a non-key column of the table. The result set is returned responsive to the top K query based on the run-time pruning performed during execution of the top K query on the table.

Abstract: Embodiments of the present disclosure provide techniques for mountless querying of listing data. A processing device obtains a query that includes a universal listing identifier of a database, wherein the universal listing identifier is different from an identifier for the database. The processing device activates, at runtime, at least one role for accessing the database and shared objects based on the universal listing identifier. The processing device generates, based on the universal listing identifier and the at least one activated role, an in-memory placeholder object associated with the database. The processing device provides access to data of the database based on the in-memory placeholder object and the query.

Abstract: Aspects of the present disclosure relate to configuring managed policy for internal stage network policy. More specifically, a method of the present disclosure includes generating and activating a set of network policies on a control plane of a service provider. The method includes determining to convert the set of network policies activated on the control plane of the service provider into an IAM policy at the service provider. The method includes utilizing at least one of the converted IAM policies of an active network policy in response to a creation of a presigned URL. The method includes determining whether an STS token is approved based on the presigned URL and at least one of the set of network policies or the IAM policy. The method includes utilizing an approved STS token to access the service provider.

Abstract: Various example embodiments described herein provide for systems, methods, devices, instructions, and the like that performs inference operations on an individual AI model using dynamically selected parallelism, where the parallelism selection can be based on one or more computational workload conditions associated with the individual AI model. For example, the AI model system of some example embodiments switches between two or more different parallelism modes of operation with respect to an individual AI model based on one or more inference request patterns or based on one or more batch characteristics of a set of inference requests received by the AI model system.

Abstract: Various embodiments described herein provide for systems, methods, devices, instructions, and like for generating communication content using one or more large language models (LLMs). In particular, some embodiments provide a communication content generation system that generates content for a communication to a target organization using one or more LLMs and information regarding the target organization provided by an organization database, which can comprise curated organization-intelligence data.

Abstract: A replication engine is provided that receives a set of interface implementations from features in a distributed data platform, where each interface implementation provides replication constraints. The replication engine constructs a dependency graph based on relationships between the features using the replication constraints and determines a synchronization order for replication operations using the dependency graph. The replication engine executes the replication operations according to the determined synchronization order by coordinating between multiple handlers including a snapshot handler for primary-side operations, an association handler for managing cross-cutting relationships, and a synchronization handler for secondary-side operations.

Abstract: Techniques for continuous ingestion of files using custom file formats are described. A custom file format may include formats not natively supported by a data system. Unstructured files (e.g., images) may also be considered custom file formats. A custom file format may be set using a user defined table function and scanner options.

Abstract: Interleaved infrastructure can interleave fragment execution during query compilation. For example, compilation can be paused, and a plan fragment is executed. The results of the plan fragment can be fed back into the compiler, which can then restart the compilation and modify the compilation based on the results of the plan fragment execution. The interleaved workflow provides a flexible, robust infrastructure, which can enable and improve a variety of database features and query optimizations.

Abstract: A cluster is provided to execute at least one containerized service in a cloud data platform. One or more monitor agents deployed on a plurality of worker nodes in the cluster collect a set of metrics associated with the at least one containerized service. A controller in the cluster aggregates the set of metrics according to at least one aggregation type defined in one or more autoscaling policies associated with the at least one containerized service and evaluates the aggregated set of metrics against criteria based on the one or more autoscaling policies to generate one or more autoscaling decisions.

Abstract: The subject technology receives a query directed to a set of source tables, each source table organized into a set of micro-partitions. The subject technology determines a set of metadata, the set of metadata comprising table metadata, query metadata, and historical data related to the query. The subject technology predicts, using a machine learning model, an indicator of an amount of computing resources for executing the query based at least in part on the set of metadata. The subject technology generates a query plan for executing the query based at least in part on the predicted indicator of the amount of computing resources. The subject technology executes the query based at least in part on the query plan.

Abstract: A method to process UDFs includes performing, by at least one hardware processor of a database system, a resolution of application components to determine a plurality of resolved components of an application and a plurality of data types corresponding to the plurality of resolved components. The method includes instantiating a plurality of UDF clients. The plurality of UDF clients are associated with the plurality of data types. The method includes detecting a received database query specifies a data type of the plurality of data types. The method includes executing the received database query at a UDF client of the plurality of UDF clients.

Abstract: Row-level security (RLS) may provide fine-grained access control based on flexible, user-defined access policies to databases, tables, objects, and other data structures. A RLS policy may be an entity or object that defines rules for row access. A RLS policy may be decoupled or independent from any specific table. This allows more robust and flexible control. A RLS policy may then be attached to one or more tables. The RLS policy may include a Boolean-valued expression.

Abstract: Systems and methods for managing column hiding are provided. The systems and methods receive, from a client device, a query associated with a table. The systems and methods determine an access restriction associated with the client device. The systems and methods identify a column of the table that is restricted by the access restriction associated with the client device. In response to identifying the column of the table that is restricted by the access restriction associated with the client device, the systems and methods provide a result of the query that excludes data corresponding to the column.