Abstract: A computer-implemented method of generating verification data for a query result provided by a large language model, LLM, includes generating a prompt for the large language model. The prompt contains a verification request for a query, the query including query text and input data from which the query result can be derived. The verification request includes instructions that cause the LLM to generate verification data that indicates a derivation of the query result from the input data. Another computer-implemented method includes receiving the verification data and processing the verification data to determine whether the query result was validly derived from the input data.

Abstract: The technology described herein determines whether a candidate text is in a requested class by using a generative model that may not be trained on the requested class. The present technology may use of a model trained primarily in an unsupervised mode, without requiring a large number of manual user-input examples of a label class. The may produce a semantically rich positive example of label text from a candidate text and label. Likewise, the technology may produce from the candidate text and the label a semantically rich negative example of label text. The labeling service makes use of a generative model to produce a generative result, which estimates the likelihood that the label properly applies to the candidate text. In another aspect, the technology is directed toward a method for obtaining a semantically rich example that is similar to a candidate text.

Abstract: Machine learning models are used to generate a plan that responds to a user request. The plan includes one or more skills selected from a list of available skills. The prompt may be written in natural language, enabling the user to express their intent without having to know which skills are available or their intricacies. In some configurations, a skill is included in the plan if an embedding representation of an example prompt associated with the skill is within a defined distance of an embedding representation of the user request. Additionally, or alternatively, the embedding distance computations are used to narrow the list of available skills, which is then used to construct a meta-prompt that selects a skill. Skills listed in the meta-prompt may include data types of parameters and return values. This allows the model that processes the meta-prompt to order skills based on data type compatibility.

Abstract: A computing system assists in large language model system assisted investigations. The computing system includes network connection hardware configured to connect to a large language model and configured provide to investigation context and investigation goals to the large language model system. The network connection receives from the large language model system, an indication of suggested steps to perform in an investigation, including specific computer executable code to perform a skill in the first step, the skill comprising a supplemental access, analytic or enrichment function. The computing system includes a user interface with a tree interface that causes display of the indication of the suggested steps in a tree format. The computing system is configured to execute the computer executable code to cause the computer system to perform the supplemental access, analytic or enrichment function.

Abstract: A large language model consumes example query expressions, including a data access function, a data analytics function, or a data enrichment function. The large language model receives a centrally managed ontology. The large language model uses the centrally managed ontology, and identifies skill ontological types from the example query expressions. The skill ontological types are normalized (to the centrally managed ontology) input arguments types or structured output. The large language model receives context for an investigation and identifies a context ontological type. The large language model receives received skills, based on correlation between a skill ontological type, having connections in a graph to the received skills, and the context ontological type. The large language model produces and provides an indication of a suggested skill for the investigation.

Abstract: The technology described herein determines whether a candidate text is in a requested class by using a generative model that may not be trained on the requested class. The present technology may use of a model trained primarily in an unsupervised mode, without requiring a large number of manual user-input examples of a label class. The may produce a semantically rich positive example of label text from a candidate text and label. Likewise, the technology may produce from the candidate text and the label a semantically rich negative example of label text. The labeling service makes use of a generative model to produce a generative result, which estimates the likelihood that the label properly applies to the candidate text. In another aspect, the technology is directed toward a method for obtaining a semantically rich example that is similar to a candidate text.

Abstract: In some examples, a method for orchestrating an execution plan is provided. The method includes receiving an input embedding that is generated by a machine-learning model and receiving a plurality of stored semantic embeddings, from an embedding object memory, based on the input embedding. The plurality of stored semantic embeddings each correspond to a respective historic plan. Each historic plan includes one or more executable skills. The method further includes determining a subset of semantic embeddings from the plurality of stored semantic embeddings based on a similarity to the input embedding, and generating a new plan based on the subset of semantic embeddings and the input embedding. The new plan may be different than the historic plans that correspond to the subset of semantic embeddings. The method further includes providing the new plan as an output.

Abstract: A computer-implemented method of generating a security language query from a user input query includes receiving, at a computer system, an input security hunting user query indicating a user intention; selecting, using a trained machine learning model and based on the input security hunting query, an example user security hunting query and corresponding example security language query; generating, using the trained machine learning model, query metadata from the input security hunting query; generating a prompt, the prompt comprising: the input security hunting user query; the selected example user security hunting query and the corresponding example security language query; and the generated query metadata; inputting the prompt to a large language model; receiving a security language query from the large language model corresponding to the input security hunting query reflective of the user intention.

Abstract: The technology described herein determines whether a candidate text is in a requested class by using a generative model that may not be trained on the requested class. The present technology may use of a model trained primarily in an unsupervised mode, without requiring a large number of manual user-input examples of a label class. The may produce a semantically rich positive example of label text from a candidate text and label. Likewise, the technology may produce from the candidate text and the label a semantically rich negative example of label text. The labeling service makes use of a generative model to produce a generative result, which estimates the likelihood that the label properly applies to the candidate text. In another aspect, the technology is directed toward a method for obtaining a semantically rich example that is similar to a candidate text.

Abstract: A user interface (UI) may be used to introduce a message into the cloud computing network. The message may be received by a service associated with the cloud computing network. The message may trigger the service to generate data in response to receiving the message. The generated data may include temporal data that includes the date and time data specifying when the message was received by the service. The generated temporal data may be forwarded to a telemetry store associated with the cloud computing network. A user or report generating operator portal may generate a report that includes the generated temporal data. The generated report may be used to determine if the service associated with the cloud computing network is functioning properly.

Abstract: Apparatus and methods can be implemented to perform software testing or to perform emulated hardware testing using a cloud architecture that can utilize centralized testing technology and can enable scaling up to test for multiple tenants and scaling up to arbitrary numbers of programs tested for each tenant. A user can configure an initial test virtual machine on a cloud platform for a cloud service over a physical network such as the Internet. Components of the cloud architecture can create a set of clones of the initial test virtual machine and inject tools into each clone for testing. Testing of one or more clones of the set can be conducted in an environment isolated from the physical network and isolated from a backend of the cloud service. Additional apparatus, systems, and methods are disclosed.

Abstract: Techniques for constrained mutation-based fuzzing are described. Machine accesses an input file of code for testing. Machine performs multiple runs of a fuzzing algorithm using the input file and the code. Each run includes: performing a mutation of one or more bytes of the input file and determining which parts of the code were executed when the code was run with the mutated input file. Machine stores, for each run, an indication of whether the mutation caused execution of a portion of the code which was not executed prior to the mutation, Machine generates heatmap of the input file based on the stored indications. The heatmap maps each of the bytes in the input file to a value indicating whether the mutation of the byte caused execution of the portion of the code for testing which was not executed prior to the mutation. Machine tailors fuzzing algorithm based on heatmap.

Abstract: A user interface (UI) may be used to introduce a message into the cloud computing network. The message may be received by a service associated with the cloud computing network. The message may trigger the service to generate data in response to receiving the message. The generated data may include temporal data that includes the date and time data specifying when the message was received by the service. The generated temporal data may be forwarded to a telemetry store associated with the cloud computing network. A user or report generating operator portal may generate a report that includes the generated temporal data. The generated report may be used to determine if the service associated with the cloud computing network is functioning properly.

Abstract: Techniques for constrained mutation-based fuzzing are described. Machine accesses an input file of code for testing. Machine performs multiple runs of a fuzzing algorithm using the input file and the code. Each run includes: performing a mutation of one or more bytes of the input file and determining which parts of the code were executed when the code was run with the mutated input file. Machine stores, for each run, an indication of whether the mutation caused execution of a portion of the code which was not executed prior to the mutation, Machine generates heatmap of the input file based on the stored indications. The heatmap maps each of the bytes in the input file to a value indicating whether the mutation of the byte caused execution of the portion of the code for testing which was not executed prior to the mutation. Machine tailors fuzzing algorithm based on heatmap.

Abstract: Apparatus and methods can be implemented to perform software testing or to perform emulated hardware testing using a cloud architecture that can utilize centralized testing technology and can enable scaling up to test for multiple tenants and scaling up to arbitrary numbers of programs tested for each tenant. A user can configure an initial test virtual machine on a cloud platform for a cloud service over a physical network such as the Internet. Components of the cloud architecture can create a set of clones of the initial test virtual machine and inject tools into each clone for testing. Testing of one or more clones of the set can be conducted in an environment isolated from the physical network and isolated from a backend of the cloud service. Additional apparatus, systems, and methods are disclosed.