Abstract: An output of a GenAI model responsive to a prompt is received. The GenAI model is configured using one or more system prompts including one or more Easter eggs. The output is scanned to confirm whether an Easter egg is present. In cases in which at least one Easter egg is present, one or more remediation actions can be initiated to thwart an information leak by the GenAI model. Related apparatus, systems, techniques and articles are also described.

Abstract: An output of a vision-language model (VLM) can be steered by receiving an input that includes both an image and text, and compositing the image with a universal image configured to elicit specific model activations associated with a targeted behavioral change. The modified image, together with the text, is then supplied to the VLM, resulting in an output that reflects the desired behavioral change. This technique enables steering of the VLM's output without modifying its internal states. The generated output is then provided to an application, process, or system for further utilization. Related apparatus, systems, techniques and articles are also described.

Abstract: An output of a generative artificial intelligence (GenAI) model is received which is responsive to a prompt by a requestor. The output is tokenized to result in a plurality of tokens. These tokens are then used to determine that the output includes at least one string comprising personally identifiable information (PII). This determined can use pattern recognition to identify tokens and sequence of tokens indicative of PII. Thereafter, a classifier is used to assign a PII type to each string in the output comprising PII. It is then determined that at least one of the PII types in the output requires redaction which results in strings having a PII type determined to require redaction to be redacted which, in turn, results in a modified output for transmission to the requester. Related apparatus, systems, techniques and articles are also described.

Abstract: A machine learning model representation is obtained from a model source and information characterizing the layers of the model representation is extracted to result in extracted model information. This extracted model information can be compared to information characterizing one or more known (i.e., previously characterized) machine learning models in order to determine whether there is a match based on layer information. A match can, in some cases, be used to determine an identity of the underlying machine learning model for the model representation. Information regarding the comparison (i.e., the model matching determination) can be provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.

Abstract: A query is received which is to be input into a machine learning model (or other artificial intelligence model). Thereafter, a plurality of historical queries of the machine learning model meeting first criteria relative to the query is determined using a first distance-based similarity analysis technique. Each of the historical queries have a known output by the machine learning model. An output of the machine learning model responsive to query is received. Next, it is determined, using a second distance-based similarity analysis technique, whether the output meets second criteria relative to each of the known outputs corresponding to the historical queries. This determination characterizes whether the query is likely to cause the machine learning model to behave in an undesired manner and can be provided to a consuming application or process. Related apparatus, systems, and techniques are also described.

Abstract: An analysis engine receives data characterizing a multimodal prompt for ingestion by a generative artificial intelligence (GenAI) model. The multimodal prompt is processed and fed into a plurality of layers from which an intermediate result of the GenAI model or a proxy of the GenAI model is obtained. The analysis engine, using a classifier and the intermediate result, determines whether the prompt elicits undesired behavior by the GenAI model. Data characterizing the determination is provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.

Abstract: The inputs and/or outputs of a generative artificial intelligence model are monitored to determine whether they contain or otherwise elicit undesired behavior by the model such as bypassing security measures, leaking sensitive information, or generating or consuming malicious content. This determination can be used to selectively trigger remediation processes to protect the model from malicious actions. Related apparatus, systems, techniques and articles are also described.

Abstract: An analysis engine receives data characterizing a multimodal prompt for ingestion by a generative artificial intelligence (GenAI) model. The multimodal prompt is processed and fed into a plurality of layers from which an intermediate result of the GenAI model or a proxy of the GenAI model is obtained. The analysis engine, using a prompt injection classifier and the intermediate result, determines whether the prompt comprises or is indicative of malicious content or elicits malicious actions. Data characterizing the determination is provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.

Abstract: Vulnerabilities in a machine learning model can be identified by receiving at least one file encapsulating the machine learning model. A computational graph corresponding to the machine learning model is then extracted from the at least on file. The computational graph is converted from a first format into a normalized computational graph having a second, different format. The normalized computational graph is decomposed into components. These components can include nodes, blocks, and edges between blocks. The normalized computational graph is scanned by iterating through the components to identify any backdoors. Data characterizing whether any backdoors were identified can be provided to a consuming application or process.

Abstract: Techniques for assessing multi-modal inputs to a machine learning model involve receiving a multimodal input containing an image, producing several transformed versions of that image, and generating embeddings for both the original and transformed images. A pairwise similarity analysis among all embeddings is conducted to determine distance values. Two dissimilarity metrics can then be calculated: one reflecting the differences among the transformed images, and another comparing the original image to its transformed versions. If the dissimilarity among the transformed images is greater than that between the original and transformed images plus a threshold, the system triggers a remediation action. This action either blocks the input from being processed by the machine learning model or prevents the model's output from being returned to the requester, thereby enhancing the reliability and security of the model.

Abstract: A query to be input into a machine learning model which is associated with a first user is received. A first embedding is generated based on the query. A plurality of historical queries of the machine learning model having a corresponding embedding meeting first criteria relative to the first embedding is then determined using a first distance-based similarity analysis technique. In addition, a plurality of other users of the machine learning model each having a corresponding user embedding meeting second criteria relative to a user embedding for the first user are determined using a second distance-based similarity analysis technique. Data indicating a potential attack on the machine learning model is provided to a consuming application or process based on the query neighbor determination and the user neighbor determination.

Abstract: The inputs and/or outputs of a generative artificial intelligence model are monitored to determine whether they contain or otherwise elicit undesired behavior by the model such as bypassing security measures, leaking sensitive information, or generating or consuming malicious content. This determination can be used to selectively trigger remediation processes to protect the model from malicious actions. Related apparatus, systems, techniques and articles are also described.

Abstract: A plurality of queries are input into an artificial intelligence (AI) model. The AI model is made up of a plurality of layers including an input layer, an output layer, and at least one intermediate layer between the input layer and the output layer. Each intermediate layer, during inference, can output a plurality of activations. Thereafter, for each query, activations are intercepted from at least one of the intermediate layers. It is then determined whether a distribution of the intercepted activations across the queries indicates that the queries seek to cause the AI model to behave in an undesired manner by conducting a distance-based similarity analysis between the intercepted activations and reference activations. Data characterizing such determination is then provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.

Abstract: A first password is received by a password encoder which uses the first password to generate a first key. This first key is used to modify weights and biases of an encoder to result in a modified encoder. Further, weights and biases of a decoder operating in tandem with the encoder based can be modified based on a second key to result in a modified decoder. First data is received which encapsulates second data in a hidden compartment. The first data is encoded by the modified encoder to result to generate an embedding. The modified decoder decodes the embedding to result in a representation of the second data which, in turn, can be provided to a consuming application or process. The first data can be input into the encoder and the decoder prior to those components being modified to result in a representation of the first data.

Abstract: A machine learning model is scanned to detect actual or potential threats. The threats can be detected before execution of the machine learning model or during an isolated execution environment. The threat detection may include performing a machine learning file format check, vulnerability check, tamper check, and stenography check. The machine learning model may also be monitored in an isolated environment during an execution or runtime session. After performing a scan, the system can generate a signature based on actual, potential, or absence of detected threats.

Abstract: Data is received which includes multimodal input for ingestion by a first generative AI (GenAI) model is received. This received data is input into the first GenAI model to result in a first output. The first output along with the received data is input into a second GenAI model to result in a second output. The first GenAI model is a modified (e.g., fine-tuned, etc.) version of the second GenAI model. When the second output indicates that guardrails associated with the second GenAI model have been triggered, one or more remediation actions are initiated. Otherwise, the first output is returned to the requestor. Related apparatus, systems, techniques and articles are also described.

Abstract: A prompt for a generative artificial intelligence (GenAI) model is received which includes unicode. Unicode fonts in the prompt are identified and then translated into a plaintext representation. Further, unicode characters in the prompt are identified which each have an associated unicode tag. It is determined, based on the associated unicode tags, whether at least a portion of the unicode characters are valid. When at least a portion of the unicode characters are determined to be valid, the unicode characters in the prompt are converted into a plaintext representation. The prompt with the translated fonts and the converted unicode fonts are passed into the GenAI model. When at least a portion of the unicode characters are not determined to be valid, the unicode characters are removed from the prompt. This prompt with the translated unicode fonts, after the unicode characters are removed, is input into the GenAI model.

Abstract: A prompt for a generative artificial intelligence (GenAI) model which contains unicode is received. The prompt is then tokenized to result in a plurality of tokens. Token forming part of a repeating sequence are identified and then removed to result in a modified set of tokens. The modified set of tokens are subsequently detokenized to result in a modified prompt. It is then determined, whether ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner. The modified prompt is passed to the GenAI model when it is determined that ingestion of the modified prompt will not result in the GenAI model behaving in an undesired manner. Otherwise, at least one remediation action is initiated when it is determined that ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner.

Abstract: An encoder receives first data encapsulating second data in a hidden compartment along with a decoder identifier corresponding to either of a first decoder or a second decoder. The encoder then generates an embedding corresponding to the first data. The first decoder decodes the embedding to result in a representation of the first data when the decoder identifier corresponds to the first decoder. The second decoder decodes the embedding to result in a representation of the second data when the decoder identifier corresponds to the second decoder. The decoded embedding can be provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.

Abstract: Data is received that characterizes artefacts associated with each of a plurality of layers of a first machine learning model. Fingerprints are then generated for each of the artefacts in the layers of the first machine learning model. These generated fingerprints collectively form a model indicator for the first machine learning model. It is then determined whether the first machine learning model is derived from another machine learning model by performing a similarity analysis between the model indicator for the first machine learning model and model indicators generated for each of a plurality of reference machine learning models each comprising a respective set of fingerprints. Data characterizing the determination can be provided to a consuming application or process. Related apparatus, systems, techniques and articles are also described.