Abstract: Implementations described herein relate to methods, devices, and computer-readable media to test a generative model. In some implementations, a method includes generating a plurality of prompts, wherein each prompt is associated with a respective test category. The method further includes, for each of the plurality of prompts, providing the prompt to a generative model and capturing a response to the prompt produced by the generative model. The method further includes storing the prompt and the response in a database. The method further includes analyzing respective pairs of prompts and corresponding responses in the database to determine generative model performance. Analyzing the respective pairs include determining, using a safety filter, a test result for each pair that indicates whether the response violates a policy associated with the test category.

Abstract: Generally discussed herein are devices, systems, and methods for improving cloud resource security. A method can include obtaining a cloud resource management log that details actions performed by users of cloud resources in a cloud portal, the actions including entries comprising at least two of a user identification (ID) of a user of the users, an operation of operations performed on the cloud resource, a uniform resource identifier (URI) of a cloud resource of the cloud resources that is a target of the operation, or a time the operation was performed. The method can include determining a respective score for each action in the cloud resource management log, comparing the respective score to a specified criterion, and providing an indication of anomalous action in response to determining the respective score satisfies the specified criterion.

Abstract: Techniques are described herein that are capable of detecting potential malicious use of a resource management agent using a resource management log. The resource management log is analyzed. The resource management log logs API requests that request that a resource management system cause the resource management agent to perform respective operations. An anomalous operation is detected among the operations based at least in part on an identified API request satisfying one or more criteria associated with anomalous behavior. The identified API request requests that the resource management system cause the resource management agent, which is loaded on a machine and which enables the resource management system to remotely manage resources associated with the machine, to perform the anomalous operation. An alert is generated to notify a user of the potential malicious use of the resource management agent based at least in part on detection of the anomalous operation.

Abstract: Generally discussed herein are devices, systems, and methods for improving cloud resource security. A method can include obtaining a cloud resource management log that details actions performed by users of cloud resources in a cloud portal, the actions including entries comprising at least two of a user identification (ID) of a user of the users, an operation of operations performed on the cloud resource, a uniform resource identifier (URI) of a cloud resource of the cloud resources that is a target of the operation, or a time the operation was performed. The method can include determining a respective score for each action in the cloud resource management log, comparing the respective score to a specified criterion, and providing an indication of anomalous action in response to determining the respective score satisfies the specified criterion.

Abstract: Generally discussed herein are devices, systems, and methods for improving cloud resource security. A method can include obtaining a cloud resource management log that details actions performed by users of cloud resources in a cloud portal, the actions including entries comprising at least two of a user identification (ID) of a user of the users, an operation of operations performed on the cloud resource, a uniform resource identifier (URI) of a cloud resource of the cloud resources that is a target of the operation, or a time the operation was performed. The method can include determining a respective score for each action in the cloud resource management log, comparing the respective score to a specified criterion, and providing an indication of anomalous action in response to determining the respective score satisfies the specified criterion.

Abstract: Techniques are described herein that are capable of detecting potential malicious use of a resource management agent using a resource management log. The resource management log is analyzed. The resource management log logs API requests that request that a resource management system cause the resource management agent to perform respective operations. An anomalous operation is detected among the operations based at least in part on an identified API request satisfying one or more criteria associated with anomalous behavior. The identified API request requests that the resource management system cause the resource management agent, which is loaded on a machine and which enables the resource management system to remotely manage resources associated with the machine, to perform the anomalous operation. An alert is generated to notify a user of the potential malicious use of the resource management agent based at least in part on detection of the anomalous operation.

Abstract: Generally discussed herein are devices, systems, and methods for improving cloud resource security. A method can include obtaining a cloud resource management log that details actions performed by users of cloud resources in a cloud portal, the actions including entries comprising at least two of a user identification (ID) of a user of the users, an operation of operations performed on the cloud resource, a uniform resource identifier (URI) of a cloud resource of the cloud resources that is a target of the operation, or a time the operation was performed. The method can include determining a respective score for each action in the cloud resource management log, comparing the respective score to a specified criterion, and providing an indication of anomalous action in response to determining the respective score satisfies the specified criterion.