Abstract: Computer-implemented testing of subject software using labelled recorded alerts. Each of the labelled recorded alerts specifies one or more testing inputs representing one or more events that previously led to prior software when the alert occurred, and a label indicating whether the alert was a true positive. For each labelled recorded alerts, testing of the subject software is automatically performed by 1) reading the one or more testing inputs of the associated labelled recorded alert, 2) applying the read one or more testing inputs to the subject software, and 3) determining whether a fault arises as a result of the application of the read one or more conditions to the subject software. Thus, a tester did not themselves need to come up with testing inputs to apply. Instead, such testing inputs were automatically obtained from labelled data of prior recorded alerts.

Abstract: Some embodiments gather security activity data from multiple environments instead of only a single environment. Activity data includes alerts, anomaly detections, and defensive actions taken automatically, in response to actual or simulated attacks. Data is cloaked to protect privacy. Security product coverage of techniques, tactics, procedures, threat categories, and other constituents of a cyberattack model is derived from the activity data via a mapping mechanism, allowing subsequent product installation or operation changes to be based on actual recorded responses of products to attacks. Coverage results may be operationalized as recommendations or proactive automated initiatives, for example. Security is enhanced on the basis of data which extends beyond the data available to any single cloud tenant.