Abstract: There are provided methods and devices for assessing the generalizability of software benchmarks. The method includes selecting a benchmark for assessment and identifying a generalizability bias in the selected benchmark. The determination of a generalizability assessment for the selected benchmark can be based on the identified generalizability bias. Subsequently, the generalizability of the selected benchmark can be determined based on the determined generalizability assessment. A computing device is provided which includes a processor and a memory storing instructions, wherein when the instructions are executed by the processor the device can be configured to perform the above method.