Abstract: In a system for evaluating classification systems such as an operating system (OS) fingerprinting tool (e.g., Nmap), information gain is used as a metric to evaluate the quality of the tool's classification tests, including fingerprinting tests and their associated probes. Information gain is determined using the OS fingerprinting tool's signature database rather than raw training samples, including taking into account signatures/data that are represented by ranges of test values, disjunctive values, and missing values. Uniform distributions over test values and classifications are assumed in applying these methods to an example signature database for Nmap. Other assumptions or a priori information (e.g., normal distributions over ranges) can also be accommodated. The information gain measure provided can be applied to other classification problems as well.

Abstract: In a system for evaluating classification systems such as an operating system (OS) fingerprinting tool (e.g., Nmap), information gain is used as a metric to evaluate the quality of the tool's classification tests, including fingerprinting tests and their associated probes. Information gain is determined using the OS fingerprinting tool's signature database rather than raw training samples, including taking into account signatures/data that are represented by ranges of test values, disjunctive values, and missing values. Uniform distributions over test values and classifications are assumed in applying these methods to an example signature database for Nmap. Other assumptions or a priori information (e.g., normal distributions over ranges) can also be accommodated. The information gain measure provided can be applied to other classification problems as well.