Intrusion detection systems (IDSs) are tools designed to detect the evidence of computer intrusions. IDSs usually rely on models of attacks (called signatures) to identify the manifestation of intrusive behavior. The quality of these models is directly correlated to the system’s ability to identify all instances of a certain attack without making mistakes. Unfortunately, writing good signatures is hard, and, in the past, a number of evaluations pointed out the poor quality of signatures used in both open-source and commercial systems.

If the models used in intrusion detection were known, it would be possible to examine them to identify possible “blind spots” that could be exploited by an attacker to perform an attack while avoiding detection. Unfortunately, commercial systems do not provide access to the signatures they use to detect intrusions. Moreover, even in the cases when detection models are available, it is extremely time-consuming to devise testing procedures that analyze the models and identify blind spots.