The use of graph based anomaly detection has applications in a variety of diverse fields including health care, networks, finance, and insurance. Detecting anomalies using graphs has become important recently due to the interdependence of data from the web, emails, phone calls, etc. In this paper, we introduce a novel approach for graph-based anomaly detection by adding background knowledge to the evaluation metrics used in a traditional graph-mining approach, where we bias the substructure discovery process towards discovering anomalous substructures. Background knowledge is added in the form of rule coverage, which reports the percentage of the final graph covered by the instances of the substructure. Since one would expect that anomalies would be infrequent, it is our hypothesis that by assigning negative weights to the rule coverage, we can discover anomalous substructures. We are able to empirically evaluate that our proposed approach is comparable in accuracy to other approaches, and because the search space is reduced, do it in a fraction of the time. We test our approach on the wellknown KDD Cup 99 network intrusion dataset.